Abstract

In this paper, we present a predictive and generative design approach for supporting the conceptual design of product shapes in 3D meshes. We develop a target-embedding variational autoencoder (TEVAE) neural network architecture, which consists of two modules: (1) a training module with two encoders and one decoder (E2D network) and (2) an application module performing the generative design of new 3D shapes and the prediction of a 3D shape from its silhouette. We demonstrate the utility and effectiveness of the proposed approach in the design of 3D car body and mugs. The results show that our approach can generate a large number of novel 3D shapes and successfully predict a 3D shape based on a single silhouette sketch. The resulting 3D shapes are watertight polygon meshes with high-quality surface details, which have better visualization than voxels and point clouds, and are ready for downstream engineering evaluation (e.g., drag coefficient) and prototyping (e.g., 3D printing).

References

1.
Pratt
,
M. J.
,
Anderson
,
B. D.
, and
Ranger
,
T.
,
2005
, “
Towards the Standardized Exchange of Parameterized Feature-Based CAD Models
,”
Comput Aided Des.
,
37
(
12
), pp.
1251
1265
.
2.
Reid
,
T. N.
,
Gonzalez
,
R. D.
, and
Papalambros
,
P. Y.
,
2010
, “
Quantification of Perceived Environmental Friendliness for Vehicle Silhouette Design
,”
ASME J. Mech. Des.
,
132
(
10
), p.
101010
.
3.
Gunpinar
,
E.
,
Ovur
,
S. E.
, and
Gunpinar
,
S.
,
2019
, “
A User-Centered Side Silhouette Generation System for Sedan Cars Based on Shape Templates
,”
Optim. Eng.
,
20
(
3
), pp.
683
723
.
4.
Schmidt
,
R.
,
Khan
,
A.
,
Kurtenbach
,
G.
, and
Singh
,
K.
,
2009
, “
On Expert Performance in 3d Curve-Drawing Tasks
,”
Proceedings of the 6th Euro Graphics Symposium on Sketch-Based Interfaces and Modeling
,
New Orleans, LA
,
Aug. 1–2
, pp.
133
140
.
5.
Jarrett
,
D.
, and
van der Schaar
,
M.
,
2019
, “
Target-Embedding Autoencoders for Supervised Representation Learning
,”
International Conference on Learning Representations
,
New Orleans, LA
,
May 6–9
.
6.
Girdhar
,
R.
,
Fouhey
,
D. F.
,
Rodriguez
,
M.
, and
Gupta
,
A.
,
2016
, “
Learning a Predictable and Generative Vector Representation for Objects
,”
European Conference on Computer Vision
,
Amsterdam, The Netherlands
,
Oct. 11–14
, Springer, pp.
484
499
.
7.
Yuan
,
Y.-J.
,
Lai
,
Y.-K.
,
Yang
,
J.
,
Duan
,
Q.
,
Fu
,
H.
, and
Gao
,
L.
,
2020
, “
Mesh Variational Autoencoders With Edge Contraction Pooling
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops
,
Virtual
,
June 16–18
, pp.
274
275
.
8.
Lun
,
Z.
,
Gadelha
,
M.
,
Kalogerakis
,
E.
,
Maji
,
S.
, and
Wang
,
R.
,
2017
, “
3d Shape Reconstruction From Sketches Via Multi-View Convolutional Networks
,”
2017 International Conference on 3D Vision (3DV)
,
Qingdao, China
,
Oct. 10–12
, IEEE, pp.
67
77
.
9.
Jin
,
A.
,
Fu
,
Q.
, and
Deng
,
Z.
,
2020
, “
Contour-Based 3d Modeling Through Joint Embedding of Shapes and Contours
,”
Symposium on Interactive 3D Graphics and Games
,
San Francisco, CA
,
Sept. 15–17
, pp.
1
10
.
10.
Nozawa
,
N.
,
Shum
,
H. P.
,
Feng
,
Q.
,
Ho
,
E. S.
, and
Morishima
,
S.
,
2021
, “
3D Car Shape Reconstruction From a Contour Sketch Using GAN and Lazy Learning
,”
Vis. Comput.
,
38
(
4
), pp.
1317
1330
.
11.
Guillard
,
B.
,
Remelli
,
E.
,
Yvernay
,
P.
, and
Fua
,
P.
,
2021
, “
Sketch2mesh: Reconstructing and Editing 3D Shapes From Sketches
,”
Proceedings of the IEEE/CVF International Conference on Computer Vision
,
Virtual
,
June 19–25
.
12.
Xiang
,
N.
,
Wang
,
R.
,
Jiang
,
T.
,
Wang
,
L.
,
Li
,
Y.
,
Yang
,
X.
, and
Zhang
,
J.
,
2020
, “
Sketch-Based Modeling With a Differentiable Renderer
,”
Comput. Animat. Virtual Worlds
,
31
(
4–5
), p.
e1939
.
13.
Nozawa
,
N.
,
Shum
,
H. P.
,
Ho
,
E. S.
, and
Morishima
,
S.
,
2020
, “
Single Sketch Image Based 3d Car Shape Reconstruction With Deep Learning and Lazy Learning
,”
Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications
,
Valetta, Malta
,
Feb. 27–29
, pp.
179
190
.
14.
Han
,
X.
,
Gao
,
C.
, and
Yu
,
Y.
,
2017
, “
Deepsketch2face: A Deep Learning Based Sketching System for 3d Face and Caricature Modeling
,”
ACM Trans. Graph.
,
36
(
4
), pp.
1
12
.
15.
Goodfellow
,
I.
,
Pouget-Abadie
,
J.
,
Mirza
,
M.
,
Xu
,
B.
,
Warde-Farley
,
D.
,
Ozair
,
S.
,
Courville
,
A.
, and
Bengio
,
Y.
,
2014
, “
Generative Adversarial Nets
,”
Advances in Neural Information Processing Systems
,
Palais des Congrès de Montréal, Montréal, Canada
,
Dec. 8–13
, pp.
2672
2680
.
16.
Kingma
,
D. P.
, and
Welling
,
M.
,
2013
, “
Auto-Encoding Variational Bayes
,” preprint arXiv:1312.6114.
17.
Oh
,
S.
,
Jung
,
Y.
,
Kim
,
S.
,
Lee
,
I.
, and
Kang
,
N.
,
2019
, “
Deep Generative Design: Integration of Topology Optimization and Generative Models
,”
ASME J. Mech. Des.
,
141
(
11
), p.
111405
.
18.
Dering
,
M.
,
Cunningham
,
J.
,
Desai
,
R.
,
Yukish
,
M. A.
,
Simpson
,
T. W.
, and
Tucker
,
C. S.
,
2018
, “
A Physics-Based Virtual Environment for Enhancing the Quality of Deep Generative Designs
,”
ASME 2018 International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Quebec City, Quebec, Canada
,
Aug. 26–29
.
19.
Fujita
,
K.
,
Minowa
,
K.
,
Nomaguchi
,
Y.
,
Yamasaki
,
S.
, and
Yaji
,
K.
,
2021
, “
Design Concept Generation With Variational Deep Embedding Over Comprehensive Optimization
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Virtual
,
Aug. 17–19
, p. V03BT03A038.
20.
Shu
,
D.
,
Cunningham
,
J.
,
Stump
,
G.
,
Miller
,
S. W.
,
Yukish
,
M. A.
,
Simpson
,
T. W.
, and
Tucker
,
C. S.
,
2020
, “
3D Design Using Generative Adversarial Networks and Physics-Based Validation
,”
ASME J. Mech. Des.
,
142
(
7
), p.
071701
.
21.
Zhang
,
W.
,
Yang
,
Z.
,
Jiang
,
H.
,
Nigam
,
S.
,
Yamakawa
,
S.
,
Furuhata
,
T.
,
Shimada
,
K.
, and
Kara
,
L. B.
,
2019
, “
3d Shape Synthesis for Conceptual Design and Optimization Using Variational Autoencoders
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Anaheim, CA
,
Aug. 18–21
, p. V02AT03A017.
22.
Yoo
,
S.
,
Lee
,
S.
,
Kim
,
S.
,
Hwang
,
K. H.
,
Park
,
J. H.
, and
Kang
,
N.
,
2021
, “
Integrating Deep Learning Into CAD/CAE System: Case Study on Road Wheel Design Automation
,”
Struct. Multidis. Opt.
,
64
(
4
), pp.
2725
2747
.
23.
Mostajabi
,
M.
,
Maire
,
M.
, and
Shakhnarovich
,
G.
,
2018
, “
Regularizing Deep Networks by Modeling and Predicting Label Structure
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 19–21
, pp.
5629
5638
.
24.
Dalca
,
A. V.
,
Guttag
,
J.
, and
Sabuncu
,
M. R.
,
2018
, “
Anatomical Priors in Convolutional Networks for Unsupervised Biomedical Segmentation
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 19–21
, pp.
9290
9299
.
25.
Umetani
,
N.
,
2017
, “
Exploring Generative 3d Shapes Using Autoencoder Networks
,”
SIGGRAPH Asia 2017 Technical Briefs
,
Bangkok, Thailand
,
Nov. 27–30
, pp.
1
4
.
26.
Gao
,
L.
,
Yang
,
J.
,
Wu
,
T.
,
Yuan
,
Y.-J.
,
Fu
,
H.
,
Lai
,
Y. K.
, and
Zhang
,
H.
,
2019
, “
Sdm-net: Deep Generative Network for Structured Deformable Mesh
,”
ACM Trans. Graph.
,
38
(
6
), pp.
1
15
.
27.
Zollhöfer
,
M.
,
Nießner
,
M.
,
Izadi
,
S.
,
Rehmann
,
C.
,
Zach
,
C.
,
Fisher
,
M.
,
Wu
,
C.
,
Fitzgibbon
,
A.
,
Loop
,
C.
, and
Theobalt
,
C.
,
2014
, “
Real-Time Non-Rigid Reconstruction Using an RGB-D Camera
,”
ACM Trans. Graph.
,
33
(
4
), pp.
1
12
.
28.
Gao
,
L.
,
Lai
,
Y.-K.
,
Yang
,
J.
,
Ling-Xiao
,
Z.
,
Xia
,
S.
, and
Kobbelt
,
L.
,
2019
, “
Sparse Data Driven Mesh Deformation
,”
IEEE Trans. Visual. Comput. Graph.
,
27
(
3
), pp.
2085
2100
.
29.
Yang
,
M. C.
,
2003
, “
Concept Generation and Sketching: Correlations With Design Outcome
,”
International Design Engineering Technical Conferences and Computers and Information in Engineering Conference
,
Chicago, IL
,
Sept. 2–6
, pp.
829
834
.
30.
Li
,
X.
,
Xie
,
C.
, and
Sha
,
Z.
,
2021
, “
Part-Aware Product Design Agent Using Deep Generative Network and Local Linear Embedding
,”
Proceedings of the 54th Hawaii International Conference on System Sciences
,
Kauai, HI
,
Jan. 5–8
, p.
5250
.
31.
Paschalidou
,
D.
,
Katharopoulos
,
A.
,
Geiger
,
A.
, and
Fidler
,
S.
,
2021
, “
Neural Parts: Learning Expressive 3d Shape Abstractions With Invertible Neural Networks
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Virtual
,
June 19–25
, pp.
3204
3215
.
32.
Vasu
,
S.
,
Talabot
,
N.
,
Lukoianov
,
A.
,
Baque
,
P.
,
Donier
,
J.
, and
Fua
,
P.
,
2021
, “
Hybridsdf: Combining Free Form Shapes and Geometric Primitives for Effective Shape Manipulation
,” preprint arXiv:2109.10767.
33.
Jones
,
R. K.
,
Hanocka
,
R.
, and
Ritchie
,
D.
,
2021
, “
The Neutrally-Guided Shape Parser: A Monte Carlo Method for Hierarchical Labeling of Over-Segmented 3D Shapes
,” preprint arXiv:2106.12026.
34.
Groueix
,
T.
,
Fisher
,
M.
,
Kim
,
V. G.
,
Russell
,
B. C.
, and
Aubry
,
M.
,
2018
, “
A Papier-Mâché Approach to Learning 3d Surface Generation
,”
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition
,
Salt Lake City, UT
,
June 19–21
, pp.
216
224
.
35.
Chen
,
Z.
, and
Zhang
,
H.
,
2019
, “
Learning Implicit Fields for Generative Shape Modeling
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Long Beach, CA
,
June 16–20
, pp.
5939
5948
.
36.
Park
,
J. J.
,
Florence
,
P.
,
Straub
,
J.
,
Newcombe
,
R.
, and
Lovegrove
,
S.
,
2019
, “
Deepsdf: Learning Continuous Signed Distance Functions for Shape Representation
,”
Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
,
Long Beach, CA
,
June 16–20
, pp.
165
174
.
You do not currently have access to this content.