A variety of metamodeling techniques have been developed in the past decade to reduce the computational expense of computer-based analysis and simulation codes. Metamodeling is the process of building a “model of a model” to provide a fast surrogate for a computationally expensive computer code. Common metamodeling techniques include response surface methodology, kriging, radial basis functions, and multivariate adaptive regression splines. In this paper, we investigate support vector regression (SVR) as an alternative technique for approximating complex engineering analyses. The computationally efficient theory behind SVR is reviewed, and SVR approximations are compared against the aforementioned four metamodeling techniques using a test bed of 26 engineering analysis functions. SVR achieves more accurate and more robust function approximations than the four metamodeling techniques, and shows great potential for metamodeling applications, adding to the growing body of promising empirical performance of SVR.

1.
Kleijnen
,
J. P. C.
, 1987,
Statistical Tools for Simulation Practitioners
,
Marcel Dekker
, New York.
2.
Jin
,
R.
,
Chen
,
W.
, and
Simpson
,
T. W.
, 2001, “
Comparative Studies of Metamodeling Techniques Under Multiple Modeling Criteria
,”
Struct. Multidiscip. Optim.
1615-147X,
23
(
1
), pp.
1
13
.
3.
Haykin
,
S.
, 1999,
Neural Networks: A Comprehensive Foundation
, 2nd Edition,
Prentice Hall
, Upper Saddle River, NJ.
4.
Cristianni
,
N.
, and
Shawe-Taylor
,
J.
, 2000,
An Introduction to Support Vector Machines and Other Kernel-Based Learning Methods
,
Cambridge University Press
, Cambridge, UK.
5.
Hearst
,
M. A.
, 1998, “
Trends Controversies: Support Vector Machines
,”
IEEE Intell. Syst.
1094-7167,
13
(
4
), pp.
18
28
.
6.
Takeuchi
,
K.
, and
Collier
,
N.
, 2002, “
Use of Support Vector Machines in Extended Named Entity
,”
Proc. of Sixth Conference on Natural Language Learning (CoNLL-2002)
,
D.
Roth
and
A.
van den Bosch
, eds., Taipei,
Taiwan
,
Association for Computational Linguistics
, New Brunswick, NJ, pp.
119
125
.
7.
Dumais
,
S. T.
,
Platt
,
J.
,
Heckerman
,
D.
, and
Saharni
,
M.
, 1998, “
Inductive Learning Algorithms and Representations for Text Categorization
,”
Proc. of 7th Int. Conference on Information and Knowledge Management
,
Bethesda
, MD, ACM, New York, pp.
148
155
.
8.
Prakasvudhisarn
,
C.
,
Trafalis
,
T. B.
, and
Raman
,
S.
, 2003, “
Support Vector Regression for Determination of Minimum Zone
,”
ASME J. Manuf. Sci. Eng.
1087-1357,
125
(
4
), pp.
736
739
.
9.
Vapnik
,
V.
, and
Lerner
,
A.
, 1963, “
Pattern Recognition Using Generalized Portrait Method
,”
Autom. Remote Control (Engl. Transl.)
0005-1179,
24
(
6
), pp.
774
780
.
10.
Vapnik
,
V.
, 1995,
The Nature of Statistical Learning Theory
,
Springer
, New York.
11.
Smola
,
A. J.
, and
Schölkopf
,
B.
, 1998, “
A Tutorial on Support Vector Regression
,”
NeuroCOLT2 Technical Report Series, NC2-TR-1998-030
,
Berlin
, Germany.
12.
Vapnik
,
V.
,
Golowich
,
S.
, and
Smola
,
A.
, 1997, “
Support Vector Method for Function Approximation, Regression Estimation, and Signal Processing
,”
Advances in Neural Information Processing Systems
,
M.
Mozer
,
M.
Jordan
, and
T.
Petsche
, eds.,
MIT Press
, Cambridge, MA, pp.
281
287
.
13.
Gunn
,
S. R.
, 1997, “
Support Vector Machines for Classification and Regression
,” Technical Report, Image Speech and Intelligent Systems Research Group,
University of Southampton
, UK.
14.
Simpson
,
T. W.
,
Peplinski
,
J.
,
Koch
,
P. N.
, and
Allen
,
J. K.
, 2001, “
Metamodels for Computer-Based Engineering Design: Survey and Recommendations
,”
Eng. Comput.
0177-0667,
17
(
2
), pp.
129
150
.
15.
Myers
,
R. H.
, and
Montgomery
,
D. C.
, 1995,
Response Surface Methodology: Process and Product Optimization Using Designed Experiments
,
Wiley
, New York.
16.
Cappelleri
,
D. J.
,
Frecker
,
M. I.
,
Simpson
,
T. W.
, and
Snyder
,
A.
, 2002, “
A Metamodel-Based Approach for Optimal Design of a PZT Bimorph Actuator for Minimally Invasive Surgery
,”
J. Mech. Des.
1050-0472,
124
(
2
), pp.
354
357
.
17.
Chen
,
W.
,
Allen
,
J. K.
,
Tsui
,
K.-L.
, and
Mistree
,
F.
, 1996, “
A Procedure for Robust Design: Minimizing Variations Caused by Noise and Control Factors
,”
J. Mech. Des.
1050-0472,
118
(
4
), pp.
478
485
.
18.
Korngold
,
J. C.
, and
Gabriele
,
G. A.
, 1997, “
Multidisciplinary Analysis and Optimization of Discrete Problems Using Response Surface Methods
,”
J. Mech. Des.
1050-0472,
119
(
4
), pp.
427
433
.
19.
Wang
,
G.
, 2003, “
Adaptive Response Surface Method Using Inherited Latin Hypercube Designs
,”
J. Mech. Des.
1050-0472,
125
(
2
), pp.
210
220
.
20.
Hernandez
,
G.
,
Simpson
,
T. W.
,
Allen
,
J. K.
,
Bascaran
,
E.
,
Avila
,
L. F.
, and
Salinas
,
F.
, 2001, “
Robust Design of Families of Products With Production Modeling and Evaluation
,”
J. Mech. Des.
1050-0472,
123
(
2
), pp.
183
190
.
21.
Lewis
,
K.
, and
Mistree
,
F.
, 1998, “
Collaborative, Sequential, and Isolated Decisions in Design
,”
J. Mech. Des.
1050-0472,
120
(
4
), pp.
643
652
.
22.
Dyn
,
N.
,
Levin
,
D.
, and
Rippa
,
S.
, 1986, “
Numerical Procedures for Surface Fitting of Scattered Data by Radial Basis Functions
,”
SIAM (Soc. Ind. Appl. Math.) J. Sci. Stat. Comput.
0196-5204,
7
(
2
), pp.
639
659
.
23.
Powell
,
M. J. D.
, 1987, “
Radial Basis Functions for Multivariable Interpolation: A Review
,”
Algorithms for Approximation
,
J. C.
Mason
and
M. G.
Cox
, eds.,
Oxford University Press
, London, pp.
143
167
.
24.
Tu
,
C.
, and
Barton
,
R. R.
, 1997, “
Production Yield Estimation by the Metamodel Method with a Boundary-Focused Experiment Design
,”
ASME Design Engineering Technical Conferences-Design Theory and Methodology
,
Sacramento
, CA, ASME, Paper No. DETC97/DTM-3870.
25.
Meckesheimer
,
M.
,
Barton
,
R. R.
,
Simpson
,
T. W.
,
Limayem
,
F.
, and
Yannou
,
B.
, 2001, “
Metamodeling of Combined Discrete/Continuous Responses
,”
AIAA J.
0001-1452,
39
(
10
), pp.
1955
1959
.
26.
Sacks
,
J.
,
Welch
,
W. J.
,
Mitchell
,
T. J.
, and
Wynn
,
H. P.
, 1989, “
Design and Analysis of Computer Experiments
,”
Stat. Sci.
0883-4237,
4
(
4
), pp.
409
435
.
27.
Koehler
,
J. R.
, and
Owen
,
A. B.
, 1996, “
Computer Experiments
,”
Handbook of Statistics
,
S.
Ghosh
and
C. R.
Rao
, eds.,
Elsevier Science
, New York, pp.
261
308
.
28.
Simpson
,
T. W.
,
Mauery
,
T. M.
,
Korte
,
J. J.
, and
Mistree
,
F.
, 2001, “
Kriging Metamodels for Global Approximation in Simulation-Based Multidisciplinary Design Optimization
,”
AIAA J.
0001-1452,
39
(
12
), pp.
2233
2241
.
29.
Pacheco
,
J. E.
,
Amon
,
C. H.
, and
Finger
,
S.
, 2003, “
Bayesian Surrogates Applied to Conceptual Stages of the Engineering Design Process
,”
J. Mech. Des.
1050-0472,
125
(
4
), pp.
664
672
.
30.
Friedman
,
J. H.
, 1991, “
Multivariate Adaptive Regression Splines
,”
Ann. Stat.
0090-5364,
19
(
1
), pp.
1
67
.
31.
Chen
,
V. C. P.
,
Ruppert
,
D.
, and
Shoemaker
,
C. A.
, 1999, “
Applying Experimental Design and Regression Splines to High-Dimensional Continuous-State Stochastic Dynamic Programming
,”
Oper. Res.
0030-364X,
47
, pp.
38
53
.
32.
Chen
,
V. C. P.
, 1999, “
Application of MARS and Orthogonal Arrays to Inventory Forecasting Stochastic Dynamic Programs
,”
Comput. Stat. Data Anal.
0167-9473,
30
, pp.
317
341
.
33.
Smola
,
A. J.
,
Schölkopf
,
B.
, and
Müller
,
K. R.
, 1998, “
The Connection Between Regularization Operators and Support Vector Kernels
,”
Neural Networks
0893-6080,
11
(
4
), pp.
637
649
.
34.
Schölkopf
,
B.
, and
Smola
,
A. J.
, 2002,
Learning with Kernels: Support Vector Machines, Regularization, Optimization, and Beyond
,
MIT Press
, Cambridge, MA.
35.
Su
,
J.
, and
Renaud
,
J. E.
, 1997, “
Automatic Differentiation in Robust Optimization
,”
AIAA J.
0001-1452,
35
(
6
), pp.
1072
1079
.
36.
Markowetz
,
F.
, 2001, “
Support Vector Machines in Bioinformatics
,” Diploma Thesis in Mathematics, University of Heidelberg, Germany.
37.
Simpson
,
T. W.
, 1998, “
A Concept Exploration Method for Product Family Design
,” Ph.D. Dissertation, G.W. Woodruff School of Mechanical Engineering, Georgia Institute of Technology, Atlanta, GA.
38.
Schmit
,
L. A.
, 1981, “
Structural Synthesis—Its Genesis and Development
,”
AIAA J.
0001-1452,
19
(
10
), pp.
1249
1263
.
39.
Arora
,
J. S.
, 1989,
Introduction to Optimum Design
,
McGraw-Hill
, New York.
40.
Sandgren
,
E.
, 1990, “
Nonlinear Integer and Discrete Programming in Mechanical Design Optimization
,”
J. Mech. Des.
1050-0472,
112
(
2
), pp.
223
229
.
41.
Ragsdell
,
K. M.
, and
Phillips
,
D. T.
, 1976, “
Optimal Design of a Class of Welded Structures Using Geometric Programming
,”
ASME J. Eng. Ind.
0022-0817,
98
(
3
), pp.
1021
1025
.
42.
Simpson
,
T. W.
,
Lin
,
D. K. J.
, and
Chen
,
W.
, 2001, “
Sampling Strategies for Computer Experiments: Design and Analysis
,”
Int. J. Reliab. Appl.
1598-0073,
2
(
3
), pp.
209
240
.
43.
McKay
,
M. D.
,
Beckman
,
R. J.
, and
Conover
,
W. J.
, 1979, “
A Comparison of Three Methods for Selecting Values of Input Variables in the Analysis of Output From a Computer Code
,”
Technometrics
0040-1706,
21
(
2
), pp.
239
245
.
44.
Kalagnanam
,
J. R.
, and
Diwekar
,
U. M.
, 1997, “
An Efficient Sampling Technique for Off-Line Quality Control
,”
Technometrics
0040-1706,
39
(
3
), pp.
308
319
.
45.
Tang
,
B.
, 1993, “
Orthogonal Array-Based Latin Hypercubes
,”
J. Am. Stat. Assoc.
0162-1459,
88
(
424
), pp.
1392
1397
.
46.
Fang
,
K.-T.
,
Lin
,
D. K. J.
,
Winker
,
P.
, and
Zhang
,
Y.
, 2000, “
Uniform Design: Theory and Application
,”
Technometrics
0040-1706,
42
, pp.
237
248
.
47.
Wang
,
W. J.
,
Xu
,
Z. B.
,
Lu
,
W. Z.
, and
Zhang
,
X. Y.
, 2003, “
Determination of the Spread Parameter in the Gaussian Kernel for Classification and Regression
,”
Neurocomputing
0925-2312,
55
(
1
), pp.
643
663
.
48.
Cameron
,
A. C.
, and
Windmeijer
,
F. A. G.
, 1997, “
An R-Squared Measure of Goodness of Fit for Some Common Nonlinear Regression Models
,”
J. Econometr.
0304-4076,
77
(
2
), pp.
329
342
.
49.
Martin
,
J. D.
, and
Simpson
,
T. W.
, 2004, “
On the Use of Kriging Models to Approximate Deterministic Computer Models
,”
ASME Design Engineering Technical Conferences-Design Automation Conference
,
Salt Lake City
,
W.
Chen
, ed.,
ASME
, New York, ASME Paper No. DETC2004/DAC-57300.
You do not currently have access to this content.