Engineering Institute

About Model Verification and Validation

Summary of Model V&V for Damage Prognosis

Because damage prognosis solutions rely on the deployment of a predictive capability, the credibility of numerical simulations must be established. This is accomplished through various activities collectively referred to as Verification and Validation (V&V). A hierarchy of three validation experiments is defined to validate various aspects of the modeling of multi-layered composite plates. The experiments are modal testing, quasi-static loading, and impact testing. Modal testing is used to extract low-frequency resonant modes, from which the linear response can be validated. Quasi-static loading is used to validate the large displacement response, although without fully exercising the fiber splitting and delamination damage modes nor the Cohesive Zone Model algorithm. Impact testing is used to validate the modeling of the high-frequency, transient wave propagation coupled with all aspects (constitutive and damage models, numerical algorithms) of the composite damage models.
The discussion presented here is restricted to the validation of the modal response of eight-ply laminated composite plates. Other aspects of the modeling are currently being validated, pending the completion of the quasi-static and impact experiments. After verifying some implementation aspects of the code, mesh convergence studies are conducted. Effect screening is performed to restrict the varying input parameters to the most significant ones. Polynomial meta-models are developed to replace the potentially expensive finite element simulations. Uncertainty is propagated to estimate the variability of predictions given input uncertainty. Test measurements are compared to predictions of modal frequencies. A final statement is made about the predictive accuracy of the composite model and the level of confidence with which modal frequency predictions can be made for potentially different multi-layered configurations.

Modal Testing of Eight-ply Laminate Composite Plates

The modal response of a population of eight nominally identical composite plates is tested. Replicate experiments are needed to estimate the variability of the modal response due to manufacturing and testing uncertainties. Each plate is 152.0 mm (6.0 inch) square, 1.0 mm (0.04 inch) thick, and made of eight orthotropic carbon fiber plies. Each ply is 0.127 mm (0.005 inch) thick. The ply orientation angles are [0; 45; 90; -45; -45; 90; 45; 0] degrees. The first five modes are identified with mean frequency values of 107.4, 191.8, 274.1, 315.3, and 398.9 Hertz. It is observed that the standard deviations of identified frequencies are less than 1¼% of mean values, which indicates that the measurements are very repeatable.

Finite Element Modeling and Computer Code Verification

The Finite Element Model (FEM) developed to simulate the modal response of the plates is meshed with 20-node quadratic hexahedra, with ten elements in each in-plane direction and one element through the thickness of each ply. The nominal orthotropic material properties are provided by the manufacturer and confirmed through coupon testing. Using the nominal material properties, natural frequencies are predicted at 107.5, 205.7, 278.1, 334.0, and 411.8 Hertz, which provides a good qualitative agreement with the identified frequencies.
The FEM is implemented and analyzed using the general-purpose finite element package HKS/ABAQUSTM [1]. Computer code verification is the first step of any V&V activity. It verifies that the code is error-free by comparing analytical, closed-form solutions to numerical results. It is verified that the predicted frequencies for a square, free-free, isotropic plate are less than 0.5% different from the closed-form solution [2]. Similar results are obtained for a square, simply supported, orthotropic plate [3]. The conclusion is that the FEM software package and composite material module solve the equations correctly for this problem.

Convergence and Calculation Verification

Calculation verification assesses the convergence of the numerical solution for the application of interest. It requires several calculations using successively refined mesh sizes and/or time steps, from which the order of convergence is estimated and the true-but-unknown solution is extrapolated. The Richardson extrapolation and Grid Convergence Index (GCI) are brought to bear to verify that the mesh size yields converged frequencies [4].
The average order of convergence for frequencies 1-5 is found equal to 1.8, close to the value of two that should be obtained because quadratic elements are used. Small GCI values (less than 1%) also suggest asymptotic convergence. Errors obtained between predictions and the Richardson estimation of the true-but-unknown solutions are less than 1%. These results indicate that the FEM and its spatial discretization yield converged natural frequencies.

Effect Screening and Parameter Down-selection

We seek next to better understand the relationship between the variability of input parameters, such as the ply orientation angles and material properties, and response feature variability. Two designs of computer experiments are performed to explore the values of frequencies for various combinations of input parameters [5]. The most significant effects are screened using methods such as the analysis-of-variance that performs multiple regression analyses and estimates the correlation between input effects and output features [6].
The analysis demonstrates that the variability of the five frequencies is controlled for the most part by four ply orientation angles, two material coefficients, and the accelerometer mass. Because the other factors do not produce significant frequency variability, they are kept constant and equal to their nominal values. The number of input parameters is hence reduced from fifteen to seven only, which greatly simplifies meta-modeling and the forward and backward propagations of uncertainty.

Meta-Modeling of the Finite Element Simulations

Based on screening results, meta-models are developed to replace the FEM simulations. Polynomial meta-models provide fast-running surrogates that accurately capture the relationship between input parameters and natural frequencies without including details of the geometry or material modeling.
The data needed to train meta-models are provided by a design of computer experiments. Confirmatory screening and model fitting are performed using a Bayesian effect screening method [7]. It provides a population of meta-models that can be characterized statistically by establishing distributions of polynomial coefficients. Model fitting errors are assessed by segregating the available data into training and validation sets, and found to be in the order of 1%. The meta-modeling study concludes that the FEM simulations can be replaced by polynomial meta-models with little loss of accuracy.

Propagation of Uncertainty from Inputs to Outputs

Variability of the ply orientation angles and material properties is propagated through the family of meta-models to estimate the natural frequency variability. Model fitting uncertainty is also accounted for by sampling the coefficients of the meta-model polynomials. A Monte Carlo simulation based on 10+6 runs yields a population of frequencies from which statistics can be extracted. The effects of input distribution and correlation structure on output statistics are also investigated [8]. The statistics of frequency values are 107.5 +/- 6.7, 206.6 +/- 15.8, 281.2 +/- 19.8, 329.0 +/- 24.8, and 408.0 +/- 27.9 Hertz (mean +/- standard deviation).

Final Assessment of Prediction Accuracy

The sources of uncertainty originating from experimental variability, coding mistakes, calculation verification, model fitting, and parametric variability are summarized in Table 1. For simplicity, statistics are averaged for the five frequencies. Aggregating the different sources of uncertainty leads to a total prediction error (mean +/- standard deviation) of 3.64 +/- 19.02 Hertz.

Table 1. Summary of the quantification of testing and modeling uncertainty.

Source of Uncertainty	Statistics
	Mean (μ)	Standard Deviation (s)
Code Error, Solution Convergence	μ1 = 1.49 Hertz	s1 = 0.87 Hertz
Meta-model Fitting Error	μ2 = 2.15 Hertz	s2 = 0.18 Hertz
Model Parameter Variability	μ3 = 0.00 Hertz	s3 = 19.00 Hertz
Experimental Variability	μ4 = 0.00 Hertz	s4 = 2.49 Hertz
Test-analysis Correlation	μC = 8.97 Hertz	sC = 19.18 Hertz

Direct comparison between experimental measurements and numerical predictions leads to a prediction error of 8.97 +/- 19.18 Hertz. The discrepancy between the two error models can be attributed to the residual modeling error, that is, the error made when the composite model is substituted to reality.
The conclusion is two-fold. First, better characterization of the composite lay-up and material properties would further constraint the variability of input parameters which, in turn, could reduce some of the residual prediction error. Reference [8] also shows that calibration can be taken advantage of to infer the values of input parameters that cannot be obtained through direct measurement or coupon testing. Second, we are confident that natural frequencies for different multi-layered configurations can be predicted with a similar degree of accuracy as long as the same modeling rules are followed.

References

[1] Abaqus™/Explicit, User’s Manual, Version 6.2, Hibbitt, Karlsson & Sorensen, Pawtucket, Rhode Island, 2001.

[2] Leissa, A.W., “The Free Vibration of Rectangular Plates,” Journal of Sound and Vibration, Vol. 31, No. 3, 1973, pp. 257-293.

[3] Srinivas, S., Rao, A.K., “Bending, Vibration and Buckling of Simply Supported Thick Orthotropic Rectangular Plates and Laminates,” International Journal of Solids and Structures, Vol. 6, 1970, pp. 1463-1481.

[4] Hemez, F.M., Doebling, S.W., Anderson, M.C., “A Brief Tutorial on Verification and Validation,” 22nd SEM International Modal Analysis Conference, Dearborn, Michigan, January 26-29, 2004. Available as unlimited, public release LA-UR-03-8491 of the Los Alamos National Laboratory, Los Alamos, New Mexico.

[5] Myers, R.H., Montgomery, D.C., Response Surface Methodology: Process and Product Optimization Using Designed Experiments, Wiley Inter-science, New York, 1995.

[6] Saltelli, A., Chan, K., Scott, M., Editors, Sensitivity Analysis, John Wiley & Sons, 2000.

[7] Kerschen, G., Golinval, J.C., Hemez, F.M., “Bayesian Model Screening for the Identification of Non-linear Mechanical Structures,” ASME Journal of Vibration and Acoustics, Vol. 125, July 2003, pp. 389-397.

[8] Hemez, F.M., Tippetts, “Verification and Validation of a Composite Model,” 23rd SEM International Modal Analysis Conference, Orlando, Florida, January 31-February 3, 2005. Available as unlimited, public release LA-UR-04-8195 of the Los Alamos National Laboratory, Los Alamos, New Mexico.