Back To: Syllabus-Mar#Mar28
Readings related to verification, validation sensitivity analysis.
The Seventh International Conference on the Simulation and Synthesis of Living Systems, August, 2000.
- Stylised Facts and the Contribution of Simulation to the Economic Analysis of Budgeting.
Bernd-O. Heine, Matthias Meyer and Oliver Strangfeld. JASSS 4(4).
- Grimm and Railsback. Chapter 3, Pattern Oriented Modeling
from their book Individual-based Modeling and Ecology.
- From Interesting Details to dynamical relevance: toward more effective use of empirical insights
in theory construction. Oswald J. Schmitz.
Oikos 94(1): 39-50 (2001).
- Errors and Artefacts in Agent-Based Modelling.
Galan etal (2009). [[Journal of Artificial Societies and Social Simulation vol. 12, no. 1 1
This one is cited a lot, so you should read it sometime (i don't like it!):
- Verification, Validation and Confirmation of Numerical Models in the Earth Sciences.
Oreskes, Naomi, Shrader-Frechette, Kristin and Belitz, Kenneth. Science 263 (5147): 641-646 (1994).
Note Well: This article uses "verification" in a way that most modellers call "validation,"
i.e., does the model "validly" model the part of the world it is meant model.
- A couple of oft-cited papers about the effects of synchronous vs asynchronous updating:
- Huberman, B. and Glance, N.. "Evolutionary games and computer simulations."
Proc. Natl. Acad. Sciences (PNAS USA) 90 (16): 7716-18 (1993)
- Scan this next one for their answer to Huberman and Glance):
Nowak, Bonfoeffer and May. "Spatial games and the maintenace of cooperation."
PNAS (USA) v91(11): pp4877-4881. 1994.
- "The poor quality of random numbers" and "Monte Carlo physics: a cautionary lesson."
Ferrenberg, Alan M., Landau, D.P., and Wong, Y.Joanna. Physical Review Letters 69(23). Dec 1992.
See also News and Views in Nature Vol 372, 1 Dec 1994.
- Jonathan Bart. "Acceptance Criteria for Using Individual-Based Models to Make Management Decisions."
Ecological Applications 5 (2): 411-420 MAY 1995. Go to browse this journal to find it.
- Edward J. Rykiel, Jr. "Testing Ecological Models: the meaning of validation."
In Ecological Modelling, 90 (3): 229-244 (1996). Go to and find the journal.
- Individuality in modelling: a simplifying assumption too far? David R. Cope. Nonlinear Analysis: Real World Applications 6(4): 691-704 (2005)
A nice tutorial:
- Stewart Robinson. "Simulation Verifcation, Validation and Confidence: A Tutorial."
Transactions of the Society for Computer Simulation International, 16 (No.2), pp. 63-69.
A good description and example applications of UA/SA methods:
A number of more technical papers, but you should definitely read these at some point
if you are going to do "real" computational modelling:
- Evaluating prediction uncertainty in simulation models. Michael D. McKay, John D. Morrison and
Stephen C. Upton. Computer Physics Communications 117(1-2): 44-51 (1999).
- Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems.
J. C. Helton and F. J. Davis. Reliability Engineering & System Safety 81(1): 23-69 (2003).
- Blower S, Dowlatabadi H, May R, Anderson R. Sensitivity and Uncertainty Analysis of complex models of
disease transmission: an HIV model, as an example. Int. Statist Rev. 1994;62:229-243.
- Helton, J.C., Johnson, J.D., Sallaberry, C.J. and Storlie, C.B. (2006).
Survey of sample-based methods for uncertainty and sensitivity analysis.
Reliability Engineering and System Safety 91(10-11): 1175-1209.
- Iman R, Helton J. An Investigation of Uncertainty and Sensitivity Analysis Techniques for
Computer Models. Risk Analysis 1980;8:71-90.
- Stochastic modelling of ecological processes using hybrid Gibbs samplers.
D.M.Walker etal. Ecological Modelling 198: 40-52 (2006).
- E.L.Ionides, C.Breto, A.A.King. Inference for Nonlinear dynamical systems.
PNAS 103(49): 18438-43 (Dec 5 2006).
- Selecting Parameters for calibration via sensitivity analysis: An inidividual-based model
of mosquitofish population dynamics. Beaudouin etal (2008). Ecological Modeling 218: 29-48.
Back To: Syllabus-Mar#Mar28