Recent Changes - Search:

PmWiki

edit SideBar

VerificationAndValidationReadings

Back To: Syllabus-Mar#Mar28


Readings related to verification, validation sensitivity analysis.

  The Seventh International Conference on the Simulation and Synthesis of Living Systems, August, 2000.

Highly recommended:

Optional Readings:

  • Stylised Facts and the Contribution of Simulation to the Economic Analysis of Budgeting.
    Bernd-O. Heine, Matthias Meyer and Oliver Strangfeld. JASSS 4(4).
  • Grimm and Railsback. Chapter 3, Pattern Oriented Modeling
    from their book Individual-based Modeling and Ecology.
  • From Interesting Details to dynamical relevance: toward more effective use of empirical insights in theory construction. Oswald J. Schmitz. Oikos 94(1): 39-50 (2001).
  • Errors and Artefacts in Agent-Based Modelling.
    Galan etal (2009). [[Journal of Artificial Societies and Social Simulation vol. 12, no. 1 1

Required Readings:

This one is cited a lot, so you should read it sometime (i don't like it!):

  • Verification, Validation and Confirmation of Numerical Models in the Earth Sciences.
    Oreskes, Naomi, Shrader-Frechette, Kristin and Belitz, Kenneth. Science 263 (5147): 641-646 (1994).
    Note Well: This article uses "verification" in a way that most modellers call "validation," i.e., does the model "validly" model the part of the world it is meant model.

Optional Readings:

  • A couple of oft-cited papers about the effects of synchronous vs asynchronous updating:
    • Huberman, B. and Glance, N.. "Evolutionary games and computer simulations."
      Proc. Natl. Acad. Sciences (PNAS USA) 90 (16): 7716-18 (1993)
    • Scan this next one for their answer to Huberman and Glance): Nowak, Bonfoeffer and May. "Spatial games and the maintenace of cooperation." PNAS (USA) v91(11): pp4877-4881. 1994.
  • "The poor quality of random numbers" and "Monte Carlo physics: a cautionary lesson."
    Ferrenberg, Alan M., Landau, D.P., and Wong, Y.Joanna. Physical Review Letters 69(23). Dec 1992.
    See also News and Views in Nature Vol 372, 1 Dec 1994.
  • Jonathan Bart. "Acceptance Criteria for Using Individual-Based Models to Make Management Decisions." Ecological Applications 5 (2): 411-420 MAY 1995. Go to browse this journal to find it.
  • Edward J. Rykiel, Jr. "Testing Ecological Models: the meaning of validation." In Ecological Modelling, 90 (3): 229-244 (1996). Go to and find the journal.
  • Individuality in modelling: a simplifying assumption too far? David R. Cope. Nonlinear Analysis: Real World Applications 6(4): 691-704 (2005)

A nice tutorial:

  • Stewart Robinson. "Simulation Verifcation, Validation and Confidence: A Tutorial."
    Transactions of the Society for Computer Simulation International, 16 (No.2), pp. 63-69.

A good description and example applications of UA/SA methods:

A number of more technical papers, but you should definitely read these at some point if you are going to do "real" computational modelling:

  • Evaluating prediction uncertainty in simulation models. Michael D. McKay, John D. Morrison and
    Stephen C. Upton. Computer Physics Communications 117(1-2): 44-51 (1999).
  • Latin hypercube sampling and the propagation of uncertainty in analyses of complex systems.
    J. C. Helton and F. J. Davis. Reliability Engineering & System Safety 81(1): 23-69 (2003).
  • Blower S, Dowlatabadi H, May R, Anderson R. Sensitivity and Uncertainty Analysis of complex models of
    disease transmission: an HIV model, as an example. Int. Statist Rev. 1994;62:229-243.
  • Helton, J.C., Johnson, J.D., Sallaberry, C.J. and Storlie, C.B. (2006).
    Survey of sample-based methods for uncertainty and sensitivity analysis.
    Reliability Engineering and System Safety 91(10-11): 1175-1209.
  • Iman R, Helton J. An Investigation of Uncertainty and Sensitivity Analysis Techniques for
    Computer Models. Risk Analysis 1980;8:71-90.
  • Stochastic modelling of ecological processes using hybrid Gibbs samplers.
    D.M.Walker etal. Ecological Modelling 198: 40-52 (2006).
  • E.L.Ionides, C.Breto, A.A.King. Inference for Nonlinear dynamical systems.
    PNAS 103(49): 18438-43 (Dec 5 2006).
  • Selecting Parameters for calibration via sensitivity analysis: An inidividual-based model
    of mosquitofish population dynamics. Beaudouin etal (2008). Ecological Modeling 218: 29-48.

Back To: Syllabus-Mar#Mar28

Edit - History - Print - Recent Changes - Search
Page last modified on March 25, 2012, at 01:02 PM