Use of Monte Carlo Simulation in Risk Assessments | US EPA (2024)

Region 3 Technical Guidance Manual, Risk Assessment

EPA's current risk assessment methods express health risks as single numerical values, or "single-point" estimates of risk. This technique provides little information about uncertainty and variability surrounding the risk estimate. Recent EPA guidance (EPA, 1992) recommends developing "multiple descriptors" of risk to provide more complete information to Agency decision-makers and the public. Monte Carlo simulation is a highly effective way to produce these multiple risk descriptors. This document recommends guidelines under which Region III risk assessors may accept the optional use of Monte Carlo simulation to develop multiple descriptors of risk. The Region will continue to require single-point risk estimates, prepared under current national guidance, in conjunction with optional Monte Carlo simulations.

SingleRiskEstimates VS. Multiple Descriptors

EPA designed its human health risk assessment guidance (e.g., EPA, 1991, 1989 and 1988) to produce protective, rather than best, estimates of risk. EPA is aware that true risks are probably less than its estimates, but has chosen a regulatory policy of giving the benefit of uncertainty surrounding the risk assessment to the exposed public.

These protective risk estimates sometimes create difficulty for Agency decision-makers and the public. Site-specific Regional risk assessments usually present risk as a single number, or single-point estimate, accompanied by a qualitative discussion of uncertainty. The public tends to focus on the single-point estimate and to overlook the uncertainty, which may span several orders of magnitude. EPA risk managers, though aware of the uncertainty, must still justify their decision to either accept or reduce the single-point risk. If the risk is close to the maximum acceptable level, it is likely that different assumptions would have produced a different risk number, leading to a different decision. In this way, single-point risk assessment methods place the risk assessor in an inappropriate risk management role.

Recent EPA guidance on risk characterization (EPA, 1992) discusses this problem in depth, and recommends the use of multiple risk descriptors in addition to protective single-point risk estimates. Inclusion of these additional risk descriptors provides the public with more complete information on the likelihood of various risk levels, and risk managers with multiple risk-based cleanup goals from which to choose. This guidance mentions Monte Carlo simulation as an effective source of multiple risk descriptors.

MonteCarloSimulation

Monte Carlo simulation is a statistical technique by which a quantity is calculated repeatedly, using randomly selected "what-if" scenarios for each calculation. Though the simulation process is internally complex, commercial computer software performs the calculations as a single operation, presenting results in simple graphs and tables. These results approximate the full range of possible outcomes, and the likelihood of each. When Monte Carlo simulation is applied to risk assessment, risk appears as a frequency distribution graph similar to the familiar bell-shaped curve, which non-statisticians can understand intuitively.

Monte Carlo simulation also has important limitations, which have restrained EPA from accepting it as a preferred risk assessment tool:

1. Available software cannot distinguish between variability and uncertainty. Some factors, such as body weight and tap water ingestion, show well-described differences among individuals. These differences are called "variability". Other factors, such as frequency and duration of trespassing, are simply unknown. This lack of knowledge is called "uncertainty". Current Monte Carlo software treats uncertainty as if it were variability, which may produce misleading results.
   
2. Ignoring correlations among exposure variables can bias Monte Carlo calculations. However, information on possible correlations is seldom available.
   
3. Exposure factors developed from short-term studies with large populations may not accurately represent long-term conditions in small populations.
   
4. The tails of Monte Carlo risk distributions, which are of greatest regulatory interest, are very sensitive to the shape of the input distributions.

Because of these limitations, Region III does not recommend Monte Carlo simulation as the sole, or even primary, risk assessment method. Nevertheless, Monte Carlo simulation is clearly superior to the qualitative procedures currently used to analyze uncertainty and variability. For baseline risk assessments at NPL sites, Region III recommends that uncertainty and variability surrounding single-point risk estimates rely on multiple descriptors of risk (EPA, 1992). Monte Carlo simulation will be an acceptable method for developing these multiple descriptors.

The following example (from Smith, in press) illustrates the advantages of Monte Carlo simulation in risk assessment:

At a Superfund site in Region III, volatile organic compounds migrated to residential wells. The single-point RME estimate of lifetime cancer risk to exposed residents, based on ingestion of tap water and inhalation while showering, was 1.14e-3.

Figure 1 shows the output of a PC-based Monte Carlo simulation program for the risk assessment. Each exposure parameter was entered as a frequency distribution (i.e., a "bell-shaped" curve showing the range of possible values, and the likelihood of each) rather than as a single number. Carcinogenic potency slopes were entered as fixed values rather than frequency distributions, so the variability in risk was due entirely to the exposure assumptions.

Risk was calculated 5000 times, with each calculation based on a different randomly-selected exposure scenario. The figure lists the RME, average, and four percentiles of risk, and shows the entire risk distribution. The RME risk estimate fell between the 95th and 99th percentiles in this example, appropriately protective as intended. This figure clearly provides more complete risk information than the single numerical RME estimate.

GuidelinesForUsing MonteCarloSimulation

Region III risk assessors believe that Monte Carlo simulation requires more development before it can serve as the primary risk assessment method, for reasons described above. However, the technique has clear advantages over the qualitative analyses of uncertainty and variability currently in use. Region III will accept Monte Carlo simulations submitted as uncertainty/variability analyses in risk assessments, under the following guidelines:

1. Include only human receptors. This guidance excludes environmental receptors.
   
2. Submit a work plan for EPA review before doing the Monte Carlo simulation, to ensure the work will be acceptable to EPA. The workplan should describe the software to be used, the exposure routes and models, and input probability distributions and their sources. EPA expects that peer-reviewed literature and site-specific data will be used whenever possible. Use professional judgment only as a last resort, and only in the form of triangular or uniform distributions. Describe how correlations among input variables will be handled.
   
3. Include only exposure variables in the Monte Carlo simulation. Enter reference doses and carcinogenic slope factors as single numbers, except for specific contaminants for which the EPA Office of Research and Development has already approved frequency distributions.
   
4. Include only significant exposure scenarios and contaminants in the Monte Carlo simulation. First, calculate RME risks for all exposure routes under current guidance. Select exposure routes for which RME risk exceeds either 1e-6 cancer risk or a non-carcinogenic hazard index of 1. Include only contaminants which contribute 1% or more of the total RME risk or hazard index.
   
5. Use Monte Carlo simulation only to analyze uncertainty and variability, as a "multiple descriptor" of risk. Include standard RME risk estimates in all graphs and tables of Monte Carlo results. Generate deterministic risks using current EPA national guidance (EPA 1992, 1991, 1989, and 1988).
   
6. Include graphs and tables showing and describing each input distribution, distributions of risk for each exposure route, and distributions of total risk (summed across exposure pathways and age groups, as appropriate under current guidance).

Region III will not accept Monte Carlo simulations which are not approved beforehand, or do not adhere to these guidelines.

Summary

Region III will accept Monte Carlo simulations that conform to the guidelines in this document, as part of baseline human health risk assessments. The most important guideline is that all risk assessments must include single-point RME risk estimates prepared under current EPA national guidance. The Region will accept Monte Carlo simulation only as an optional addition to, not a substitute for, current risk assessment methods.

Reference

• EPA, 2000. Risk Characterization Handbook, (U.S. Environmental Protection Agency, Office of Science Administration, Washington, DC).
• EPA, 1991. Standard Default Exposure Factors, Risk Assessment Guidance for Superfund, Volume I: Human Health Evaluation Manual Supplemental Guidance, (U.S. Environmental Protection Agency Office of Solid Waste and Emergency Response, Toxics Integration Branch, Washington, DC, OSWER Directive 9285:6-03).
• EPA, 1989. Risk Assessment Guidance for Superfund, Volume I: Human Health Evaluation Manual (Part A), (U.S. Environmental Protection Agency Office of Solid Waste and Emergency Response, Toxics Integration Branch, Washington, DC, EPA/540/1-89/002).
• EPA, 2011.Exposure Factors Handbook- 2011 Edition, (U.S. Environmental Protection Agency National Center forEnvironmental Assessment, Washington, DC, EPA/600/R-09/052F).
• Smith, R.L. In press. Use of Monte Carlo simulation for human exposure at a Superfund site. Submitted to Risk Analysis, May 1993.

For additional information, call 215-597-6682.

Approved by:
Thomas C. Voltaggio, Director
Hazardous Waste Management Division