Index > 2 Development of a strategy, implementation and execution program to remediate radioactively contaminated sites >

2.7.2 Site characterisation data quality objectives

Contents
2.7.2.1 Step 1: State the problem
2.7.2.2 Step 2: Identify the decision
2.7.2.3 Step 3: Identify the inputs to the decision
2.7.2.4 Step 4: Define the boundaries of the study
2.7.2.5 Step 5: Develop a decision rule
2.7.2.6 Step 6: Specify limits on decision errors
2.7.2.7 Step 7: Optimise the design for collecting data

The site characterization Data Quality Objectives Process should provide for early involvement of stakeholders and use a graded approach to data quality requirements. This graded approach should define the data quality requirements according to the type of survey being designed, the risk of making a decision error based on the data collected, and the consequences of making such an error. The approach should also provide a more effective survey design combined with a basis for judging the usability of the data collected.

Data Quality Objectives are qualitative and quantitative statements derived from the outputs of the Data Quality Objectives Process that should enable:

  • To clarify the study objective, e.g., define the boundary of the site to investigate;
  • To define the most appropriate type of data to collect, e.g., radiological and/or non-radiological data;
  • To determine the most appropriate conditions for collecting the data;
  • To specify limits on decision errors which will be used as the basis for establishing the quantity and quality of data needed to support the decision.
Figure 2.10 The data quality objectives process
Figure 2.10 The data quality objectives process

The Data Quality Objectives Process may consist of seven steps, as shown in Figure 2.10. The output from each step may influence the choices that will be made later in the process. Even though the Data Quality Objectives Process is depicted as a linear sequence of steps, in practice it is iterative; the outputs of one step may lead to reconsideration of prior steps as illustrated in Figure 2.11.

Figure 2.11 Repeated applications of the Data Quality Objectives Process throughout the radiation survey and site investigation process
Figure 2.11 Repeated applications of the Data Quality Objectives Process throughout the radiation survey and site investigation process

For example, defining the survey unit boundaries may lead to classification of the survey unit, with each area or survey unit having a different decision statement. This iteration is encouraged since it ultimately may lead to a more efficient survey design. The first six steps of the Data Quality Objectives Process should produce the decision performance criteria that will be used to develop the survey design and should be completed before the final survey design is developed, and every step should be completed before data collection begins.
The final step of the process should develop a survey design based on the Data Quality Objectives.

Data Quality Objectives for data collection activities should describe the overall level of uncertainty that the decision-maker is willing to accept for survey results. This uncertainty should be used to specify the quality of the measurement data required in terms of objectives for precision, accuracy, representativeness, comparability, and completeness.

The Data Quality Objectives Process should remain flexible considering the requirements of each specific situation. For surveys that have multiple decisions, such as characterisation or final status surveys, the Data Quality Objectives Process may be used repeatedly throughout the performance of the survey. Decisions made early in decommissioning are often preliminary in nature. For this reason, a scoping survey may only require a limited planning and evaluation effort. As the site investigation process nears conclusion the necessity of avoiding a decision error becomes more critical.

Depending on the definition of the problem, it should not be absolutely necessary that each step or each activity in a step will be implemented in a consecutive way in each process. This means that, on a case-by-case basis, it may be decided that specific steps or specific activities in a step may not be executed.

The steps within the Data Quality Objectives Process are briefly discussed in the next paragraphs, especially as they relate to final status survey planning, and list the outputs for each step in the process. The outputs from the Data Quality Objectives Process should be included in the documentation for the survey plan.

2.7.2.1 Step 1: State the problem

Any decision making process requires the problem to be defined so that the focus of the survey will be unambiguous. Since many sites or facilities may present a complex interaction of technical, economic, social, and political factors, the success of a project is critically linked to a complete but uncomplicated definition of the problem.
Four activities may be associated with this step:

  • Identifying members of the planning team and stakeholders;
  • Identifying the primary decision-maker or decision-making method;
  • Developing a concise description of the problem;
  • Specifying available resources and relevant deadlines for the study.

The expected outputs of this step should be:

  • A list of the planning team members and identification of the decision-maker;
  • A concise description of the problem, which would typically involve the release of all or some portion of the site to demonstrate compliance with the regulation;
  • A summary of available resources and relevant deadlines for the survey which are typically identified on a site-specific basis.

2.7.2.2 Step 2: Identify the decision

The goal of this step should be to define the question that the survey should attempt to resolve and identify alternative actions that may be taken based on the outcome of the survey. The combination of these two elements is called the decision statement. The decision statement would be different for each type of survey in the radiation survey and site investigation process, and would be developed based on the objectives of the survey.
Four activities should be associated with this step in the Data Quality Objectives Process:

  • Identifying the principal study question;
  • Defining the alternative actions that could result from a resolution of the principal study question;
  • Combining the principal study question and the alternative actions into a decision statement;
  • Organising multiple decisions.

The expected output from this step should be a decision statement that links the principal study question to possible solutions to the problem.

For a final status survey, the principal study question could be whether the level of residual radioactivity in the survey units or in a portion of the site is below the release criterion. Alternative actions may include further remediation, re-evaluation of the modelling assumptions used to develop the derived concentration guideline levels (DCGL), re-assessment of the survey unit to see if it can be released with passive controls, or a decision not to release the survey unit. The decision statement may also determine whether or not all the survey units in a portion of the site satisfy the release criterion.

2.7.2.3 Step 3: Identify the inputs to the decision

Collecting data or information is necessary to resolve most decision statements. In this step, the planning team should focus on the information needed for the decision and identify the different types of information needed to resolve the decision statement.

The key activities for this step should include:

  • Identifying the information required to resolve the decision statement; asking general questions such as whether information on the physical properties of the site is required, or whether information on the chemical characteristics of the radionuclide or the matrix is required; determining which environmental variables or other information are needed to resolve the decision statement.
  • Determining the sources for each item of information; identifying and listing the sources for the required information.
  • Identifying the information needed to establish the action level or the derived concentration guideline level based on the release criterion (the actual numerical value should be determined in Step 5).
  • Confirming that appropriate measurement methods exist to provide the necessary data; preparing a list of potentially appropriate measurement techniques based on the information requirements determined previously in this step.

The expected outputs of this step should be:

  • A list of informational inputs needed to resolve the decision statement;
  • A list of environmental variables or characteristics that will be measured.

For the final status survey, the list of information inputs generally should involve measurements of the radioactive contaminants of concern in each survey unit. These inputs should include identifying survey units, classifying survey units, identifying appropriate measurement techniques including measurement costs and detection limits, and whether or not background measurements from a reference area or areas need to be performed. The list of environmental variables measured during the final status survey should typically be limited to the level of residual radioactivity in the affected media for each survey unit.

2.7.2.4 Step 4: Define the boundaries of the study

During this step, the planning team should develop a conceptual model of the site based on existing information collected in Step 1 of the Data Quality Objectives Process or during previous surveys. Conceptual models describe a site or facility and its environs, and present hypotheses regarding the radio-nuclides present and potential migration pathways. These models may include components from computer models, analytical models, graphic models, and other techniques. Additional data collected during decommissioning should be used to expand the conceptual model.

The purpose of this step should be to define the spatial and temporal boundaries that will be covered by the decision statement so that data can be easily interpreted. These attributes should include:

  • Spatial boundaries that define the physical area under consideration for release (site boundaries);
  • Spatial boundaries that define the physical area to be studied and locations where measurements could be performed (actual or potential survey unit boundaries);
  • Temporal boundaries that describe the time frame the study data represents and when measurements should be performed;
  • Spatial and temporal boundaries developed from modelling used to determine the derived concentration guideline levels.

There should be seven activities associated with this step:

  • Specifying characteristics that define the true but unknown value of the parameter of interest;
  • Defining the geographic area within which all decisions must apply;
  • When appropriate, dividing the site into areas or survey units that have relatively homogeneous characteristics;
  • Determining the time frame to which the decision applies;
  • Determining when to collect data;
  • Defining the scale of decision making;
  • Identifying any practical constraints on data collection.

The expected outputs of this step should be:

  • A detailed description of the spatial and temporal boundaries of the problem (a conceptual model);
  • Any practical constraints that may interfere with the full implementation of the survey design.

Specifying the characteristics that define the true but unknown value of the parameter of interest for the final status survey typically should involve identifying the radio-nuclides of concern. If possible, the physical and chemical form of the radio-nuclides should be described. For example, describing the residual radioactivity in terms of total uranium is not as specific or informative as describing a mixture of uraninite (UO2) and uranium metaphosphate (U(PO3)4) for natural abundances of 234U, 235U, and 238U.

As an example, the study boundary may be defined as the property boundary of a facility or, if there is only surface contamination expected at the site, the soil within the property boundary to a depth of 15 cm. When appropriate (typically during and always before final status survey design), the site should be subdivided into survey units with relatively homogeneous characteristics based on information collected during previous surveys.

The time frame to which the final status survey decision applies is typically defined by the regulation. Temporal boundaries may also include seasonal conditions such as winter snow cover or summer drought that affect the accessibility of certain media for measurement.

For the final status survey, the smallest, most appropriate subsets of the site for which decisions will be made should be defined as survey units. The size of the survey unit and the measurement frequency within a survey unit should be based on classification, site-specific conditions, and relevant decisions used during modelling to determine the derived concentration guideline levels.

2.7.2.5 Step 5: Develop a decision rule

The purpose of this step should be to define the parameter of interest, specify the action level (or derived concentration guideline level), and integrate previous Data Quality Objectives outputs of the data quality requirements process into a single statement that describes a logical basis for choosing among alternative actions.
Three activities should be associated with this step:

  • Specifying the statistical parameter that characterises the parameter of interest;
  • Specifying the action level for the study;
  • Combining the outputs of the previous Data Quality Objectives Process steps into an “if…then…" decision rule that defines the conditions that would cause the decision-maker to choose among alternative actions.

Certain aspects of the site investigation process, such as historical site assessments, may not be so quantitative that a statistical parameter can be specified. Nevertheless, a decision rule should still be developed that defines the conditions that would cause the decision-maker to choose among alternatives.
The expected outputs of this step should be:

  • The parameter of interest that characterises the level of residual radioactivity;
  • The action level;
  • An “if…then…” statement that defines the conditions that would cause the decision-maker to choose among alternative actions.

The parameter of interest should be a descriptive measure (such as a mean or median) that specifies the characteristic or attribute that the decision-maker would like to know about the residual contamination in the survey unit.

The action level should be a measurement threshold value of the parameter of interest that provides the criterion for choosing among alternative actions.

The mean concentration of residual radioactivity may be the parameter of interest used for making decisions based on the final status survey. The definition of residual radioactivity will depend on whether or not the contaminant appears as part of background radioactivity in the reference area. If the radionuclide is not present in background, residual radioactivity should be defined as the mean concentration in the survey unit. If the radionuclide is present in background, residual radioactivity should be defined as the difference between the mean concentration in the survey unit and the mean concentration in the reference area selected to represent background.

A decision rule may state that, if the mean concentration in the survey unit is less than the investigation level, then the survey unit is in compliance with the release criterion. To implement the decision rule, an estimate of the mean concentration in the survey unit will be required. An estimate of the mean of the survey unit distribution may be obtained by measuring radionuclide concentrations in soil at a set of randomly selected locations in the survey unit. A point estimate for the survey unit mean may be obtained by calculating the simple arithmetic average of the measurements. Due to measurement variability, there might be a distribution of possible values for the point estimate for the survey unit mean, however. In this case, statistical decision rules should be used to assist the decision-maker.

2.7.2.6 Step 6: Specify limits on decision errors

Decisions based on survey results may often be reduced to a choice between “yes” or “no”, such as determining whether or not a survey unit meets the release criterion. When viewed in this way, two types of incorrect decisions, or decision errors, may be identified:

  1. Incorrectly deciding that the answer is “yes” when the true answer is “no”; and
  2. Incorrectly deciding the answer is “no” when the true answer is “yes”.

The distinctions between these two types of errors are important for two reasons:

  1. The consequences of making one type of error versus the other may be very different; and
  2. The methods for controlling these errors are different and involve trade-offs.

For these reasons, the decision-maker should specify levels for each type of decision error.

The purpose of this step should be to specify the decision-maker’s limits on decision errors, which should be used to establish performance goals for the data collection design. The goal of the planning team should be to develop a survey design that reduces the chances of making a decision error.

While the possibility of a decision error can never be totally eliminated, it can be controlled. To control the possibility of making decision errors, the planning team should attempt to control uncertainty in the survey results caused by sampling design error and measurement error. Sampling design error may be controlled by collecting a large number of samples. Using more precise measurement techniques or field duplicate analyses may reduce measurement error. Better sampling designs may also be developed to collect data that more accurately and efficiently represent the parameter of interest. Every survey may use a slightly different method of controlling decision errors, depending on the largest source of error and the ease of reducing those error components.

The estimate of the standard deviation for the measurements performed in a survey unit should include the individual measurement uncertainty as well as the spatial and temporal variations captured by the survey design. For this reason, individual measurement uncertainties should not be used during the final status survey data assessment. However, individual measurement uncertainties may be useful for determining an a priori estimate of the standard deviation during survey planning. Since a larger value of the standard deviation results in an increased number of measurements needed to demonstrate compliance during the final status survey, the decision maker may seek to reduce measurement uncertainty through various methods (e.g., different instrumentation). There may be trade-offs that should be considered during survey planning. For example, the costs associated with performing additional measurements with an inexpensive measurement system may be less than the costs associated with a measurement system with better sensitivity (i.e., lower measurement uncertainty, lower minimum detectable concentration). However, the more expensive measurement system with better sensitivity may reduce the standard deviation and the number of measurements necessary to demonstrate compliance to the point where it is more cost-effective to use the more expensive measurement system. For surveys in the early stages of the radiation survey and site investigation process, the measurement uncertainty and instrument sensitivity may become even more important. During scoping, characterisation, and remedial action support surveys, decisions about classification and remediation should be made based on a limited number of measurements. When the measurement uncertainty or the instrument sensitivity values approach the value of the derived concentration guideline level, it may become more difficult to make these decisions. From an operational standpoint, when operators of a measurement system have an a priori understanding of the sensitivity and potential measurement uncertainties, they will be able to recognise and respond to conditions that may warrant further investigation, e.g., changes in background radiation levels, the presence of areas of elevated activity, measurement system failure or degradation, etc.

The probability of making decision errors may be controlled by adopting a scientific approach, called hypothesis testing. In this approach, the survey results may be used to select between one condition of the environment (the null hypothesis) and an alternative condition (the alternative hypothesis). The null hypothesis is treated like a baseline condition that is assumed to be true in the absence of strong evidence to the contrary. Acceptance or rejection of the null hypothesis will depend upon whether or not the particular survey results are consistent with the hypothesis.

A decision error occurs when the decision-maker rejects the null hypothesis when it is true, or accepts the null hypothesis when it is false.

When the null hypothesis is rejected when it is true, this is sometimes referred to as a false positive error. The probability of making such a decision error, or the level of significance, is denoted by alpha (α). Alpha reflects the amount of evidence the decision-maker would like to see before abandoning the null hypothesis, and is also referred to as the size of the test.

When the null hypothesis is accepted when it is false, this is sometimes referred to as a false negative error. The probability of making such a decision error is denoted by beta (β). The term (1-β) is the probability of rejecting the null hypothesis when it is false, and is also referred to as the power of the test.

There is a relationship between α and β that is used in developing a survey design. In general, increasing α decreases β and vice versa, holding all other variables constant. Increasing the number of measurements typically results in a decrease in both α and β.

Five activities should be associated with specifying limits on decision errors:

  • Determining the possible range of the parameter of interest; establishing the range by estimating the likely upper and lower bounds based on professional judgement;
  • Identifying the decision errors and choosing the null hypothesis:
    • Defining both types of decision errors and establishing the true condition of the survey unit for each decision error;
    • Specifying and evaluating the potential consequences of each decision error;
    • Establishing which decision error has more severe consequences near the action level, consequences including health, ecological, political, social, and resource risks;
    • Defining the null hypothesis and the alternative hypothesis and assigning the appropriate term to the appropriate decision error;
  • Specifying a range of possible parameter values, a gray region, where the consequences of decision errors are relatively minor; specifying a gray region will be necessary because variability in the parameter of interest and unavoidable imprecision in the measurement system may combine to produce variability in the data such that a decision may be “too close to call” when the true but unknown value of the parameter of interest is very near the action level;
  • Assigning probability limits to points above and below the gray region that reflect the probability for the occurrence of decision errors;
  • Graphically representing the decision rule.

The expected outputs of this step should be decision error rates based on the consequences of making an incorrect decision. Certain aspects of the site investigation process, such as historical site assessments, may not be so quantitative that numerical values for decision errors can be specified. Nevertheless, a “comfort region” should be identified where the consequences of decision errors are relatively minor.

2.7.2.7 Step 7: Optimise the design for collecting data

This step should produce the most resource-effective survey design that is expected to meet the Data Quality Objectives. It may be necessary to work through this step more than once after revisiting previous steps in the Data Quality Objectives Process.

Six activities should be included in this step:

  • Reviewing the Data Quality Objectives outputs and existing environmental data to ensure they are internally consistent;
  • Developing general data collection design alternatives;
  • Formulating the mathematical expressions needed to solve the design problem for each data collection design alternative;
  • Selecting the optimal design that satisfies the Data Quality Objectives for each data collection design alternative; if the recommended design will not meet the limits on decision errors within the budget or other constraints, the planning team will need to relax one or more constraints. Examples include:
    1. Increasing the budget for sampling and analysis;
    2. Using exposure pathway modelling to develop site-specific derived concentration guideline levels;
    3. Increasing the decision error rates, not forgetting to consider the risks associated with making an incorrect decision;
    4. Increasing the width of the gray region by decreasing the minimum value of the gray region;
    5. Relaxing other project constraints, e.g., schedule;
    6. Changing the boundaries; it may be possible to reduce measurement costs by changing or eliminating survey units that will require different decisions;
    7. Evaluating alternative measurement techniques with lower detection limits or lower survey costs;
    8. Considering the use of passive controls when releasing the survey unit rather than unrestricted release.
  • Selecting the most resource-effective survey design that satisfies all of the Data Quality Objectives; typical sites (e.g., mixed-waste sites) may require the planning team to consider alternative survey designs on a site-specific basis.
  • Documenting the operational details and theoretical assumptions of the selected design in the quality assurance project plan, the field sampling plan, the sampling and analysis plan, or the decommissioning plan, all of the decisions that will be made based on the data collected during the survey should be specified along with the alternative actions that may be adopted based on the survey results.

Key inputs for a final status survey design should include:

  • Investigation levels and derived concentration guideline levels for each radio-nuclide of interest;
  • Acceptable measurement techniques for scanning, sampling, and direct measurements, including detection limits and estimated survey costs;
  • Identification and classification of survey units;
  • An estimate of the variability in the distribution of residual radioactivity for each survey unit, and in the reference area if necessary;
  • The decision-maker’s acceptable a priori values for decision error rates (α and β).
Change this : "A decision rule may state that, if the mean concentration in the survey unit is less than the investigation level, then the survey unit is in compliance with the release criterion. To implement the decision rule, an estimate of the mean concentration in the survey unit will be required." By : A decision rule may state that, if the, real but unknown, mean concentration in the survey unit is less than the investigation level, then the survey unit is in compliance with the release criterion. To implement the decision rule, an estimate of the mean concentration in the survey unit will be required.
– by Rafael Garcia-Bermejo Fernandez about 6 years ago