Grantee Research Project Results
Final Report: Statistical Issues Related to the Implementation of Benchmark Dose MethodEPA Grant Number: R825385
Title: Statistical Issues Related to the Implementation of Benchmark Dose Method
Investigators: Patil, G. P. , Stiteler, W. M. , Taillie, C. , Banga, S.
Institution: Pennsylvania State University
EPA Project Officer: Aja, Hayley
Project Period: January 1, 1997 through December 31, 2002
Project Amount: $299,823
RFA: Environmental Statistics (1996) RFA Text | Recipients Lists
Research Category: Environmental Statistics , Human Health , Aquatic Ecosystems
Objective:Develop likelihood-based procedures for calculating confidence limits on the risk function and on the effective dose?benchmark dose (BMD) for continuous responses with emphasis on skew (nonnormal) distributed responses. Assess the sensitivity to model mis-specification. Examine the statistical validity of BMD-determination by inversion of an upper confidence curve on the risk function.
A BMD for continuous responses may be defined as a lower confidence limit on the effective dose corresponding to a specified risk level r. However, calculating such a confidence limit is not straightforward. By contrast, it is technically easier to obtain confidence limits on the risk function R(d). One approach that has been suggested for BMD determination is to first obtain a pointwise upper confidence curve U(d) on the risk function and then to invert this relationship by solving the equation U(d)=r. The solution d is purported to be the desired BMD (i.e., a lower confidence limit on the effective dose corresponding to the risk level r).
Project research has addressed the following issues:
- Development of a likelihood contour method for obtaining (asymptotic) likelihood-based confidence limits on any smooth univariate function of the parameters in a multi-parameter statistical model. The risk function and the effective dose are both examples of such univariate real-valued functions.
- Proof that the contour method equations for obtaining confidence limits on the risk function have exactly two solutions (corresponding to upper and lower confidence limits) in the large sample limit. For finite sample sizes, more than two solutions are possible depending upon the model. For the normal homoscedastic model, there are exactly two solutions for all sample sizes.
- Development of starting values for iterative solution of the contour method equations. This is accomplished by replacing the equations by the lowest order terms in their asymptotic expansions and solving the simplified approximating equations.
- Implementation of the contour method and assessment of its behavior for small sample sizes. Simulation is used to obtain achieved coverage levels that are compared with nominal levels. Comparison also is made with levels achieved by the MLE confidence limits (i.e., point estimate plus or minus a multiple of the standard error as determined from Fisher's information matrix). We also compute the coverage probabilities achieved by using the starting values described in item 3 above. Simulations have been carried out for upper confidence limits on the risk function in the case of two models: (1) normal homoscedastic model with the mean as a quadratic function of the dose, and (2) gamma distributed responses with constant (but unknown) index parameter and the log of the mean as a quadratic function of the dose. Two additional skew families, the lognormal and reciprocal gamma, are included by transformation to the normal and gamma, respectively.
Results indicate that asymptotic confidence limits computed from the contour method yield coverage probabilities that match the nominal levels to within a percentage point or two, and that convergence to nominal levels is quite rapid with increasing sample sizes. Using the starting values as final values achieves a coverage that is somewhat inferior to the final iterative solution for small sample sizes. Thus, full iterative solution will usually be worth the computational effort unless the sample sizes are large.
On the other hand, coverages achieved by MLE confidence limits can be off by 5 or 10 percentage points for non-gaussian responses, and the coverage converges to nominal levels only slowly with increasing sample sizes. In addition, the use of MLE confidence limits with non-gaussian data consistently resulted in undercoverage of the true risk function and failed to be conservative.
- Effect of the direction of adversity. For symmetric models like the normal homoscedastic, the formalism is the same regardless of the direction of adversity. However, skew models like the gamma require separate development, depending on whether the direction of adversity is to the left or to the right. Our simulation results for the gamma indicate that LR coverage probability as well as accuracy of starting values is about the same for each direction.
- Assessing the effect of model mis-specification. A principal goal of our research has been to extend the BMD methods to non-normal models. But this raises two related questions: (1) Suppose the data are actually non-normal, but you analyze it as if it followed a normal model. What are the consequences for your BMD determinations? In other words, how important is it to have non-normal methodology available? (2) Given that a non-normal model is recognized as appropriate, how accurately must that model be specified? For example, what are the consequences if you analyze the data using a lognormal model when the data actually follow a gamma model?
We have simulated data from lognormal, gamma, and reciprocal gamma distributions, but then analyzed the data as if it came from a normal distribution. Depending upon parameter values, coverage probabilities can be very poor. Two interesting conclusions also emerge:
(a) the coverage probability typically gets worse as the sample size increases (probably a result of inconsistency of the normal based estimators when applied to nonnormal data); and
(b) the normal based methods consistently give overcoverage when applied to the (short-tailed) gamma data and undercoverage when applied to the (long-tailed) lognormal and reciprocal gamma data.
Item (b) raises the issue of characterizing distributional families on the basis of the pattern of overcoverage or undercoverage of the risk function when data from the distribution is analyzed using normal theory methods. However, we have not obtained any results in this direction.
The simulated data from the gamma and reciprocal gamma distributions also have been log transformed and then analyzed using normal theory methods. Here again, coverage of the risk function is poor, indicating that the log transformation is not effective in mitigating skewness except for the lognormal distribution.
- Extension of the likelihood contour method to direct determination of (asymptotic) BMD levels rather than inverting an upper confidence curve on the risk function. Let U(d) be the pointwise upper confidence curve on the risk function as determined by the likelihood ratio method. In general, the solution of the equation U(d)=r does not have a unique solution d. We have shown that the smallest (i.e., most conservative) solution is the same as the lower confidence limit on the effective dose EDr as determined by the likelihood ratio method. Since the latter is known to be asymptotically valid, this provides a proof that the inversion method is asymptotically valid for determining benchmark dose levels provided likelihood ratio methods are employed.
- Investigate the statistical validity of the inversion method for determining BMD levels. In item 7, we have shown that the inversion method is asymptotically valid when likelihood ratio methods are employed. Both italicized qualifications are important. Here, we study the inversion method in the more general setting of small sample sizes and other methods for obtaining the upper confidence limit U(d) on the risk function. The BMD is taken to be the smallest solution of the equation U(d)=r for fixed r. Regardless of the distributional model, we have shown that the inversion method is conservative in the sense that its achieved coverage is at least as large as the coverage of the risk achieved by U(d). Depending upon the model, the overcoverage can be zero (i.e., the inversion method can be correct). A sufficient condition for overcoverage to be zero is that the upper confidence curve on the risk function be monotone increasing (at least beyond some initial dip) with probability one.
A detailed investigation has been made for the normal homoscedastic model whose mean is a polynomial function of the dose. An exact method, based on the noncentral t-distribution, is available for obtaining upper confidence limits on the risk function. We have used the Abramowitz-Stegun approximation to the noncentral t-distribution to carry out the inversion analytically and to study its properties. For t-based intervals with the normal homoscedastic model, the conclusions are:
(a) the inversion overcoverage vanishes identically when the mean is a straight line function of the dose; but
(b) the inversion overcoverage is strictly positive when the mean is a quadratic function of the dose (Kodell-West model).
Further, in case (b), the achieved coverage can range anywhere from the nominal level all the way up to 100 percent (exclusively), depending upon the parameters of the model.
An analytic expression, involving the bivariate noncentral t probability integral, has been obtained for the overcoverage probability in case (b). A Mathematica program has been written for efficient evaluation of this integral.
- Investigate the accuracy of the Abramowitz-Stegun approximation to the noncentral t-distribution (used in item 8, above). Results show that the approximation is highly accurate (effectively exact for most practical purposes) for degrees of freedom as small as 10. By comparison, the more familiar approximation based on a normal distribution with the same mean and variance is poor unless degrees of freedom are very large.
Journal Articles on this Report : 2 Displayed | Download in RIS Format
|Other project views:||All 6 publications||2 publications in selected types||All 2 journal articles|
||Banga SJ, Patil GP, Taillie C. Sensitivity of normal theory methods to model misspecification in the calculation of upper confidence limits on the risk function for continuous responses. Environmental and Ecological Statistics 2000;7(2):177-189.||
||Banga S, Patil GP, Taillie C. Likelihood contour method for the calculation of asymptotic upper confidence limits on the risk function for quantitative responses. Risk Analysis 2001;21(4):613-624.||
Supplemental Keywords:benchmark dose; coverage probability; deviance; dose-response; gamma distribution; health effects; human health; inversion method; likelihood contour method; likelihood ratio; LOAEL; lognormal distribution; NOAEL; profile likelihood; reciprocal gamma distribution; risk assessment.,
Progress and Final Reports:Original Abstract
The perspectives, information and conclusions conveyed in research project abstracts, progress reports, final reports, journal abstracts and journal publications convey the viewpoints of the principal investigator and may not represent the views and policies of ORD and EPA. Conclusions drawn by the principal investigators have not been reviewed by the Agency.
Project Research Results
2 journal articles for this project