Statistical Practice

A Comparison of Nine Con dence Intervals for a Poisson Parameter When the Expected Number of Events is 5
Lawrence BARKER
narrower expected widths being considered desirable. We make recommendations concerning when to use which of the intervals considered, based on considerations of coverage probability, expected width of con dence interval, and, to a small extent, computational convenience. 2. METHODS

Let fXi gn 1 be a collection of independent, identically disi= tributed Poisson random variables. Con dence intervals for can be constructed by the Wald method, by exact inference, from a variance stabilizing transformation, or by many other techniques. When n is small, actual and nominal coverage can differ substantially. We compare nine con dence intervals for a Poisson mean with respect to coverage and expected width of con dence limits. We show that, for small n : of the condence intervals considered, only the exact interval maintains coverage probabilities; while the scores interval approximately maintains coverage probability, its expected width exceeds the exact intervals; and a modi ed version of the con dence interval based on the variance stabilized con dence interval has coverage properties near the nominal and an expected width that is not particularly large. Thus, we recommend that investigators desiring true cover not less than nominal use the exact interval and those willing to accept approximate coverage use a modi ed form of the variance stabilized interval. KEY WORDS: Con dence interval; Exact inference; Poisson distribution; Small sample; Scores interval.

Typically, studies of the comparative performance of condence intervals rely on simulations (Newcombe 1998a; Newcombe 1998b). Here we precisely calculate, up to numerical limits described later, the coverage properties and expected widths of the con dence limits considered on a grid of parameter values. Since T is suf cient for , we need only consider the case fn = 1g and con dence intervals for fn g, n times the parameter . Any nominal 95% con dence interval for n can be expressed as the interval L(T ) to U (T ), for appropriate de ned L and U (e.g., for the standard Wald interval for n , L(T ) would be T Z =2 (T )0:5 and U (T ) would be T + Z =2 (T )0:5 . From rst principals, coverage of con dence interval = C( ) =
1 X i= 0

(IfL(i) n U (i)g)en in =(i!);

1. fXi gn 1 i=

INTRODUCTION

where Ifg is the indicator function of the bracketed event. Since T can take on only non-negative integer values, E(width of condence interval) = EW( ) =
1 X i= 0

Let be a collection of independent, identically distributed Poisson random variables. Consider constructing a (1 )100% con dence interval for . The simplest, and perhaps most widely used, solution is the familiar Wald interval, P X Z =2 (X=n)0:5 , where X = T =n; T = n 1 Xi , and Z =2 i= is the (1 =2)100th percentile of the standard normal distribution. The Wald interval asymptotically provides (1 )100% coverage for . The statistic T , which is complete and suf cient statistic for , has a Poisson n distribution; hence, we need only consider con dence intervals for n times . However, n itself can be small if the sample size is small or if the sample size is large and is small enough. If n is small, say < 5, (as will be shown), the Wald interval performs poorly in terms of coverage. Here, we compare the performance of nine nominal 95% con dence intervals for a Poisson parameter in terms of both coverage and expected width of con dence interval, with

(U (i)

L(i))en in =(i!):

The sums can be approximated by nite sums. Coverage probabilities and expected length of con dence intervals, with n ranging from 0.0 to 5.0 in steps of 0.1, are calculated to an accuracy of 0:005. The range of 0.0 to 5.0 is arbitrary, but reasonable. Coverage rates and expected widths were calculated for nine nominal 95% con dence intervals. Values of zero were substituted for negative lower con dence limits (e.g., in method 1, when X Z =2 (X=n)0:5 was negative, a lower bound of zero was used). Coverage rates and expected widths vary with . If strong prior knowledge exists, a con dence interval which provides good coverage and comparatively narrow intervals for near the likely state of nature can be chosen. Here, average coverage Lawrence Barker is a Health Scientist, Centers for Disease Controland Preven- and expected width are measured through integrated coverage tion, MS E-62, 1600Clifton Avenue, Atlanta, GA 30333(E-mail: lsb8@cdc.gov). over the interval (a; b) (IC(a;b) ) and integrated expected width 2002 American Statistical Association c
The American Statistician, May 2002, Vol. 56, No. 2 85

Figure 2. Plot of coverage probabilities of nominal 95% con dence Figure 1. Plot of coverage probability of nominal 95% con dence intervals W and VS against n . Dashed line represents 95% coverage. intervals (bottom to top) MW, WCC, S, MVS, RVS, FT, and E against n . Dashed line represents 95% coverage.

over the interval (a; b) (IEW (a;b) ), de ned, respectively, as Z and Z

b

C( )d =(b
a

a);

EW( )d =(b
a

a);

where Simpsons rule is used for integration. Naturally, IC(a;b) and IEW(a;b) depend on a and b; the values used here are arbitrary, but reasonable. The con dence intervals considered are: 1. Wald Interval (W). The W interval is derived from the asymptotic standard normal distribution of (X )=(X=n)0:5 . This quantity can be inverted to provide the interval X Z =2 (X=n)0:5 . 2. Modi ed Wald interval (MW). If X = 0, the W interval is degenerate at 0. Since few users would consider a degenerate interval, W is used when X = 0 and the limits of method 9 are = substituted otherwise. 3. Wald interval with continuity correction (WCC). The W interval uses a continuous distribution (normal) to approximate a discrete distribution (Poisson). A continuity correction might make this approximation more accurate. The WCC interval is given by

Figure 3a. Plot of expected lengths of nominal 95% con dence intervals MW, WCC, MVS, and RVS against n .

X Z

=2 (fX

+ 0:5g=n)0:5 :

4. Scores (S). The S interval is derived from the asymptotic standard normality of (X )=( =n)0:5 . This quantity can be inverted to provide the S interval
86 Statistical Practice

Figure 3b. Plot of expected lengths of nominal 95% con dence intervals S, FT, and E against n .

Table 1. Expected Width of Seven Con dence Intervals for Selected Values of n , Where n is the Number of Observations and is the Poisson Parameter Expected width of nominal 95% con dence interval Wald with continuity correction 2.36 3.27 4.12 4.93 5.68 6.37 7.04 7.64 8.20 8.72 Modi ed Variance stabilizing 3.70 4.24 4.82 5.42 6.00 6.56 7.09 7.60 8.08 8.53 Recentered Variance stabilizing 3.16 4.06 4.87 5.60 6.26 6.86 7.41 7.92 8.40 8.86

n 0.5 1.0 1.5 2.0 2.5 3.0 3.5 4.0 4.5 5.0

Modi ed Wald 3.35 3.72 4.62 4.88 5.52 6.15 6.76 7.33 7.87 8.38

Scores 4.62 5.32 5.96 6.55 7.09 7.60 8.09 8.54 8.98 9.39

Freeman Tukey 2.67 3.64 4.53 5.36 6.10 6.76 2.36 7.91 8.41 8.88

Exact 4.18 4.92 5.56 6.15 6.69 7.20 7.68 8.14 8.59 9.02

X + (Z

=2 )

=(2n) (Z

=2 )[4X

+ (Z

=2 )

=n]0:5 =(4n)0:5 :
0:5

5. Variance stabilizing (VS). The quantity (X 0:5 0:5 )=(1=4n) is asymptotically standard normal. This can be inverted into the interval X + (Z
=2 ) 2

Thus, the variance of (X)0:5 , the variance stabilized form of methods 5 and 6, is 1=4n + O(1= ). If c = 3=8, the variance of (X + c)0:5 is 1=4n + o(1= ). Thus, for large , the variance of (X + 3=8)0:5 depends less on than the variance of (X)0:5 . We can invert the quantity [(X + c)0:5 ( +c)0:5 ]=(1=4n)0:5 to obtain a nominal (1 )100% con dence interval for , X + (Z
=2 ) 2

=(4n) Z
2

=2 (X=n)

0:5

=(4n) Z

=2 [(X

+ 3=8)=n]0:5 :

Except for the additive term (Z =2 ) =(4n), VS is the same as W. 6. Modi ed variance stabilizing (MVS). If X = 0, VS is degenerate at (Z =2 )2 =4n. Since few users would consider a degenerate interval, the VS interval is used when X = 0 and = method 9s limits are substituted otherwise. 7. Recentered variance stabilizing (RVS). It is well known that, for any positive constant c, [(X + c)0:5 ( + c)0:5 ]=(1=4n)0:5 is asymptotically standard normal. Anscombe (1948) showed that, as tends to in nity, var(X + c)
0:5

Although the argument for use of RVS applies to large , the method is still applicable to small . 8. Freeman and Tukey (FT). Freeman and Tukey (1950) showed that n0:5 f[(X)0:5 + (X + 1)0:5 ]
0:5

0:5

+ ( + 1)0:5 ]g

has an asymptotically standard normal distribution. Thus, X + (X + 1)0:5 Z

0:5 ); =2 (1=n

= [1 + (3

8c)=(8 )] =(4n) + o(1= ):

is a nominal (1 )100% con dence intervalfor 0:5 +( +1)0:5 . The inverse of f (u) = u0:5 + (1 + u)0:5 is, for v 1, g(v) =

Table 2. IC( 0 ;2) , Minimum Coverage Over (0,2), IC( 2 ;5) , Minimum Coverage Over (2,5), IC( 0 ;5) , IEW( 0;2) , IEW( 2 ;5) , and IEW( 0 ;5) , Calculated for the Con dence Intervals MW, WCC, S, MVS, RVS, FT, and E Nominal 95% con dence interval MW WCC S MVS RVS FT E Minimum coverage over 0 n 2 0.99 0.75 0.91 0.98 0.98 0.65 0.95 Minimum coverage over 2 n 5 0.84 0.86 0.93 0.89 0.88 0.86 0.95 Minimum coverage over 0 n 5 0.84 0.75 0.91 0.89 0.88 0.86 0.95

IC(0;2) 0.99 0.94 0.95 0.99 0.99 0.90 0.97

IC(2;5) 0.92 0.92 0.96 0.96 0.94 0.93 0.96

IC(0;5) 0.95 0.93 0.96 0.97 0.96 0.92 0.96

IEW(0;2) 3.83 3.23 5.28 4.28 4.00 3.60 4.85

IEW(2;5) 6.71 6.97 8.05 7.04 7.35 7.28 7.65

IEW(0;5) 5.56 5.47 6.94 5.94 6.01 5.81 6.53

IC(a;b) = integrated coverage for a n b IEW(a;b) = integrated mean width of con dence interval for a n b MW = Wald con dence interval modi ed by exact limits when X = 0 WCC = Continuity corrected Wald con dence interval S = Scores interval MVS = Con dence interval based on variance stabilizing transformation, modi ed by exact limits when X = 0 RVS = Con dence interval based on recentered variance stabilizing transformation FT = Con dence interval based on transformation of Freeman and Tukey (1950) E = Exact con dence interval. The American Statistician, May 2002, Vol. 56, No. 2 87

Table 3. Recommendations for Con dence Interval to Use When n is Small and Little Prior Knowledge is Available Condition Experimenter: desires true coverage nominal coverage; tolerates con dence interval not in closed form Experimenter:tolerates approximate coverage; tolerates wide con dence intervals; desires con dence interval in closed form Experimenter: tolerates crudely approximate coverage; desires narrow con dence intervals; desires con dence interval in closed form
0:5

Recommendation E, exact interval

Reason No other interval provides true coverage nominal coverage

S, scores interval

No other closed form interval comes particularly close to preserving nominal coverage, but resulting interval can be wide

MVS, interval

modi ed

variance

stabilized

MVS comes closest to preserving nominal coverage among remaining intervals (MW, WCC, RVS, FT) while having a fairly narrow expected width

f(v 2 1)=(2v)g2 . If X gfX

0:5

+ (X + 1)0:5 Z

0:5 ) =2 (1=n

is 1,

+ (X + 1)0:5 Z

0:5 )g; =2 (1=n

is a nominal (1 )100% con dence interval for . If not, the upper endpoint is de ned as above and the lower endpoint is de ned as zero. 9. Exact method (E). The exact (1 )100% lower con dence limit for is the smallest value such that
S X i= 0

en (n )i =(i!) =2;

if such a exists, and 0 if it does not. The exact (1 )100% upper con dence limit for is the largest value such that
1 X i= S

en (n )i =(i!) =2:

The con dence limits must be determined numerically. Finally, formal Bayesian methods are not explicitly considered. However, informal prior knowledge, such as a strong a priori belief that n falls in a speci ed interval, is discussed. 3. RESULTS

Figure 1 displays some coverage probabilities for the nominal 95% con dence intervals W and VS. The coverage probability of VS has a discontinuity, omitted in Figure 1, at = (Z =2 )2 =(4n); the coverage probability can be quite large there, since VS can be degenerate at (Z =2 )2 =(4n). Figure 1 shows that, for small n , coverage is far below 95%. As will be seen later, other intervals give coverage probabilitiesmuch nearer the nominal for small n . If n is large enough so that P fX = 0g is negligible, the difference between coverage probabilities and expected widths of W and MW (VS and MVS) are negligible. Hence, intervals W and VS are not considered further. Figure 2 displays the coverage probabilities of con dence intervals MW, WCC, S, MVS, RVS, FT, and E, plotted against n . A dashed line indicates95% coverage.Figure 2 shows that all con dence intervals except E can achieve coverage of less than
88 Statistical Practice

95%. Figure 2 shows that S comes close to providing uniform 95% coverage, but is anticonservative for some n . Figures 3a and 3b display, respectively,the expected widths of con dence intervals MW, WCC, MVS, and RVS (S, FT, and E), plotted against n . Figures 3a and 3b show that expected widths can vary considerably, both by n and among con dence limits. These are supplemented by Table 1, which displays the expected widths of the con dence intervals for selected values of n . As Table 1, and other data not presented indicate, S has the greatest expected width for all values of n considered; in particular, the expected width of the nonconservative interval S, somewhat counterintuitively, exceeds that of the conservative interval E. As Table 1, and other data not presented indicate, WCC has the smallest expected width for n < 1:84, and MW has the smallest expected width for n > 1:84. Table 2 summarizes Figures 2 and 3 by means of IC(0;2) , minimum coverage over 0 n 2, IC(2;5) , minimum coverage over 2 n 5, IC(0;5) , minimum coverage over 0 n 5, IEW(0;2) , IEW(2;5) , and IEW (0;5) , calculated for the con dence intervals MW, WCC, S, MVS, RVS, FT, and E. All intervals except E have minimum coverages considerably below the nominal value. Intervals S, MVS, and RVS have minimum coverages that are not too much below the nominal value; all these have integrated coverages near the nominal value. Among S, MVS, and RVS (those with minimum coverage not too far beneath nominal), MVS has the smallest integrated width over the interval (0,5). 4. CONCLUSIONS AND RECOMMENDATIONS

We make recommendationsbased on three general principles: actual coverage should be little, if any, less than nominal coverage; smaller expected widths of con dence intervals are better; and some investigators prefer closed form con dence limits, for reasons of computational simplicity. These principles are listed in what is typically order of importance. The interval chosen depends on the relative importance the investigator places on these principles. Only E maintains nominal coverage; this is as expected, since other con dence intervals considered rely on asymptotic normality. Hence, if the investigator can not tolerate coverage smaller than the nominal value, E should be used.

S comes close to maintainingthe nominal coverage. However, the expectedwidth of S is greater than that of E. Thus, S should be used only if the investigator tolerates some anticonservativeness and believesthat the need for a closed form expressionoutweighs the greater expected width. Of the remaining intervals (MW, WCC, MVS, RVS, FT), all except WCC and FT provide average coverage reasonably close to the nominal values. MVS and RVS provide minimum coverage closest to the nominal value. Of these, MVS has the shortest integrated width. Thus, for investigators seeking closed form con dence intervals and who are willing to tolerate a perhaps non-negligibleamount of anti-conservativeness,we recommend interval MVS. Recommendations are summarized in Table 3. The preceding recommendations are made in the absence of prior knowledge about n , beyond its being small. Speci c knowledge, even if not expressed as a formal prior, can alter recommendations. Strong a priori belief about n can lead to different recommendations. For example, if one strongly be-

lieved that n did not exceed 2, then MW looks more attractive (high coverage, narrow expected width of con dence limits). Finally, we have considered only the case n 5. Other analyses, not reported here, suggest that all the con dence intervals considered perform reasonably well for large n . Thus, for large n , the choice of con dence interval is not as critical.
[Received September 2001. Revised January 2002.]

REFERENCES
Anscombe, F. J. (1948), The Transformation of Poisson, Binomial, and Negative Binomial Data, Biometrika, 35, 246254. Freeman, M. F., and Tukey, J. W. (1950), Transformations Related to the Angular and the Square Root, Annals of Mathematical Statistics, 21, 607611. Newcombe, R. G. (1998a), Two-Sided Con dence Intervals for the Single Proportion: Comparison of Seven Methods, Statistics in Medicine, 17, 857872. (1998b), Interval Estimation for the Difference Between Independent Proportions:Comparison of Eleven Methods,Statistics in Medicine, 17, 873 890.

The American Statistician, May 2002, Vol. 56, No. 2

89