- Research
- Open Access
- Open Peer Review

# Evaluation of stability of directly standardized rates for sparse data using simulation methods

- Joan K. Morris
^{1}Email authorView ORCID ID profile, - Joachim Tan
^{1}, - Paul Fryers
^{2}and - Jonathan Bestwick
^{1}

**Received:**19 June 2017**Accepted:**9 December 2018**Published:**22 December 2018

## Abstract

### Background

Directly standardized rates (DSRs) adjust for different age distributions in different populations and enable, say, the rates of disease between the populations to be directly compared. They are routinely published but there is concern that a DSR is not valid when it is based on a “small” number of events. The aim of this study was to determine the value at which a DSR should not be published when analyzing real data in England.

### Methods

Standard Monte Carlo simulation techniques were used assuming the number of events in 19 age groups (i.e., 0–4, 5–9, ... 90+ years) follow independent Poisson distributions. The total number of events, age specific risks, and the population sizes in each age group were varied. For each of 10,000 simulations the DSR (using the 2013 European Standard Population weights), together with the coverage of three different methods (normal approximation, Dobson, and Tiwari modified gamma) of estimating the 95% confidence intervals (CIs), were calculated.

### Results

The normal approximation was, as expected, not suitable for use when fewer than 100 events occurred. The Tiwari method and the Dobson method of calculating confidence intervals produced similar estimates and either was suitable when the expected or observed numbers of events were 10 or greater. The accuracy of the CIs was not influenced by the distribution of the events across categories (i.e., the degree of clustering, the age distributions of the sampling populations, and the number of categories with no events occurring in them).

### Conclusions

DSRs should not be given when the total observed number of events is less than 10. The Dobson method might be considered the preferred method due to the formulae being simpler than that of the Tiwari method and the coverage being slightly more accurate.

## Keywords

- Direct standardization
- Monte Carlo simulation
- Confidence interval coverage
- Tiwari
- Dobson

## Background

Directly standardized rates (DSRs) are routinely produced by national organizations to compare rates, such as for diseases, across different geographic areas. They are calculated by applying the observed age specific rates in each population to a population with a standard age distribution. As the rates usually apply to specific causes of death or disease, which are generally independent (apart from, for example, contagious diseases) and occur relatively infrequently, the Poisson distribution is used to model the occurrence of such events and to derive the confidence intervals for the standardized rates. There are four different approaches to estimating the confidence intervals for these rates: (i) using the normal approximation of the Poisson distribution, (ii) treating the DSR as a weighted sum of Poisson variables, (iii) assuming a gamma distribution, or (iv) assuming a beta distribution. The three most common methods of calculating confidence intervals for directly standardized rates are, firstly, the normal approximation for the total number of events [1] used, for example, in the IARC Cancer Incidence in Five Continents [2]. Secondly, the Dobson method which is an example of treating the DSR as a weighted sum of Poisson variables [3] and used, for example, by Public Health England in Official Statistics, such as for the Public Health Outcomes Framework [4]. Thirdly the Tiwari modified gamma method [5] which uses the beta distribution and is a modification of the gamma method proposed by Fay and Feuer [6] used by the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute in Bethesda, Maryland and the Italian Association of Cancer Registries [7].

It is known that the Normal approximation is appropriate only for large numbers of events (such as those occurring in whole countries) [8]. The Stata module “distrate” for calculating confidence intervals for directly standardized rates [7, 9] provides two methods for calculating confidence intervals: the Tiwari method as the default method with the Dobson method available as an option. It states that the Tiwari method “produces valid confidence intervals even when the number of cases is very small.” The Stata manual does not specify what “small” is [9]. There is uncertainty as to how “small” is too small, with the Australian Institute of Health and Welfare and the US Centers for Disease Control and Prevention recommending below 25 events [10, 11].

However, small numbers of events often occur. For example, in the IARC Cancer Incidence in Five Continents publication, there were many cancers with less than ten cases in several populations [2]. A simulation study by Ng et al. [12] examined in detail the relative performance of many different models over several different scenarios. They concluded that the methods by Dobson and Tiwari performed the most consistently. However, their simulations were limited to only considering standardizing for six age groups, where all populations had more than 10 events and did not have widely varying age-specific event rates and age distributions. We were therefore interested in examining the sensitivity of the Dobson and Tiwari methods for fewer than 10 events occurring over 19 different age groups and with widely varying age-specific risks and population sizes. The aim of this study was to investigate the accuracy of the two methods compared with that of the normal approximation using simulation procedures which mirror a variety of plausible real-world scenarios occurring in the production of age-specific rates in England. Real data were then used to demonstrate the methods.

## Methods

### Definition of an accurate confidence interval

A confidence interval is considered “accurate” if the probability that it includes the true value (the “coverage”) is close to the stated target probability; i.e., a 95% confidence interval should include the true value approximately 95% of the time. In this analysis we defined the terms “conservative,” “liberal,” and “accurate” as follows. If the coverage exceeds the stated probability by more than 40% the confidence interval is “conservative.” If the coverage is under the stated probability by more than 40% the confidence interval is “liberal.” Otherwise, it is accurate. In other words, a 95% confidence interval is considered accurate if its coverage is between 93% (100–(1.4 × 5))% and 97 (100–(0.6 × 5))%. It is generally considered preferable for confidence intervals to be conservative rather than liberal, but that can depend on the context of the analysis and the nature of any decisions to be based on the analysis.

### Simulation methods

The number of events occurring in each of 19 5-year age specific categories (i.e., 0–4, 5–9, ... 90+ years) was estimated using the random Poisson generating function in Stata for the following scenarios and restrictions which are sufficient to uniquely specify the age specific rates (*θ*_{i}) and the size of the exposed population in each age specific category (*d*_{i}).

- 1.
The size of the exposed population in each age category (

*d*_{i}) was assumed to be a linear function of age with the ratio of the population in the youngest category to that in the oldest category being 1 (all age categories the same size), 5 or 50. This is not restrictive as when calculating the DSRs the order of the age groups is unimportant. The total exposed population was 190,000, with an average of 10,000 in each category. An additional scenario assumed the sample population had the same age distribution as that of the European Standard Population. - 2.
The age specific rates (

*θ*_{i}) were assumed to be a linear function of age with the ratio of the rate (i.e., relative risk) in the oldest to that in the youngest being 1 (no association with age), 50, 500, or 5000. This is not restrictive as when calculating the DSRs the order of the age groups is unimportant. - 3.
The total expected number of events (∑

*d*_{i}*θ*_{i}) that would occur across all age groups was specified as: 1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,25 and 100.

- 1.
the true DSR (\( \frac{1}{\sum {w}_i}\sum {w}_i{\theta}_i \)) was calculated using the age specific rates (

*θ*_{i}) and the European Standard Population weights (*w*_{i}) - 2.
the observed DSRs and confidence intervals were divided by the true DSR to enable simple comparisons between the scenarios to be made

- 3.
the inclusion of 1 within the specified confidence intervals was noted

- 4.
the variation of (European standard population/sample population) divided by sum of (European standard population/sample population)

^{2}was calculated as a measure of how much the sample population differs from the European standard population.

Stata Version 14 was used to perform all simulations [9].

#### Calculating a directly standardized rate (DSR)

*R*, is calculated as a weighted average of

*n*age-specific rates \( \left(\frac{x_i}{d_i}\right) \):

Where

*x*_{i} are the observed age-specific numerator events;

*d*_{i} are the age-specific denominator populations;

*w*

_{i}are taken from the 2013 European Standard Population (ESP):

Age group | 0–4 | 5–9 | 10–14 | 15–19 | 20–24 | 25–29 | 30–34 | 35–39 | 40–44 | 45–49 |
---|---|---|---|---|---|---|---|---|---|---|

2013 ESP | 5000 | 5500 | 5500 | 5500 | 6000 | 6000 | 6500 | 7000 | 7000 | 7000 |

Age group | 50–54 | 55–59 | 60–64 | 65–69 | 70–74 | 75–79 | 80–84 | 85–89 | 90+ |
---|---|---|---|---|---|---|---|---|---|

2013 ESP | 7000 | 6500 | 6000 | 5500 | 5000 | 4000 | 2500 | 1500 | 1000 |

#### The normal approximation for calculating confidence intervals [4]

Where

*R*, *x*_{i}, *d*_{i} and *w*_{i} are defined as in A.

\( {Z}_{1-\raisebox{1ex}{$\alpha $}\!\left/ \!\raisebox{-1ex}{$2$}\right.} \) is the \( 100\left(1-\frac{\alpha }{2}\right) \)th percentile value of the inverse standard normal distribution.

#### The Dobson method of calculating confidence intervals [3]

*α*)% lower and upper confidence limits,

*LCL*and

*UCL*, are defined as:

Where

*R*, *x*_{i}, *d*_{i} and *w*_{i} are defined as in A.

*Inv* Χ^{2}(*π*, *ν*) is the 100(1 − *π*)th percentile value of the inverse chi-squared distribution with *ν* degrees of freedom

#### The Tiwari modified gamma method of calculating confidence intervals [5]

*α*)% lower and upper confidence limits,

*LCL*and

*UCL*, are defined as:

Where

*R*, *x*_{i}, *d*_{i} and *w*_{i} and *Inv* Χ^{2}(*π*, *ν*) are defined as in A and C

### Analysis of real data

The numbers of suicides occurring from 2013 to 2015 in the 326 local authority districts in England for males and females were analyzed using the procedure “distrate” in Stata Version 14. In each district the gender-specific DSRs were calculated using the 2013 European Standard Population weights and the Tiwari method to calculate the 95% confidence intervals. In addition, using the European Standard Population, the overall standardized rates for England were calculated for males and females separately. Suicides were chosen for the analysis because of the small numbers of deaths occurring at district level.

## Results

Empirical results from 10,000 simulations of weighted sums of Poisson parameters. (EU = distribution of sample population is same as European standard population)

Normal approximation | Dobson method | Tiwari method | |||||||||
---|---|---|---|---|---|---|---|---|---|---|---|

Expected events | Population ratio youngest vs. oldest | Incidence ratio oldest vs. youngest | 95% LCL | 95% UCL | Coverage (%) | 95% LCL | 95% UCL | Coverage (%) | 95% LCL | 95% UCL | Coverage (%) |

5 | 1 | 1 | 0.05 | 1.9 | 89.9 | 0.27 | 2.3 | 96.4 | 0.28 | 2.3 | 97.0 |

5 | 1 | 5000 | 0.02 | 1.9 | 87.6 | 0.25 | 2.4 | 94.6 | 0.26 | 2.4 | 96.9 |

5 | 50 | 1 | 0.03 | 1.9 | 87.0 | 0.24 | 2.4 | 94.0 | 0.26 | 2.6 | 97.9 |

5 | 50 | 5000 | 0.06 | 1.8 | 89.2 | 0.27 | 2.3 | 95.5 | 0.28 | 2.4 | 97.9 |

5 | EU | 1 | 0.12 | 1.9 | 86.9 | 0.32 | 2.3 | 98.0 | 0.32 | 2.3 | 98.0 |

5 | EU | 5000 | 0.12 | 1.9 | 87.0 | 0.32 | 2.3 | 97.9 | 0.32 | 2.3 | 97.9 |

10 | 1 | 1 | 0.34 | 1.6 | 92.2 | 0.44 | 1.9 | 96.3 | 0.45 | 1.9 | 96.4 |

10 | 1 | 5000 | 0.30 | 1.7 | 92.2 | 0.41 | 1.9 | 96.1 | 0.43 | 1.9 | 96.6 |

10 | 50 | 1 | 0.30 | 1.6 | 91.2 | 0.41 | 1.9 | 95.4 | 0.42 | 2.0 | 97.7 |

10 | 50 | 5000 | 0.33 | 1.6 | 92.0 | 0.43 | 1.9 | 95.8 | 0.44 | 1.9 | 96.6 |

10 | EU | 1 | 0.38 | 1.6 | 92.7 | 0.48 | 1.8 | 97.5 | 0.48 | 1.8 | 97.5 |

10 | EU | 5000 | 0.38 | 1.6 | 92.6 | 0.48 | 1.8 | 97.6 | 0.48 | 1.8 | 97.6 |

15 | 1 | 1 | 0.46 | 1.5 | 93.2 | 0.53 | 1.7 | 96.1 | 0.53 | 1.7 | 96.2 |

15 | 1 | 5000 | 0.43 | 1.5 | 93.3 | 0.51 | 1.7 | 96.0 | 0.51 | 1.7 | 96.3 |

15 | 50 | 1 | 0.43 | 1.5 | 92.1 | 0.50 | 1.7 | 95.6 | 0.51 | 1.8 | 97.0 |

15 | 50 | 5000 | 0.45 | 1.5 | 92.5 | 0.52 | 1.7 | 96.2 | 0.52 | 1.7 | 96.6 |

15 | EU | 1 | 0.49 | 1.5 | 91.8 | 0.56 | 1.6 | 96.3 | 0.56 | 1.6 | 96.3 |

15 | EU | 5000 | 0.49 | 1.5 | 92.0 | 0.56 | 1.6 | 96.3 | 0.56 | 1.6 | 96.3 |

25 | 1 | 1 | 0.58 | 1.4 | 93.9 | 0.62 | 1.5 | 95.9 | 0.62 | 1.5 | 96.0 |

25 | 1 | 5000 | 0.56 | 1.4 | 93.9 | 0.61 | 1.5 | 96.0 | 0.61 | 1.5 | 96.1 |

25 | 50 | 1 | 0.56 | 1.4 | 93.4 | 0.60 | 1.5 | 95.5 | 0.61 | 1.5 | 96.4 |

25 | 50 | 5000 | 0.57 | 1.4 | 93.5 | 0.62 | 1.5 | 95.6 | 0.62 | 1.5 | 95.9 |

25 | EU | 1 | 0.61 | 1.4 | 94.7 | 0.65 | 1.5 | 95.5 | 0.65 | 1.5 | 95.5 |

25 | EU | 5000 | 0.61 | 1.4 | 94.8 | 0.65 | 1.5 | 95.5 | 0.65 | 1.5 | 95.5 |

100 | 1 | 1 | 0.79 | 1.2 | 94.9 | 0.80 | 1.2 | 95.6 | 0.80 | 1.2 | 95.6 |

100 | 1 | 5000 | 0.78 | 1.2 | 95.0 | 0.79 | 1.2 | 95.8 | 0.79 | 1.2 | 95.8 |

100 | 50 | 1 | 0.78 | 1.2 | 94.5 | 0.79 | 1.2 | 95.4 | 0.79 | 1.2 | 95.7 |

100 | 50 | 5000 | 0.79 | 1.2 | 94.9 | 0.80 | 1.2 | 95.8 | 0.80 | 1.2 | 95.8 |

100 | EU | 1 | 0.80 | 1.2 | 94.6 | 0.81 | 1.2 | 95.5 | 0.81 | 1.2 | 95.5 |

100 | EU | 5000 | 0.80 | 1.2 | 94.5 | 0.81 | 1.2 | 95.4 | 0.81 | 1.2 | 95.4 |

Figure 2b plots the median values of the upper and lower 95% confidence limits and shows that the Dobson and Tiwari methods are very similar, with the Normal approximation predicting both the upper and lower confidence limits to be lower than those predicted by the other methods. In addition, using the Normal approximation to calculate the lower 95% confidence interval occasionally resulted in incorrect negative values being predicted.

The proportion of times the observed number of events or more occurred according to the expected number of events when the incidence in the oldest age group is 500 times the incidence in the youngest age group

Observed number of events | Expected number of events | ||||
---|---|---|---|---|---|

1 | 2 | 3 | 4 | 5 | |

Proportion of times the observed number of events or more occurred | |||||

None | 100.0 | 100.0 | 100.0 | 100.0 | 100.0 |

1 | 63.1 | 86.9 | 95.1 | 98.3 | 99.5 |

2 | 26.8 | 60.0 | 79.8 | 91.2 | 96.2 |

3 | 7.8 | 33.1 | 57.3 | 76.5 | 88.1 |

4 | 1.8 | 14.7 | 35.1 | 56.7 | 73.2 |

5 | 0.3 | 5.5 | 18.1 | 36.8 | 55.5 |

6 | 0.0 | 1.6 | 7.9 | 21.3 | 38.3 |

7 | 0.0 | 0.5 | 3.2 | 10.7 | 24.5 |

8 | 0.1 | 1.4 | 4.9 | 13.7 | |

9 | 0.0 | 0.3 | 2.1 | 7.2 | |

10 | 0.0 | 0.1 | 0.8 | 3.3 | |

11 | 0.0 | 0.0 | 0.3 | 1.5 | |

12 | 0.0 | 0.1 | 0.6 | ||

13 | 0.0 | 0.0 | 0.2 | ||

14 | 0.0 | 0.1 | |||

15 | 0.0 | 0.0 | |||

16 | 0.0 |

### Real data analysis

Number of suicides in each district by gender

Numbers of suicides | Number of districts | |
---|---|---|

Female | Male | |

0 | 1 | 0 |

1 | 5 | 1 |

2 | 15 | 0 |

3 | 19 | 1 |

4 | 21 | 1 |

5 | 27 | 0 |

6–9 | 97 | 6 |

10–19 | 110 | 80 |

20–49 | 28 | 178 |

50–99 | 3 | 51 |

100+ | 0 | 8 |

Total | 326 | 326 |

Amongst the nine districts in which fewer than 10 suicides in men were observed there were only two districts where the confidence interval (consistent for both methods) did not include the DSR for England of 1.58 per 10,000 (six and nine suicides in these two districts). Similarly amongst the 185 districts in which fewer than 10 suicides in women were observed there were only three districts where the confidence interval (consistent for both methods) did not include the DSR for England of 0.47 per 10,000 (one, one, and three suicides in these districts). In routine publication of DSRs the confidence intervals are presented to aid interpretation, with the guidance being that districts where the 95% CI includes the England comparator value should not assume there is any underlying reason for the difference as it is likely to be due to chance variation. For these data there are therefore only two districts in which this assumption may be misleading for the male suicide rates. For females the numbers of counts are so small that the confidence intervals are not reliable in many districts.

## Discussion

The strength of this study is that it is based on realistic simulated data. Firstly, as is standard practice, the European Standard Population weights for 19 5-year age groups were used. Secondly, the age specific rates were allowed to vary by as much as 5000 times for the oldest compared with the youngest age group. These variations do occur in real observed data: for example, the age specific rate of stroke deaths in men aged 90+ is over 5000 times that in men aged 15–19 [13]. Thirdly, the age distributions in the samples were allowed to vary considerably from the European Standard Population. This variation is measured by calculating the standardized variation of the ratio of the European Standard Population over the sample population for each age group (see methods section). The standardized variation ranged from 0 (for the scenario with the sample population having the same distribution as the standard population) to 0.00108 (ratio of size of the largest to smallest sample age group is equal to five) and 0.0133 (ratio of size of the largest to smallest sample age group is equal to 50). The observed variations in 326 local authority districts in England for males and females from 2013 to 2015 were from 0.0000183 to 0.00175, smaller than the most extreme scenarios modeled.

In all the simulations the total population size was 190,000. This is similar to 170,000, which is the average population size in the 326 local authority districts in England and also allows 10,000 people in each of the 19 age groups. The results are not sensitive to this assumption.

A limitation of this analysis is that it depends on the assumption that the occurrence of an event follows a Poisson distribution. Poisson distributions assume events are independent, an infinite number of events may occur over a long period of time, and that events occur only rarely in short periods of time. For most diseases and causes of death event independence is likely to be a reasonable assumption. It will not apply to rates of contagious diseases, for example, where the occurrences are not independent, or where extreme weather or events cause spikes in deaths from a single external factor. However, these confidence interval methods should not be used in such cases. The assumption of an infinite number of events occurring over a long period of time is also generally reasonable as for example when analysing a specific cancer the size of the population (i.e., the total number of deaths that could occur) is considerably greater than the numbers of actual deaths likely to occur and can therefore be thought of as infinite.

As stated in the methods section, specifying linear functions for the incidence and the population sizes is not restrictive, because when calculating the DSRs the order of the age groups is irrelevant. The effect of altering the incidence and age association is to create different amounts of clustering within the numbers of observed events; a ratio of 5000:1 in incidence between the highest and lowest categories will result in the lowest categories having very few events in contrast to a ratio of 1 which will spread the events evenly.

Both the Dobson and the Tiwari methods of calculating confidence intervals are influenced by the variation of the ratio of the population years in the standard population to the population years in the sample. Ng et al. [12] specified that if this variation was small (< 0.01 – as is the case in most of our modeled scenarios and the real data) then the Tiwari method was the recommended method if it was acceptable that the coverage was above the nominal value – as has been seen in these simulations. Ng et al. recommended the original gamma method proposed by Fay and Feuer [6] if one is more concerned with a symmetrical confidence interval that is closer to the nominal coverage.

The normal approximation for calculating confidence intervals was included in this study in order to provide a benchmark against which to judge the Dobson and Tiwari methods. In practice if the normal approximation was the method of choice then the use of continuity corrections to improve the fit to the normal distribution would need to be investigated [14], such as that suggested by Begaud et al. [15]. However as even standard spreadsheet packages can calculate the chi-squared distribution and can therefore calculate confidence intervals using both Dobson and Tiwari methods, efforts to improve the normal approximation were not considered further.

In agreement with Ng et al. [12] the coverage of the Dobson and Tiwari methods are both considered accurate for 10 or more observed events. Coverage from the Dobson method is consistently closer to 95%, with the coverage from the Tiwari method tending to be above 95%. However, as the confident limits from the two methods differ by less than 0.1% the differences in estimates are not significant.

## Conclusion

The results from this simulation confirm those predicted from other studies [3, 12, 15] and lead to the recommendation that at least 10 events must have occurred for a directly standardized rate to be published. Both the Dobson and Tiwari methods produce “accurate” confidence intervals when 10 or more events are observed. As expected, the normal approximation should not be used for fewer than 25 events. The Dobson method might be considered the preferred method due to the formulae being simpler than that of the Tiwari method and the coverage being slightly closer to 95%.

## Declarations

### Acknowledgements

Not applicable

### Funding

This study was funded by Public Health England.

### Availability of data and materials

Not applicable

### Authors’ contributions

JKM and JT created the simulation procedures and interpreted the results. JB and PF provided comments on the manuscript. All authors read and approved the final manuscript.

### Ethics approval and consent to participate

Not applicable

### Consent for publication

Not applicable

### Competing interests

The authors declare that they have no competing interests.

### Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

**Open Access**This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

## Authors’ Affiliations

## References

- Office for National Statistics. 2012. http://www.ons.gov.uk/ons/guide-method/user-guidance/health-and-life-events/age-standardised-mortality-rate-calculation-template.xls (Accessed 28 May 2017).
- Forman D, Bray F, Brewster DH, Gombe Mbalawa C, Kohler B, Piñeros M, Steliarova-Foucher E, Swaminathan R, Ferlay J. Cancer Incidence in Five Continents, Vol. X: IARC; 2014.Google Scholar
- Dobson AJ, Kuulasmaa K, Eberle E, Scherer J. Confidence intervals for weighted sums of Poisson parameters. Stat Med. 1991;10:457–62.View ArticleGoogle Scholar
- Public Health England. Technical Guide: Confidence Intervals. London: Public Health England; 2018. https://fingertips.phe.org.uk/documents/PHDS%20Guidance%20-%20Confidence%20Intervals.pdf (Accessed 8 Oct 2018Google Scholar
- Tiwari RC, Clegg LX, Zou Z. Efficient interval estimation for age-adjusted cancer rates. Stat Methods Med Res. 2006;15:547–69.View ArticleGoogle Scholar
- Fay MP, Feuer EJ. Confidence intervals for directly standardized rates: a method based on the gamma distribution. Stat Med. 1997;16:791–801.View ArticleGoogle Scholar
- Consonni D, Coviello E, Buzzoni C, Mensi C. A command to calculate age-standardized rates with efficient interval estimation. Stata J. 2012;(4):688–701.Google Scholar
- Newcombe RG. Two-sided confidence intervals for the single proportion: comparison of seven methods. Stat Med. 1998;17:857–72.View ArticleGoogle Scholar
- StataCorp. Stata Statistical Software: Release 14. College Station: StataCorp LP; 2015.Google Scholar
- Australian Institute of Health and Welfare. Principles on the use of direct age-standardization in administrative data collections: for measuring the gap between Indigenous and non-Indigenous Australians. Cat. no. CSI 12. Canberra: AIHW; 2011.Google Scholar
- Curtin LR, Klein RJ. Direct standardization (age-adjusted death rates), Statistical Notes, No. 6. March 1995. Hyattsville: US Dept. of Health and Human Services, Public Health Service, Centers for Disease Control and Prevention, National Center for Health Statistics; 1995.Google Scholar
- Ng HKT, Filardo G, Zheng G. Confidence interval estimating procedures for standardized incidence rates. Comput Stat Data Anal. 2008;52:3501–16. https://doi.org/10.1016/j.csda.2007.11.004.View ArticleGoogle Scholar
- Office for National Statistics. Mortality Statistics: Deaths registered in England and Wales series DR. 2013. http://www.ons.gov.uk/ons/rel/vsob1/mortality-statistics%2D%2Ddeaths-registered-in-england-and-wales%2D%2Dseries-dr-/2013/dr-tables-2013.xls (Accessed 28 May 2017).
- Fleiss JL. Statistical methods for rates and proportions. 2nd ed. New York: Wiley; 1981.Google Scholar
- Begaud B, Martin K, Abouelfath A, Tuber-Bitter P, Moore N, Moride Y. An easy to use method to approximate Poisson confidence limits. Euro J Epi. 2005;20:213–6. https://doi.org/10.1007/s10654-004-6517-4.View ArticleGoogle Scholar