Automatic electronic reporting improved the completeness of AMI and stroke incident surveillance in Tianjin, China: a modeling study

Abstract

Background

AMI and stroke are the leading causes of premature mortality and hospitalizations in China. Incidence data at the population level for the two diseases is limited and the reliability and completeness of the existing incidence registry have not been investigated. We aim to assess if the completeness of case ascertainment of AMI and stroke incidence has improved since the implementation of electronic reporting and to estimate the incidence of AMI and stroke in Tianjin, China.

Methods

We applied the DisMod II program to model the incidence of AMI and stroke from other epidemiological indicators. Inputs include mortality rates from Tianjin’s mortality surveillance system, and the point prevalence, remission rates and relative risks taken from IHME’s Global Burden of Disease studies. The completeness of AMI and stroke incidence reporting was assessed by comparing the sex and age-specific incidence rates derived from the incidence surveillance system with the modeled incidence rates.

Results

The age and sex standardized modeled incidence per 100,000 person-year decreased (p < 0.0001) from 138 in 2007 to 119 in 2015 for AMI and increased (p < 0.0001) from 520 in 2007 to 534 in 2015 for stroke. The overall completeness of incidence report was 36% (95% CI 35–38%) for AMI and 54% (95% CI 53–55%) for stroke. The completeness was higher in men than in women for both AMI (42% vs 30%, p < 0.0001) and stroke (55% vs 53%, p < 0.0001) and was higher in residents aged 30–59 than those aged 60 or older for AMI (57% vs 38%, p < 0.0001). The completeness of reporting increased by 7.2 (95% CI 4.6–9.7) and 15.7 (95% CI 14.4–16.9) percentage points for AMI and stroke, respectively, from 2007 to 2015 among those aged 30 or above. The increases were observed in both men and women (p < 0.0001) and were more profound (p < 0.0001) among those aged between 30 and 59 and occurred primarily during the 2010 and 2015 period.

Conclusions

Completeness of AMI and stroke incidence surveillance was low in Tianjin but has improved in recent years primarily owing to the incorporation of an automatic reporting component into the information systems of health facilities.

Introduction

Reliable estimations of representative population-level disease incidence are crucial for the prioritization of health service planning and policy making, and provide important indicators for assessing the effectiveness of preventive measures and management of risk factors. Such estimates also provide essential inputs for the calculation of the Disability-Adjusted Life Year, a vital metric for the evaluation of burden of diseases [1]. Estimates of epidemiological data come from different sources: mortality data are typically derived from civil registration and vital statistics, while incidence data are from disease registry or population-level cohort studies [2]. For many diseases, incidence is often more difficult to measure than mortality due to substantial incompleteness and significant internal inconsistency [3]. Lack of consistency in epidemiology estimates, an indicator for potential measurement error, exists even within a well-defined population [2].

AMI and stroke are major causes of hospitalization and the leading causes of premature mortality [4, 5] in China. Given the increasing burden of AMI and stroke, it is essential to monitor their incidence. Empirical data on the incidence of the two diseases at either national or sub-national levels are limited. Existing data sources do not yield sufficient and long-term continuous AMI incidence information [6,7,8,9]. For instance, the China Acute Myocardial Infarction Registry covered over 100 hospitals from all provinces and municipalities throughout Mainland China. The registry, which obtains clinical characteristics, diagnosis, treatment and outcomes of Chinese AMI patients, may not provide AMI incidence estimates that fully represent what occurs at the population level [4].

Tianjin, the third largest city (population 15.6 million) in China, has been routinely collecting incidence data of major non-communicable diseases (NCDs) including AMI and stroke through the Incidence Surveillance System established in 1984. In 2007, Tianjin incorporated the NCDs Incidence Surveillance component into the Hospital Information System (HIS) of pilot hospitals, reporting new cases directly via HIS rather than exclusively relying on conventional manually reporting cards. The derived incidence estimates have been utilized in burden of disease studies, despite the fact that the quality of the data—including evaluations of data completeness and consistency with external datasets or other relevant epidemiological indicators—has not be investigated [10,11,12].

Originally developed for the Global Burden of Disease studies, DisMod II has been extensively used to supplement observational data and assess internal consistency [2]. DisMod II is a multistate life table that describes the transitions between the disease states “health”, “diseased” and “dead” by using the transition rates from incidence to remission to case fatality [13]. By solving a set of linear differential equations, DisMod II can estimate age-specific incidence as well as prevalence of a disease given sufficient data on other disease variables. Compared with DisMod, an Incidence–Prevalence–Mortality model that requires three transition hazards incidence (from health to disease), remission (from disease to health) and case fatality (from disease to cause-specific death) as inputs, DisMod II allows for a wider range of input variables including incidence as a population rate, prevalence, duration, and mortality. Further, the application of DisMod II in estimating the incidence of a disease does not require multiple independent incidence data sources as prerequisites, a limitation inherent in the capture-recapture approach.

In this study, we examined the completeness and consistency of the reported AMI and stroke incidence rate by comparing the observed rates with model-based predicted incidence rates, and assessed if the completeness of reported incidence in Tianjin has improved since the introduction of the direct automatic reporting through HIS.

Methods

Data

Incidence rate

The age- and sex specific incidence rate data have been published elsewhere and are publicly available [10, 14, 15]. Incidence cases in published datasets were originally extracted from the Incidence Surveillance System in Tianjin, which was initially launched for a community-based program on the prevention and control of major NCDs including AMI and stroke [12]. The study subjects are residential registered cases who have experienced AMI or stroke, diagnosed by a hospital or clinic in 2007, 2010 or 2015. Cases were included in the dataset if they met all the following inclusion criteria:

• 1. Stroke detected and diagnosed by a medical practitioner within 28 days of onset;

• 2. AMI/stroke onset within the study period. Recurrence within 28 days was not included in incidence calculations, and recurrence after 28 days was considered as a new case;

• 3. Registered permanent (Hukou) resident of Tianjin at the time of AMI/stroke onset. Hukou based registration a household registration record officially identifies a person as a permanent resident of an area and includes identifying information such as name, parents, spouse and date of birth.

The recording process and quality control of the data have been described elsewhere [16]. New case reporting cards were required to be completed or entered into the Non-Communicable Disease Incidence Surveillance System by clinicians in hospitals or community clinics in Tianjin. The records include gender, date of birth, the diagnosis and diagnostic basis, date of events, job type, insurance status, smoking status, and district of residence. In 2007, the Tianjin CDC (Center for Disease Control and Prevention) incorporated the NCDs incidence surveillance component into the HIS of pilot hospitals [17]. By 2015, over 70% of the new cases from the chronic non-communicable incidence surveillance system were reported directly via HIS [18]. New cases in the surveillance system, either reported by cards or via HIS, were verified through various methods including regular training for doctors and checking daily reports for recurrence, logical errors and codes on a case-by-case basis at three levels (hospital, district and municipal CDC).

Incidence classification was coded in accordance with the International Classification of Disease, 10th Edition (ICD-10). The diseases used as incidence outcome measures in this study are AMI (ICD-10, I21) and stroke (ICD-10, I60-I64). Definitions included patients with symptoms, and imaging, laboratory and clinical examinations. Hemorrhagic stroke was defined as a stroke event with the diagnosis of subarachnoid hemorrhage or intracerebral hemorrhage, and ischemic stroke was defined as a stroke event with the diagnosis of thrombosis or embolism (hemorrhagic stroke ICD-10: I60, I61, I62; ischemic stroke: I63).

Cause-specific mortality rate

The age- and sex specific mortality rate of AMI and stroke for the year 2007, 2010 and 2015 were extracted from published reports and datasets [15, 19]. The original data were obtained from the all-cause mortality surveillance system, which monitors the entire residential population of the city. Deaths were ascertained through the procedures that have been proposed to be used in the Disease Surveillance Points in China. The recording process and quality control of the data have been described elsewhere [16, 20]. In brief, practicing clinicians from hospitals or community clinical centers completed death certificates and submitted them to the mortality surveillance system. Trained community clinicians (community health workers or village doctors) investigated the underlying causes of non-hospital deaths by interviewing relatives of the deceased and by reviewing available medical records on a door-to-door basis. The district and municipal CDCs oversee and check the quality of death certificates at the primary and secondary levels, respectively. The municipal CDC also provides technical training and support to staff involved in the surveillance process [20]. Cause of death classification for the study period was based on ICD-10.

Prevalence, remission rate and relative risk

Data on the prevalence, remission rate and relative risk of AMI and stroke in Tianjin were taken from the Global Burden of Disease (GBD) and Local Burden of Disease (LBD) studies conducted by the Institute for Health Metric and Evaluation (IHME). These epidemiological estimates were modeled using the DisMod-MR 2.0, a Bayesian mixed-effects meta-regression modeling technique developed for IHME’s GBD/LBD studies [21]. The estimates are initially made at the global level then sequentially revised down to the national and subnational levels using progressively more detailed data.

The prevalence, expressed as a proportion, is the number of cases in a population at a moment in time. The remission rate is the number of cases that resolve or are cured per person-year among patients. The incidence rate is the number of new cases per person-year. Stroke events are categorized into four groups in IHME’s dataset: acute/chronic ischemic stroke, and acute/chronic hemorrhage stroke. We used the sum of the prevalence of each stroke subtype as the prevalence of all stroke, and the weighted mean of the remission rates/relative risks as the remission rate/relative risk of all stroke. The percent weight given to each subtype was calculated as its fraction of all stroke prevalence. As the estimates from GBD studies were not produced on a yearly basis, we utilized the 2016 estimates and the mean of the estimates for the year 2005 and 2010 as the approximation for inputs of DisMod II for the year 2015 and 2007, respectively.

Population structure

Single-year permanent population estimates by sex and five-year age groups were taken from the Residence Registry Section of the Municipal Bureau of Public Security. We used WHO World Standard Population based on world average population between 2000 and 2025 for the purpose of calculating age standardized rates. Ethical approval for this study is not required because all data sources are publicly available aggregated data.

Model setting and analysis

The raw input data and population data were recorded by sex and 5-year age groups. To produce smooth rates for the modeled outputs, DisMod II first interpolates the input data into single age estimates using a cubic spline model for all input data (mortality rate, prevalence, remission rate and relative risk). Figure 1 shows the model input data (both original and smoothed) by age for AMI in men in 2007.

Fig. 1

The data inputs for the DisMod II: actual and smoothed AMI prevalence, remission and mortality rate for male in 2007. Relative risk is not presented in the graph given the scale of y-axis

Analyses were conducted with and without accounting for trends in incidence rates. When trends were accounted for, an annual change in the AMI incidence rate of − 2% for both males and females, and an annual change in the stroke incidence rate of 3% and 2% for males and females, respectively, since the year 2000 were incorporated into the models. These trends were based on studies of AMI and stroke trends in Tianjin [10, 12, 22, 23]. We conducted DisMod II uncertainty analysis to assess the 95% confidence interval of the modeled incidence estimates. In the uncertainty analysis, DisMod II conducts a bootstrapping exercise where the input variables are assumed to follow a specific distribution. We allowed the estimates of AMI and stroke prevalence to vary with a normal distribution and set the number of bootstrap iterations to 100. Analyses of uncertainty for models that included trends did not converge under our modeling framework; uncertainty estimates in these cases were not provided.

We calculated the proportion of incidence cases ascertained by the incidence surveillance system (completeness of reporting) as the observed incidence rate divided by the modeled incidence rate accounting for trends (the incidence rate ratio, IRR). We tested if the IRR increased from 2007 to 2010 and 2015 using one-sided Z-test. All the statistical tests were performed with R Version 3.5.3 using a 0.05 significance level. The RECORD (REporting of studies Conducted using Observational Routinely-collected health Data) guidelines were followed. DisMod II analytical software tool is freely available for use and can be downloaded from http://www.epigear.com/index_files/dismod_ii.html. The datasets analyzed in the study are available from the corresponding author upon request.

Results

23,371 AMI incident cases and 152,394 stroke incident cases were ascertained by Tianjin’s incidence surveillance system for the three years, which constituted an age-standardized incidence rate of 51 (male 65, female 36) and 316 (male 389, female 242) per 100,000 population per year for AMI and stroke, respectively. Overall, 92.4% of the AMI incident cases were diagnosed based on electrocardiography (ECG) and/or imaging examination including ultrasound test and coronary angiography (CAG), and 93.7% of the stroke incident cases were diagnosed based on CT or MRI scanning. The overall observed AMI incidence to mortality rate ratio was 0.90 (95% CI 0.88–0.91) from 2007 to 2015, and observed stroke incidence to mortality rate ratio increased significantly from 2.90 (95% CI 2.84–2.96) in 2007 to 7.71 (95% CI 7.54–7.90) in 2015 (Additional file 1: Figure S1).

Table 1 shows that AMI and stroke incidence rates, both reported and modeled, are higher (p < 0.0001) in men than in women, and are higher (p < 0.0001) in older age groups than in younger age groups. The observed AMI incidence rate showed a monotonic downward trend from 2007 to 2015 for both men and women, and for the majority of age groups. The observed stroke incidence rate showed a monotonic upward trend for both men and women, and for all the age groups, with the upward trend becoming more pronounced after 2010. An overall decrease in modeled AMI incidence and increase in modeled stroke incidence rates, including estimates that both did and did not account for temporal trends, were found among men and women for the majority of age groups. The age and sex standardized modeled incidence per 100,000 person-years decreased (p < 0.0001) from 138 (95% CI 125–145) in 2007 to 119 (95% CI 113–125) in 2015 for AMI and increased (p < 0.0001) from 520 (95% CI 450–582) in 2007 to 534 (95% CI 534–618) in 2015 for stroke.

Table 1 Number and rate of AMI/stroke incidence in Tianjin, 2007, 2010 and 2015

Figure 2 presents the age-specific observed and modeled incidence rates for AMI and stroke in 2007 and 2015. The figures show that the reported incidence rates are significantly lower than the modeled estimates for both men and women. The trended estimates from DisMod II for stroke incidence is higher than the non-trended estimates. The overall ratio of observed to modeled incidence was approximately 38.8% and 57.5% for AMI and stroke, respectively. Table 2 presents the sex and age-group specific ratio of observed to modeled incidence rate, both trended and non-trended estimates, for AMI and stroke. For AMI, there is no significant difference between trended and non-trended ratios, for either men or women, and either younger or older groups. While for stroke, the trended ratio is lower (p < 0.05) than non-trended one for both men and women, and both younger and older groups. The trended ratio for AMI increased significantly from 2007 to 2015 in the 30–59 age group for both men (0.64 vs 0.73, p = 0.026) and women (0.29 vs 0.44, p < 0.0001). The trended ratio for stroke increased in both the younger (30–59) age group (0.48 verse 0.78 for men, and 0.36 verse 0.55 for women, all p < 0.0001), and the older (≥ 60) age group (0.50 verse 0.61 for men, and 0.55 verse 0.65 for women, all p < 0.0001).

Fig. 2

Age specific incidence rates of AMI and stroke in Tianjin, 2007 and 2015. Note: The line for the modeled AMI incidence and that for the modeled AMI incidence accounting for trends virtually overlap

Table 2 Ratio of reported to modeled* incidence rate of AMI and stroke in Tianjin, 2007, 2010 and 2015

For those aged 30 years or above, the ratio was higher in men than in women, for both AMI and stroke incidence, and for both younger subgroup (age 30–59) and older subgroup (age ≥ 60), except for stroke incidence among the older age group (age ≥ 60). For the 60-plus age group, the ratio for stroke was consistently higher (p < 0.0001) than that for AMI for both men and women. For the 30–59 age group, the ratio for stroke was lower (p < 0.0001) in 2007 and 2010, but higher (p = 0.058) in 2015 than that for AMI. Figure 3 illustrates the age-specific ratio of reported to modeled incidence rate for AMI and stroke by sex and year. It shows that the increases in the ratio for both AMI and stroke occurred primarily during the 2010 and 2015 period.

Fig. 3

Age-specific ratio of reported to modeled incidence rate (trended) of AMI and stroke by sex and year. Red, green and blue solid lines represent estimates for 2007, 2010 and 2015, respectively. Dashed lines are upper and lower boundaries of the 95% confidence intervals

Discussion

This is the first published study that aims to estimate the completeness of routine incidence surveillance for cardiac and cerebrovascular diseases in Tianjin. Our findings indicate that the overall case ascertainment was approximately 39% and 58% for AMI and stroke, respectively. Eliminating trends in the incidence rate of stroke in the DisMod II settings resulted in observed and estimated stroke incidence being less divergent at both younger and older ages than without trends. From 2007 to 2015, an overall increase in the estimated completeness for both AMI and stroke was observed, predominantly attributed to the improvement in the case reporting among the population aged 60 years or younger after the year 2010.

The relatively low overall completeness of routine incidence reporting under passive surveillance in Tianjin could be attributed to several major reasons. First, neither cause-specific mortality data nor healthcare claim data has been linked to the incidence surveillance of the two diseases. Passive incidence registry is often unlikely to capture incident cases that lead to sudden deaths before patients reach health facilities. The average time from AMI symptoms onset to hospital arrival is 4 h in China [24]. In Tianjin, over 70% of AMI and stroke deaths occur at home [25], indicating that successfully identifying and deriving death certificates cases is important to supplement incidence surveillance data. In addition, the incidence registry in Tianjin fails to record incidence cases in instances where healthcare is either not sought, or is accessed in health facilities out of Tianjin. Healthcare claim data, routinely collected by the Municipal Bureau of Human Resources and Social Security, are important to AMI and stroke incidence surveillance as a means of finding cases not captured by local health facilities. Second, patients with minor events may be less likely to seek medical care urgently and are more often coded as a non-specific “other” diagnosis in the administrative database of health facilities. The proportion of patients with minor stroke is higher than reported in even high-quality stroke incidence studies and the underestimation is likely to be much greater in incidence surveillance from low-to-middle income countries, where healthcare utilization is often low [26]. However, it’s impossible to estimating the number of minor cases in Tianjin is problematic since incidence rates stratified by disease severity were not presented in our dataset. Lastly, unlike death certificates that are required by law to be completed and submitted by practicing clinicians from health facilities, AMI and stroke incidence reporting is not mandatory in Tianjin [12]. Although local health workers have been trained to submit accurate and timely case reports, the reporting frequency of health workers and/or facilities has not been continuously monitored. The passive surveillance in Tianjin relies heavily on an extensive network and the cooperation of health workers at all levels of facilities. It is difficult to assure the completeness and timeliness of data submission without individualized feedback or incentives for health workers and health facilities. In their absence, the distributions of monthly incident cases by health facility appear to demonstrate spurious fluctuations and zero reporting during the study period [25].

Our results suggest that the estimated completeness of reporting of AMI and stroke incidence was lower in female and older population, which could, in part, be explained by behavioral, diagnostic and pathological factors. The age of patients in China is negatively associated with the decision to seek health care [27, 28], likely owing to limited access to healthcare services and health insurance availability. Women in China are less likely to seek health care due to higher sensitivity to cost, social power relations and inherent inequality [27, 29]. It has been reported that un-witnessed sudden cardiac death is more likely to occur among women and older people [30] due to unusual pathophysiological mechanisms [31, 32], lower awareness of the warning signs and symptoms [33] and a higher chance of being unemployed and/or living alone. This hypothesis is supported by evidence that a higher proportion of AMI and stroke deaths for women and older people in Tianjin occur at home [25]. In addition, women have less typical symptoms than men and women with minor stroke or transient ischemic attack are less likely to be diagnosed with a stroke despite having similar symptoms at presentation [34].

The improvement in the estimated completeness could be attributed to several factors. First, Tianjin started to incorporate NCDs Incidence Surveillance program into the HIS of some pilot hospitals in 2007, before which new incident cases were exclusively reported through manually completed paper cards. By 2015, over 70% of new cases from the chronic non-communicable incidence surveillance system were reported directly via HIS. The automatic reporting component embedded in HIS significantly reduced the workload of health workers, improving the timeliness and continuity of routine case reporting. Second, out-of-pocket costs are associated with prolonged time in care-seeking and hospital arrival after the onset of cardiovascular disease in China [24]. China’s health care system reform beginning in 2009 has rapidly expanded health insurance coverage, significantly reducing the share of out-of-pocket expenditure and increasing the accessibility and timely utilization of healthcare services [35]. Increased access to and utilization of healthcare services allows the primary care computer systems to systematically identify patients with cardiovascular and cerebrovascular disease [36].

Extensively implemented secondary and tertiary prevention programs and campaigns in recent years may have further increased the likelihood of AMI and stroke patients being captured by the surveillance system. In 2011, Tianjin initiated the Stroke Screening and Prevention Project that screens for eight major risk factors of stroke and recommends labs and imaging tests to be performed on the basis of personal risk estimation [37, 38]. This proactive community-based free screening program has enhanced the public’s awareness of the warning signs and symptoms of stroke, resulting in more timely and appropriate health-seeking decisions. Consequently, the screening project has increased the chance of being diagnosed and coded as having stroke for patients at high risk and patients with minor stroke events [39]. Since 2014, Tianjin has gradually established chest pain centers and a chest pain rescue network that aims to optimize the diagnosis and treatment processes for patients with acute chest pain, especially AMI [40]. Increased access to urgent treatment and reduced hospital arrival time would lower the proportion of uncaptured incident patients who die at home or on the way to healthcare facilities. However, linking cause-specific mortality to the incidence surveillance of the two diseases is warranted to ascertain patients that were missed in the information system of health care facilities.

This study has limitations. First, we assumed that the estimate from DisMod II reflects the true incidence rate and is the “gold standard” for measuring the completeness of reported incident events. Results from some previous studies have shown that the DisMod II estimates for both men and women were very similar to estimates derived from the routine health information system or population health survey [3, 41]. Scarborough et al. compared DisMod II estimates of age-specific incidence rates for AMI with those observed in the external dataset from England [42]. The DisMod II model estimates were unable to replicate age-specific incidence rates of AMI derived from a population-based study, although they were of similar magnitude. Our estimate of the completeness of case reporting is likely to be biased if DisMod II estimate is an imperfect “gold standard” and the inputs, including mortality rate, for DisMod II are not accurate. Second, the lack of any other rigorous research on the prevalence of AMI/stroke, especially the relative risk of mortality and remission of patients with AMI/stroke in Tianjin, required that we utilize the epidemiological estimates provided by IHME’s mixed-effects meta regression modeling using the DisMod-MR 2.0. The accuracy of these local estimates has not been previously investigated. However, importantly, these estimates are internally consistent with the observed mortality rate derived from local all-cause mortality surveillance. Third, the ideal input data for the DisMod II would be estimates of the increased all-cause mortality for people who have had AMI or stroke. Given the lack of direct measures, we used mortality data where AMI or stroke was indicated as the underlying primary cause of death. This metric does not account for increased mortality risk from other conditions (e.g. increased risk of respiratory disease and peripheral vascular disease) [42]. Lastly, under the constraint of data availability, we were not able to extend the analysis to more recent settings, which may presumably dilute the full impacts of the direct automatic electronic reporting system.

Our model is open to further research and validity investigation. Reliable incidence data from extensive prospective follow-up studies and other independent incidence data sources with information provided at the individual level that could be linked to elements of CDC’s incidence surveillance data to supplement the assessment of completeness of routine reporting. Moreover, alternative prevalence and remission rate data will help to further confirm the validity of our findings.

Conclusions

Our findings indicate that the reporting of AMI and stroke incidence in Tianjin has been incomplete. However, the completeness of the surveillance has been improving since 2010 primarily owing to the incorporation of an automatic reporting component into the information systems of health facilities, the increase in the utilization of healthcare service and campaigns promoting access to prevention services and timely emergency treatment.