An improved method for physician-certified verbal autopsy reduces the rate of discrepancy: experiences in the Nouna Health and Demographic Surveillance Site (NHDSS), Burkina Faso

Verbal autopsy (VA) is a technique used to determine the cause of death by asking caregivers, friends, or family members about signs and symptoms exhibited by the deceased in the period before death. It is usually done by trained fieldworkers using a standardized questionnaire that collects details on signs, symptoms, complaints, and any medical history or events prior to death [1].

The World Health Organization (WHO) recommends the use of verbal autopsy to measure specific causes of death [2, 3]. The purpose of verbal autopsy is to describe the causes of death at the community or population level where limited or no vital registration is completed with medical certificates. Indeed, medically certified cause of death data are available for less than one-third of the more than 57 million deaths occurring worldwide annually. The majority of deaths lacking such data are from developing countries [4]. Information about cause of death is essential for public health planning, priority setting, monitoring, and evaluation, but the collection of such information in countries with incomplete or no vital registration systems remains a substantial challenge [5]. Reliable data on cause-specific mortality is also needed by countries to keep track of progress toward the Millennium Development Goals [3, 6].

The use of physician-certified verbal autopsies (PCVA) is common in the majority of developing countries, as well as for Health and Demographic Surveillance Sites (HDSS) that are members of International Network for the Demographic Evaluation of Populations and Their Health in Developing Countries (INDEPTH) [6]. Within the INDEPTH Network, 36 HDSS in 20 countries regularly use VA to assess cause of death [6]. However, the data collection tools are not yet harmonized, which has led to substantial variability in the coding process across sites [7, 8]. Recently, there have been several attempts to introduce alternative methods such as a computer-based verbal autopsy coding method (InterVA) to replace the PCVA approach [6]. This probability-based method was tested in several settings [9–11]. However, the results still show some discrepancies in comparison to PCVA results [12]. Few studies are available on the use of different physician coding methods that produce better results. In contrast, the study by Joshi and colleagues comparing results involving multiple coders versus one single coder suggest that advantages attained from the multiple-coding system remain limited [13]. However, in this study, the approaches for cause of death assignment used either a panel of expert physicians or involved two or more physician coders who independently reviewed the data to arrive at a final diagnosis [14, 15]. The method proposed in this paper was tested with the intention to build on former approaches, such as those presented in the study by Joshi et al [13].

In this study we compare the usual WHO-recommended PCVA procedure with a locally-adapted method that incorporates the use of a panel of physicians after a discrepancy among three physician coders arises.

Our findings provide evidence that the choice of verbal autopsy coding method has a highly significant impact on the results of PCVA. It indicates that the improvement of an empirical method of PCVA, like the WHO-recommended method [3], through use of a physician panel in case of COD-coding disagreements, leads to a high reduction of the proportion of undetermined causes of death. Although this method of panel coding necessitates additional resources and time for physician coders as compared to the standard coding procedure described by Soleman et al [5], and especially to the single-coding procedure described by Joshi et al [13], it brings a large improvement in the existing methods determining the probable causes of death.

The findings of Fottrell et al [9] support our results, as an initial agreement of 60% among two physicians was shown to increase to more than 80% when a review is done by one additional physician. Thus, our findings are in stark contrast to those of Joshi and colleagues who suggest reducing coding to one physician only [9]. We cannot exclude the fact that the quality of coding might depend on the VA questionnaire used per HDSS site, as discussed in several INDEPTH meetings over the past five years. The current VA questionnaire available through WHO/INDEPTH tries to overcome limitations of existing VA questionnaires, offering separate versions for different age groups and providing comparability over different countries. Thus, NHDSS has moved to the updated WHO/INDEPTH questionnaire in 2011. For coding, the Nouna site uses the restricted classification list suggested during the INDEPTH Meeting in Uganda in 2008, comparable with other HDSS sites.

On one hand, our data possibly suggest that the new multicoding system of deaths doesn't necessarily affect the mortality pattern, although it results in changes in the proportion of deaths within the different groups of leading causes of death. Undeniably, the suggested method is more time-consuming and costly, but it is also more efficient. However, this is the first time that such a panel discussed the questionable cases. In summary, the procedure might be especially helpful in HDSS sites where high rates of undetermined CODs are observed.

The use of automated Bayesian models to assign the most likely causes of death tested by Byass [18] are currently under investigation in the Nouna HDSS. The main gains achieved from this method are a reduction in time and cost needed to complete the coding process. Additionally, as the model doesn't involve different physicians over time or in different countries, it aims to provide comparable results within HDSS sites over time and across different HDSS sites. However, its use is still limited to certain sites and the result comparison with PCVA approaches still shows some discrepancies in comparison to the PCVA results [12].

However, given that the computer-based probability approach to VA interpretation is designed to overcome the weakness of physicians' reviews, preliminary results are promising, but are not fully convincing [9–11]. At present, the main problem in choosing an optimal method of coding for VA is that no gold standard is available and comparison among various methods remains limited. Currently, both methods might profit from comparing their results with those by the other method.

For resource-poor settings, a reliable and affordable method of VA coding remains a necessity, as mortality data remain important to guide decision-makers for health planning purposes. While waiting to scale up use of the computer-based model, the improved WHO method proposed in our study could be applied as an alternative method for coding, as it offers a good rate of concordance among physician coders.

Despite verbal autopsy being a useful tool in determining causes of death, the method has some limitations. Previous studies note these shortcomings in detail [2, 5, 9, 13]. Because verbal autopsy is based on data collected through an interview process, and based on signs and symptoms exhibited, it is subject to recall bias and misreporting. Physicians have different experiences and knowledge in coding that could lead to different interpretations of the diagnosis [5, 9].

While PCVA has some well-known limitations [4], the shortcomings of the tool are known and quantifiable. These deficiencies, however, should not prevent countries requiring information on causes of death from benefiting from the use of VA when no practical alternative for obtaining these data exists. Few studies are available on the use of different physician coding methods that produce better results. However, the approaches for cause of death assignment most commonly used either a panel of expert physicians or two or more physician coders who independently review the data and arrive at a diagnosis [14, 15]. Despite its acknowledged limitations [13], PCVA is still considered the best possible method to get cause of death estimates in areas where vital events registrations systems are limited or not available.

Cohen's kappa as a measure of agreement couldn't be applied, as both samples used here were independent (years 2009 and 2010) and the coding was done independently in a blind manner by different physicians. Given this constraint, we focused only on the comparison of concordance rate between the two samples. This approach to analysis allowed us to attain a simple but effective measure for the improvement of the extended coding method.

There is a need for further study to confirm our findings in other settings. This will have the advantage of adopting a unique method for HDSS sites within the INDEPTH Network and to some extent to other sites outside the network interested in more accurate physician-certified verbal autopsy coding methods.

Verbal autopsy remains essential to capture and determine probable cause of death, especially in the context of low-income countries like Burkina Faso, where it is estimated that roughly 75% of deaths occur at home [19]. The VA process has the ability to contribute substantially by informing policymakers on real mortality data and allowing countries to monitor trends toward attainment of Millennium Development Goals, in particular those related to maternal and child health outcomes. The advantage of involving a physician panel in the coding process as suggested here is obvious, as it allows coding of an additional 50% of VAs. Importantly, this method promotes interactive discussions among physicians involved in the coding process, similar to what physicians are already doing during their clinical presentations on patients. Thus, the panel method provides a framework for scientific discussion among physicians, allowing everyone to update their knowledge.

Our study presents an alternative method of PCVA that substantially reduces the proportion of undetermined causes of death and therefore contributes to the death codification. We also aim to advocate for harmonization in the PCVA process, while encouraging the validation of the computer-based method of death coding.