Standardised reporting of burden of disease studies: the STROBOD statement

Abstract

Background

The burden of disease (BOD) approach, originating with the Global Burden of Disease (GBD) study in the 1990s, has become a cornerstone for population health monitoring. Despite the widespread use of the Disability-Adjusted Life Year (DALY) metric, variations in methodological approaches and reporting inconsistencies hinder comparability across studies. To tackle this issue, we set out to develop guidelines for reporting DALY calculation studies to improve the transparency and comparability of BOD estimates.

Methods and Findings

The development of the STROBOD statement began within the European Burden of Disease Network, evolving from initial concepts discussed in workshops and training sessions focused on critical analysis of BOD studies. In 2021, a working group was formed to refine the preliminary version into the final Standardised Reporting of Burden of Disease studies (STROBOD) statement, consisting of 28 items structured across six main sections. These sections cover the title, abstract, introduction, methods, results, discussion, and open science, aiming to ensure transparency and standardization in reporting BOD studies. Notably, the methods section of the STROBOD checklist encompasses aspects such as study setting, data inputs and adjustments, DALY calculation methods, uncertainty analyses, and recommendations for reproducibility and transparency. A pilot phase was conducted to test the efficacy of the STROBOD statement, highlighting the importance of providing clear explanations and examples for each reporting item.

Conclusions

The inaugural STROBOD statement offers a crucial framework for standardizing reporting in BOD research, with plans for ongoing evaluation and potential revisions based on user feedback. While the current version focuses on general BOD methodology, future iterations may include specialized checklists for distinct applications such as injury or risk factor estimation, reflecting the dynamic nature of this field.

Introduction

The burden of disease (BOD) approach gained prominence in the 1990s, with the launch of the first Global Burden of Disease (GBD) study. Today, the Institute for Health Metrics and Evaluation has transformed the GBD study into a worldwide collaborative effort, generating estimates for 371 diseases and injuries, and 88 risk factors, in 204 countries and territories [1]. Furthermore, several countries are conducting national BOD studies, and several international organisations have embraced the BOD approach [2,3,4,5]. Central to this approach is the Disability-Adjusted Life Year (DALY), which allows integrating the impact of morbidity and mortality into a single population health indicator, thereby providing a more comprehensive basis for evaluating, comparing, and ranking the health impact of diseases, injuries, and risk factors. Motivated by the strength of the DALY metric, it is also increasingly used in cost-effectiveness analyses, health impact assessments, and quantitative microbial risk assessments [6, 7].

Although the general concept of the DALY metric is well established, there are various methodological choices and assumptions to be made when calculating DALYs [6, 8, 9]. Recent systematic literature reviews of European BOD studies have shown a wide variety in methodological assumptions used to quantify DALYs, but also important inconsistencies in the reporting of methods and particular assumptions [2, 10, 11, 11, 12]. For instance, studies differ in the choice of the reference life table for calculating Years of Life Lost (YLL) (e.g., aspirational versus national life expectancies [10, 11, 11]), in the use of specific sets of disability weights (e.g., GBD or national elicitation exercises [13, 14]), or in the application of social weighting functions (i.e., age weighting and time discounting [2, 11]). This methodological heterogeneity, and the lack of consistent reporting, hampers interpretation and comparability of BOD estimates, thereby limiting the power of the DALY metric as a tool for ranking and prioritisation. For instance, Wyper et al. [14] have documented that the choice of the reference population used in calculating age-standardised rates can have a major impact on the results, rankings, and conclusions.

To address this challenge, we introduce the Standardised Reporting of Burden of Disease studies (STROBOD) statement.

Aim and scope

The aim of the STROBOD statement is to serve as a standard protocol for reporting DALY calculations. The widespread application of the STROBOD statement will increase consistency and transparency in reporting of BOD studies, which will enhance usability of BOD estimates. The statement will furthermore serve as an educational tool, as it helps researchers and students to understand the different choices and assumptions that need to be made when calculating DALYs. The STROBOD statement is however not intended to serve as a tool to assess the quality of BOD studies or estimates.

The STROBOD statement focuses on the DALY methodology, and can be used in combination with other reporting guidelines. For instance, the Guidelines for Accurate and Transparent Health Estimates Reporting (GATHER) can be used to describe input data and estimation methods for new global health estimates [15]. Likewise, the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement can be used when reporting on health economic evaluations [16], of which a DALY calculation can be a subcomponent.

Development of the STROBOD statement

The basic idea of the STROBOD statement was proposed within the context of the European Burden of Disease Network (COST Action CA18218) [17].

A preliminary version of the statement was developed by BD in the context of workshops and training schools on the general concepts of BOD, where it was used as a tool to support an exercise on critical reading of BOD studies. This version included basic items on study setting, data adjustments, DALY methods, and uncertainty. Throughout the workshops and training schools, the relevance of the tool was established.

In 2021, a working group was established to further develop this preliminary version into an initial version of the STROBOD statement. This working group consisted of the Core Group members of the European Burden of Disease Network, and met online on regular intervals throughout 2023. In initial meetings, the organisation of the working group was defined, and the scope, purpose and structure of the statement was discussed. It was decided to include a pilot phase and an additional appendix with a detailed explanation of each item, including examples. The scope was defined to be generic, thereby excluding specific methods involving, for example, injury BOD calculations or comparative risk factors (for which add-on statements could be developed). The purpose was defined to be descriptive rather than normative. The structure was defined to be similar to existing statements such as GATHER and CHEERS.

Based on the discussions in the working group, it was decided to add a number of items to the preliminary version, so that the STROBOD statement would cover all relevant aspects of a scientific manuscript or report. We thus included sections on the title, abstract, introduction, results, discussion, and open science. The decision to include the latter section was made to emphasise that future BOD studies should be both easily accessible and reproducible. Specific adjustments were made to the items in the methods section. Detailed questions on different data adjustment steps were merged into one item, and an extra item was added on the listing of epidemiological input parameters. Likewise, items were added on the durations and conditional probabilities used in disease model(s), while the preliminary version only included items on disability weights. Discussions in the working group mainly were linked to the description of the items in the statement table. The supplementary file with detailed explanations and examples provided a way to accommodate requests for further detail (S2 Appendix).

To test the initial protocol, a pilot phase was run by applying it to peer-reviewed published BOD papers. The pilot was conducted by postgraduate supervised students of the Core Group members, while the set of papers used in this test differed widely in scope and complexity. The test applications further highlighted the need to provide clear and detailed explanations and examples for each of the items.

The STROBOD statement

STROBOD comprises a checklist of 28 items (Table 1). For each item, the authors need to specify on which page the relevant information can be found. An electronic version of the checklist and a more detailed explanation and elaboration document, describing the interpretation and rationale of each reporting item along with examples of good reporting, are available in the appendix (S2 Appendix).

Table 1 STROBOD checklist of items that should be included in reports of Disability-adjusted life year calculations

The STROBOD reporting items are structured in six main sections – i.e., title, abstract, introduction, methods, results, discussion, and open science (S3 Appendix). The title needs to clearly identify the study as a BOD assessment, to facilitate retrieval via database searches. The abstract needs to present key findings, while the introduction needs to present the policy relevance of the study. Most of the items of the checklist relate to the study methods. Authors need to define the study setting (in terms of causes, reference population, and reference year), and describe all data inputs, including methods for data adjustments. Specific DALY calculation methods include the Years Lost due to Disability (YLD) perspective, valuation used in Years of Life Lost (YLL) calculations, disease model with disability weights, durations and severity distributions, and possible multimorbidity adjustments and social weighting. Uncertainty analyses are recommended, and include assessing parameter uncertainty and performing scenario and sensitivity analyses. The results should be presented in a way that supports comparability, and should be discussed in light of previous evidence, in particular focussing on differences arising through different DALY methods or data manipulations. Finally, it is recommended to make the source code available, and describe funding source and conflict of interests.

Implications and limitations

This first version of the STROBOD statement provides a valuable new resource for BOD researchers and experts. We will actively promote its uptake and use, so that it can become a standard in the field. It is also intended that the STROBOD statement will be evaluated over time, and possibly revised in light of user experiences.

The STROBOD statement focuses on general BOD methodology. Specific applications, such as the estimation of DALYs for injuries or risk factors, require dedicated checklists. For instance, injury BOD studies are typically based on an integration of information on the cause-of-injury, which pertains to the intent and mechanism of injury, and the nature-of-injury, which pertains to the type of injury and the severity of their consequences [11]. Likewise, BOD studies for risk factors require additional information on the exposure to the concerned risk factor, and the associations with the concerned health outcomes. The integration of these additional inputs come with additional methodological challenges [18]. To address these specific needs, it is foreseen that the STROBOD statement can form the basis for add-on checklists dedicated to these specific applications. Although the STROBOD statement is intended for studies estimating DALYs, we encourage that studies exclusively estimating YLL, or YLD, can also adhere to the statement by completing the relevant checklist items. In addition to these add-on checklists, we also foresee that the STROBOD statement can be used in complement to other statements, such as GATHER [15]. Indeed, STROBOD has a specific focus on the DALY metric, while GATHER applies to any health metric—STROBOD thus provides specific detail on the methodological choices and assumptions specific to the DALY, which is a level of detail that exceeds the scope of GATHER. Vice versa, GATHER has a stronger focus on the statistical modelling techniques used to generate coherent global health estimates, which exceeds the scope of STROBOD.

The STROBOD statement supports reporting of BOD studies, but does not provide practical guidance on how to perform a BOD study or translate the results to end users. To understand the practical steps in a BOD study, researchers may refer to Devleesschauwer et al. [9] and also benefit from existing tools such as the software tool for burden of infectious disease calculations [19]. Knowledge translation of BOD estimates is an area of active development, and is described in more detail by Lundkvist et al. [20] and Cuschieri et al. [21].

The STROBOD statement was developed within the context of the European Burden of Disease Network, with specific inputs from the network’s Core Group. As a consequence, inputs from BOD experts from other continents were not sought, nor was the broader scientific community consulted. Despite the significant variability in the application of BOD methodology, we did not discern any distinct geographical patterns in this diversity. This suggests that our current working group possessed a sufficiently comprehensive perspective on BOD methodology and reporting requirements. Moreover, the decision to develop the statement exclusively with BOD experts was purposeful, given the specialized nature of this methodology within the broader scientific community. Nevertheless, we eagerly anticipate the implementation of the statement in the field, its incorporation into publications, and its use in future training initiatives.

To encourage the adoption of the STROBOD statement, we will proactively engage with the EQUATOR network and editors of relevant scientific journals to advocate for its endorsement as a standard practice in BOD studies. Future activities will also include an application of the statement to published papers, to structurally assess the completeness and quality of reporting of published BOD papers—and to assess how well the current version of the statement is understood by researchers. A first application of the statement to the GBD study (see S4 Appendix) highlighted that certain methodological choices, such as social weighting factors—explicitly excluded from GBD studies—may still hold relevance for national BOD studies and should be reported accordingly. Additionally, the use of the STROBOD statement underscored the importance of thoroughly documenting methodological decisions, including data sources and adjustments, as it significantly improves the transparency and clarity of BOD estimates. We are committed to learning from these experiences, and fostering ongoing discussions with experts and institutions from around the world. Based on these inputs, we anticipate making necessary revisions to the statement, and publishing improved versions of the statement, and extensions to specific topics.