Table 3 Definitions of metrics referred to in this paper

Question endorsement ratio

The proportion of “yes” responses for a given question.

\( \frac{1}{2}*\ \left(\frac{\left( Yes\ survey\ 1\right)}{Total\ \#\ questions} + \frac{\left( Yes\ survey\ 2\right)}{Total\ \#\ questions}\right) \)

Question proportion agreement*

The proportion of questions for which first and second verbal autopsy survey rounds were consistent (p₀)

\( {p}_0=\frac{\#\ \left( Yes\ survey\ 1,\ Yes\ survey\ 2\right) + \#\ \left( No\ survey\ 2,\ No\ survey\ 1\right)}{Total\ \#\ questions} \)

Question kappa

Proportion question agreement (p₀) adjusted by proportion expected agreement (p_e) for first and second survey rounds

\( \mathrm{kappa}=\frac{p_o-{p}_e}{1-{p}_e} \)

Question proportion gain

The proportion of questions with a “no” response in the first survey round reported as a “yes” response in the second survey round.

\( Proportion\ Gain = \frac{\left( No\ survey\ 1,\kern0.5em Yes\ survey\ 2\right)}{Total\ \#\ Yes\kern0.5em survey\kern0.5em 2} \)

Question proportion loss

The proportion of questions with a “yes” response in the first survey round reported as a “no” response in the second survey round.

\( Proportion\ Loss=\frac{\#\ \left( Yes\ survey\;1, No\ survey\kern0.5em 2\right)}{Total\ \#\ Yes\ survey\ 1} \)

Decedent proportion agreement

Proportion agreement for all question responses for a given decedent

\( {p}_0=\frac{\#\ \left( Yes\ survey\ 1, Yes\ survey\ 2\right)+\#\left( No\ survey\ 2, No\ survey\;1\right)}{Total\ \#\ questions} \)

Decedent question kappa

Kappa for question response for a given decedent

COD proportion agreement

Proportion agreement for COD predictions for all decedents

\( {p}_0=\frac{{\displaystyle \sum \left( COD\ j\ survey\ 1, COD\ j\ survey\ 2\right)}}{Total\ \#\ decedents} \)

COD kappa

Kappa for COD predictions for a given decedent

Cause-specific proportion agreement

Proportion agreement for COD predictions for a given COD

\( {p}_{0,j}=\frac{{\displaystyle \sum \left(\left( COD\ j\ survey\ 1, COD\ j\ survey\ 2\right)+\left( Not\ COD\ j\ survey\ 1, Not\ COD\ j\ survey\ 2\right)\right)}}{Total\ \#\ questions} \)

Cause-specific kappa

Kappa for COD predictions for a given COD

Prediction match

Binary indicator. If predicted COD was the same for first and second VAs, if prediction match = 1; if not the same, prediction match = 0.

Chance-corrected concordance (CCC)

The sensitivity of a cause of death estimate adjusted for chance.

\( CC{C}_j=\frac{\left(\frac{T{P}_j}{T{P}_j+F{N}_j}\right)-\left(\frac{1}{N}\right)}{1-\left(\frac{1}{N}\right)} \)

where TP_j is true positives or number of decedents with gold standard cause j correctly assigned to cause j, FN is false negatives or the number of decedents incorrectly assigned to cause j, and N is the number of causes analyzed. TP plus FN equals the true number of deaths due cause j.