Critical elements of synthesis questions are incompletely reported: survey of systematic reviews of intervention effects

Objectives: To examine the characteristics of population, intervention and outcome groups and the extent to which they were completely reported for each synthesis in a sample of systematic reviews (SRs) of interventions. Study design and setting: We coded groups that were intended (or used) for comparisons in 100 randomly sampled SRs of public health and health systems interventions published in 2018 from the Health Evidence and Health Systems Evidence databases. Results: Authors commonly used population, intervention and outcome groups to structure comparisons, but these groups were often incompletely reported. For example, of 41 SRs that identified and/or used intervention groups for comparisons, 29 (71%) identified the groups in their methods description before reporting of the results (e.g., in the Background or Methods), 12 (29%) defined the groups in enough detail to replicate decisions about which included studies were eligible for each synthesis, 6 (15%) provided a rationale, and 24 (59%) stated that the groups would be used for comparisons. Sixteen (39%) SRs used intervention groups in their synthesis without any mention in the methods. Reporting for population, outcome and methodological groups was similarly incomplete. Conclusion: Complete reporting of the groups used for synthesis would improve transparency and replicability of reviews, and help ensure that the synthesis is not driven by what is reported in the included studies. Although concerted effort is needed to improve reporting, this should lead to more focused and useful reviews for decision-makers.