Evidence Synthesis (Systematic Reviews): Overview of Evidence Synthesis

"Evidence synthesis uses formal, explicit, and rigorous methods to bring together the findings of research already completed, to provide an account of the totality of what is known from that pre-existing research." 1

A systematic review is a type of evidence synthesis that evaluates the "evidence on a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review." The methods used must be reproducible and transparent. 2

Evidence synthesis requires:

• A clearly formed question
• Transparent and reproducible methods of identifying, categorizing, and analyzing the literature
• A comprehensive search of the literature 
• Explicit inclusion and exclusion criteria
• Efforts to reduce bias in study selection and reporting

1. Gough D, Davies P, Jamtvedt G, et al. Evidence Synthesis International (ESI): Position Statement. Systematic Reviews. 2020;9(1):155. doi:10.1186/s13643-020-01415-5

2. Wright RW, Brand RA, Dunn W, Spindler KP. How to write a systematic review. Clin Orthop Relat Res. 2007;455:23-29. doi:10.1097/BLO.0b013e31802c9098

Type of review most appropriate for your topic

• A systematic review might not be the most appropriate type of review to answer your question. See the table on the right for information on different types of evidence synthesis reviews. 

Timing

• Evidence synthesis is time consuming, and most projects take between 6-18 months to complete
  • Try the PredicTER tool to see how long the steps of a typical systematic review may take.

Study planning

• Evidence synthesis is a rigorous process, and best practice recommends writing and registering a research protocol detailing your research question and the methods you intend to follow to answer that question. 

Resources

• Evidence synthesis should be conducted by a team of researchers. Most guidelines recommend that at least three people participate in screening and data extraction.
• Literature searching for evidence synthesis can be quite complex as the goal is to identify all published studies on the topic of interest. This requires searching in multiple resources using a combination of controlled vocabulary and keywords. Consult with a librarian or include one on your team for best results.
• You will need access to software for citation management and to assist with screening, data extraction, and quality analysis.
• Did I mention the amount of time it takes to conduct proper evidence synthesis? Make sure you allocate an appropriate amount of time to each task. 

1. Develop a Research Topic

• The PICO (Population, Intervention, Comparator/Comparison, Outcomes) help to clarify the critical elements of a key question
• A research question might be refined after performing a preliminary literature search

2. Define Inclusion and Exclusion Criteria

• Define which subjects or studies will be included in the review
• Determined by the research question
• Need to be defined before the final search is conducted

3. Write and Register a Protocol

• Outlines the methods and approach for conducting the review including eligibility criteria, data sources, and planned analysis
• A registered or published protocol ensures transparency and minimizes bias
• Often a requirement for publication 

4. Search the Literature

• Search at least three bibliographic databases
• Use controlled vocabulary and keywords 
• More details are provided in Literature Search tab

5. Select Studies

• Usually conducted in two passes
      i. Review title and abstract
      ii. Review full text
• At least two independent reviewers with a third person available to resolve conflicts
• Keep a log of excluded studies with reasons for exclusion

6. Assess study quality 

• Each study meeting the inclusion criteria is assessed for quality
• More studies may be excluded after this step
• Must be documented

7. Extract Data

• Reviewers will extract data from the included documents to be analyzed either qualitatively or quantitatively
• Reviewers may develop standardized forms

8. Analyze and Present Results

• Either qualitative or quantitative (Meta - Analysis)
• May have to perform sensitivity analyses and examine funnel and forest plots

9. Interpret Results

Reviewers should comment on:

• Strength of the evidence
• Applicability of the results
• Benefits/costs/tradeoffs
• Limitations
• Implication for future research

10. Update the Review as Needed

• New research findings can quickly make Evidence Synthesis reviews out of date.