Evidence Synthesis (Systematic Reviews): Overview

"Evidence synthesis uses formal, explicit, and rigorous methods to bring together the findings of research already completed, to provide an account of the totality of what is known from that pre-existing research." 1

A systematic review is a type of evidence synthesis that evaluates the "evidence on a clearly formulated question that uses systematic and explicit methods to identify, select, and critically appraise relevant primary research, and to extract and analyze data from the studies that are included in the review." The methods used must be reproducible and transparent. 2

Evidence synthesis requires:

• A clearly formulated question
• Transparent and reproducible methods of identifying, categorizing, and analyzing the literature
• A comprehensive search of the literature 
• Explicit inclusion and exclusion criteria
• Efforts to reduce bias in study selection and reporting

1. Gough D, Davies P, Jamtvedt G, et al. Evidence Synthesis International (ESI): Position Statement. Systematic Reviews. 2020;9(1):155. doi:10.1186/s13643-020-01415-5

2. Wright RW, Brand RA, Dunn W, Spindler KP. How to write a systematic review. Clin Orthop Relat Res. 2007;455:23-29. doi:10.1097/BLO.0b013e31802c9098

• Type of review most appropriate for your topic
  • A systematic review might not be the most appropriate type of review to answer your question. See the table on the right for information on different types of evidence synthesis reviews. 
• Timing
  • Evidence synthesis is time consuming, and most projects take between 6-18 months to complete
    • Try the PredicTER tool to see how long the steps of a typical systematic review may take.
• Study planning
  • Evidence synthesis is a rigorous process, and best practice recommends writing and registering a research protocol detailing your research question and the methods you intend to follow to answer that question. 
• Resources
  • Evidence synthesis should be conducted by a team of researchers. Most guidelines recommend that at least three people participate in screening and data extraction.
  • Literature searching for evidence synthesis can be quite complex as the goal is to identify all published studies on the topic of interest. This requires searching in multiple resources using a combination of controlled vocabulary and keywords. Consult with a librarian or include one on your team for best results.
  • You will need access to software for citation management and to assist with screening, data extraction, and quality analysis.
  • Did I mention the amount of time it takes to conduct proper evidence synthesis? Make sure you allocate an appropriate amount of time to each task. 

Although systematic reviews are the most recognized type of evidence synthesis, there are many other types of evidence reviews. Elements like the size of your team, timeline for the project, and specificity of your research question can help determine the type of review that best serves your needs.

Source: Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Information & Libraries Journal. 2009;26(2):91-108. doi:10.1111/j.1471-1842.2009.00848.x

Not sure which type of review is best for your project? Contact a librarian, or try the Right Review, a tool that can help you select one of 41 types of evidence synthesis across many disciplines.   

1. Develop a Research Topic

• The PICO (Population, Intervention, Comparator/Comparison, Outcomes) help to clarify the critical elements of a key question
• A research question might be refined after performing a preliminary literature search
   

2. Define Inclusion and Exclusion Criteria

• Define which subjects or studies will be included into the review
• Determined by the research question
• Need to be defined before the search is conducted
   

3. Search the Literature

      More details are provided in Literature Search

4. Select Studies

• Usually conducted in two passes
      i. Review title and abstract
      ii. Review full text
• At least two independent reviewers with a third person availabe for conflicts
• Keep a log of excluded studies with reasons for exclusion
   

5. Assess study quality 

• Each study meeting the inclusion criteria is accessed for quality
• More studies may be excluded after this step
• Must be documented
   

6. Extract Data

• Reviewers will extract data from the included documents to be analyzed either qualitatively or quantitatively
• Reviewers may develop standardized forms
   

7. Analyze and Present Results

• Either qualitative or quantitative (Meta - Analysis)
• May have to perform sensitivity analyses and examine funnel and forest plots
   

8. Interpret Results

Reviewers should comment on:

• Strength of the evidence
• Applicability of the results
• Benefits/costs/tradeoffs
• Limitations
• Implication for future research
   

9. Update the Review as Needed

New research findings can quickly make Evidence Synthesis reviews out of date. 

Search "systematic review" in Sage Research Methods database