Skip to Main Content

Evidence Synthesis, Systematic Review Services : Develop a Protocol

Why Develop a Protocol?

As with any other research project, your research team should write a detailed protocol for how your systematic review will be conducted very early in the systematic review process. This will be a plan of action that your team will follow. 

 A protocol will describe the following general elements:

  • Scope and rationale of the review
  • How the team will execute and document the search for the research publications to be analyzed
  • What inclusion/exclusion criteria will be used to screen and select the final research publications to be analyzed
  • How the data collected will be analyzed

Systematic review reporting standards or reporting guidelines, such as PRISMA and ROSES, will detail the required elements your protocol document will need to make explicit, and later guide what will need to be included in any manuscripts reporting the results of the systematic review. Reporting standards/guidelines may differ across disciplines.

A reporting standard/guideline is developed by an organization and tells you WHAT you need to report on. The protocol is written by you and your team, and will specifically describe HOW those standards/guidelines will be met.

By creating a protocol, not only does the team have a plan of action, they also have minimized the risk for bringing subjectivity and inconsistency into the review process. Protocols should also be registered and published in a registry as a means to publicize the team’s intent to conduct the review. This is considered a best practice as it can reduce duplication of effort by other researchers intending to conduct a similar review project, and allow for peer-review of the project’s methodology.  Some journals may require that a systematic review manuscript submission have a registered protocol before considering the manuscript for publication.

Suggested places to register and publish a protocol are listed here.

Protocol Examples

As you prepare to write your protocol, it may be helpful to review others.

Here are two protocols published in the journal Environmental Evidence.

  • Coates, J., Bostick, K.J., Jones, B.A. et al. What is the impact of aminoglycoside exposure on soil and plant root-associated microbiota? A systematic review protocol. Environ Evid 11, 18 (2022). https://doi.org/10.1186/s13750-022-00274-y

  • Bekchanova, M., Campion, L., Bruns, S. et al. Biochar’s effect on the ecosystem services provided by sandy-textured and contaminated sandy soils: a systematic review protocol. Environ Evid 10, 7 (2021). https://doi.org/10.1186/s13750-021-00223-1

Here are two protocols published in the journal Campbell Systematic Reviews.

  • Filges, T., Dietrichson, J. Viinholt, B.C.A., & Dalgaard, N.T. (2022). Service learning for improving academic success in students in grade K to 12: A systematic review. Campbell Systematic Reviews, 18(1), e1210. https://doi.org/10.1002/cl2.1210 
  • Reith-Hall, E., & Montgomery, P. (2023). Communication skills training for improving the communicative abilities of student social workers. Campbell Systematic Reviews, 19(1), e1309. https://doi.org/10.1002/cl2.1309

Here is a protocol published in the PROCEED registry

Here is a scoping review protocol registered with the Open Science Framework (OSF) Registries:

  • What are the strategies that farmer organizations use, and what impacts do those strategies have on small-scale producers' livelihoods and the environment? A scoping review protocol. Available: https://osf.io/cxrwb 

Checklists of Reporting Standards by Discipline (forms, flow diagrams)

Inclusion and Exclusion Criteria

Inclusion and exclusion criteria set the boundaries for the review. They are determined after establishing the research question and ideally in advance of the comprehensive literature search. (It is important to note that exploratory or scoping searches should be performed in order to determine appropriate criteria). Some common variables used as inclusion and exclusion criteria are described below. Additionally, the popular mnemonics such as PICO/PECO, SPICE, and others you may use to help frame your research question can also serve to guide the criteria you use to include or exclude research studies. 

Information about inclusion and exclusion criteria should be recorded as a paragraph or table within the methods section of the review. 

Common inclusion/exclusion criteria are:

  • Date (or date range)
  • Exposure of Interest (required experience or condition of participant or subject)
  • Geographic location of study
  • Language (of the study)
  • Participants (by demographic)
  • Peer review (or not--and may depend upon the research question to include, then, the gray literature)
  • Reported outcomes
  • Setting (specific location of research participants)
  • Study Design
  • Type of Publication (original research or other--akin to the peer-review condition, above)

Guide to Systematic Review Tools and Services, University of Melbourne Libraries
https://unimelb.libguides.com/sysrev/inclusion-exclusion-criteria