Contact:
Systematic Review Team
Contact:
sysrev@hsl.wisc.edu
Systematic Review Toolkit
- CovidenceUser-friendly tool that allows you to do title/abstract screening, full-text screening, data abstraction, and quality assessment in one place. One free personal review. Ebling Library has an institutional subscription with unlimited reviews.
- EndNoteEndNote is a citation management program that you can use to import references from multiple databases; organize and manage references; and format bibliographies and manuscripts.
- AbstrackrAn open source abstract review tool created by Brown University, Abstrackr is a free program that you can use to facilitate citation screening for systematic reviews. The program will allow participants to collaboratively screen and annotate abstracts for relevance.
- Excel Templates for Searching and ScreeningPublicly available templates for documenting your search and the numbers for your PRISMA tree, as well as templates for title/abstract and full-text screening, provided by the University of Texas.
- DistillerSRThis fee-based online application is designed to assist with abstract and full-text screening, as well as data extraction and data output.
- Systematic Review Data Repository (SRDR)From AHRQ, SRDR is a free and easy-to-use tool for data extraction and management designed specifically for systematic reviews. In addition to providing data management and storage, SRDR also contains a repository of data from other systematic reviews, with the option to publish your data when your review is complete.
- Creating a PRISMA flow diagramA guide from University of North Carolina Libraries on how to create a PRISMA flow diagram.
More free systematic review tools are available online. You can search these tools and find associated articles through the Systematic Review Toolbox:
- SR Toolbox (currently unavailable)A comprehensive list of systematic review tools. You can select a tool based on the part of the systematic review you are working on, the cost, discipline, or the underlying approach.
Research Question Frameworks/Models
- PICO - Patient/Population | Intervention | Comparison | Outcome
For systematic reviews, use PICO to develop a narrow, clinically-oriented question. See PICO for Clinical Questions for more information.
- PCC - Population | Concept | Context
For scoping reviews, the Joanna Briggs Institute recommends using PCC (Population, Concept, Context) to develop the broader questions more suitable for this type of review. See Developing the title and question (of a Scoping Review) for more information.
- SPIDER - Sample | Phenomenon of Interest | Design | Evaluation | Research
For qualitative or mixed-method systematic reviews, consider using SPIDER which focuses more on the key elements of these types of non-quantitative studies. See the article Beyond PICO: The SPIDER Tool for Qualitative Evidence Synthesis for more information.
- Other Models/Frameworks for Research Questions
For other alternative models for framing your review question, see University of Maryland's comprehensive Frameworks for Research Questions guide for more information.
Online Training
- AHRQ Training Modules for the Systematic Reviews Methods GuidePresentations based on the AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews, providing training in selecting eligibility criteria, searching the literature, data extraction, quality assessment, and other steps in the creation of a systematic review.
- Campbell Collaboration Online Training VideosA series of videos from the annual Campbell Colloquium, containing methodological training from experts on systematic reviews and metaanalysis. Videos are divided into introductory and advanced methods.
- Cochrane Introduction to Systematic Reviews (free with registration)Basic overview of the systematic review process, from formulating a question to publishing your review. From the Cochrane Collaboration. Additional training materials are available to registered Cochrane authors or contributors.
- JBI Manual for Evidence SynthesisPractical guidance from the Joanna Briggs Institute on conducting systematic reviews, scoping reviews, and other types of evidence syntheses
Quality Assessment Tools
- Appendix A: Tools To Assess Risk of Bias of Individual OutcomesIn: Viswanathan M, Ansari MT, Berkman ND, Chang S, Hartling L, McPheeters LM, Santaguida PL, Shamliyan T, Singh K, Tsertsvadze A, Treadwell JR. Assessing the Risk of Bias of Individual Studies in Systematic Reviews of Health Care Interventions. Agency for Healthcare Research and Quality Methods Guide for Comparative Effectiveness Reviews. March 2012. AHRQ Publication No.
12-EHC047-EF. Available at: www.effectivehealthcare.ahrq.gov/ - CASP ChecklistsA set of critical appraisal checklists. Checklists are available for systematic reviews, qualitative studies, RCTs, case-control studies, diagnostic studies, cohort studies, and economic evaluations.
- Diagnostic Accuracy ToolWhiting P, Rutjes AWS, Dinnes J, et al. Development and validation of methods for assessing the quality of diagnostic accuracy studies. Health Technol Assess 2004;8(25):iii, 1-234.
- LEGEND Evidence Evaluation ToolsA series of critical appraisal tools from the Cincinnati Children's Hospital. Contains tools for a wide variety of study designs, including prospective, retrospective, qualitative, and quantitative designs.
- Mixed Methods Appraisal Tool (MMAT)A validated mixed methods appraisal tool from McGill University.
- The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomised studies in meta-analysesValidated tool for assessing case-control and cohort studies.
- Observational Epidemiology Quality Rating ToolSanderson S, Tatt ID, Higgins JP. Tools for assessing quality and susceptibility to bias in observational studies in epidemiology: a systematic review and annotated bibliography. Int J Epidemiol 2007;36:666-76.
Systematic Review Databases
- Campbell Collaboration Library of Systematic ReviewsSystematic Reviews in education, crime and justice, and social welfare
- Cochrane Database of Systematic ReviewsSystematic reviews in health care and health policy.
- Database of Abstracts of Reviews of Effectiveness (DARE)Effectiveness and cost-effectiveness of health care interventions
- OTSeekerSystematic reviews, randomized controlled trials and other resources for occupational therapy interventions. Abstract only.
- PsycBITE (Psychological Database for Brain ImpairmentTreatment Efficacy)Database of studies of cognitive, behavioural and other treatments for psychological problems and issues occurring as a consequence of acquired brain impairment (ABI). Rates studies for their methodological quality. Freely available.
- SpeechBITE (Speech Pathology Database for Best Interventions and Treatment Efficacy)Systematic reviews and other literature related to speech pathology interventions. Evidence is graded; highest-quality evidence is listed first for each search.
Systematic Review Protocol Registries
- The Campbell CollaborationSystematic reviews of the effects of social interventions.
- Cochrane CollaborationSystematic reviews on health care and health policy.
- PROSPEROInternational prospective register of systematic reviews
Style Guides & UW-Madison Resources
Style Guides
Suggested UW-Madison Resources
Methods and Further Reading
- Systematic Reviews: The ProcessGuide from the Duke University Medical Center Library with a description of the systematic review process links to guidance and resources for conducting a systematic review.
- Cochrane Handbook for Systematic Reviews of InterventionsMethodological guidance for the preparation and maintenance of Cochrane intervention reviews. Many of the principles and methods described here are appropriate for systematic reviews applied to other types of research and to systematic reviews of interventions undertaken by others.
- JBI Manual for Evidence SynthesisPractical guidance from the Joanna Briggs Institute on conducting systematic reviews, scoping reviews, and other types of evidence syntheses
- AHRQ Methods Guide for Effectiveness and Comparative Effectiveness ReviewsThe AHRQ Methods Guide for Effectiveness and Comparative Effectiveness Reviews, provides detailed information on selecting eligibility criteria, searching the literature, data extraction, quality assessment, and other steps in the creation of a systematic review.
- PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analysis)The aim of the PRISMA Statement is to help authors improve the reporting of systematic reviews and meta-analyses.
- IOM Systematic Review StandardsA list of standards for systematic reviews from the Institute of Medicine, including a list of the steps that a systematic review should include and what those steps should entail.
AI in Systematic Reviews
Artificial Intelligence has generated much interest among researchers conducting systematic reviews and other evidence synthesis projects. The information below provides links to guidance, tools, and articles on the use of AI in the systematic review process.
Guidance for the use of AI in systematic reviews:
- Cochrane Handbook 4.6.6.2: Automating the Selection Process
The linked guidance from Cochrane discusses the use of automated tools/artificial intelligence in three areas of the systematic review process. They give examples of where it is currently being used, as well as areas where it is not currently recommended. Covidence has integrated a few of the tools--Cochrane RCT classifier, relevancy ranking, and text highlighting--into their product. See Automated tools for systematic reviews below for more details. - Artificial intelligence (AI) technologies in Cochrane
View the recent recording of a Cochrane web clinic on the use of AI technologies in Cochrane reviews. Learn how Cochrane currently uses machine learning, the challenges and opportunities of generative AI use in evidence syntheses, and their focus on responsible use of AI in systematic reviews. - 2020 Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA)
PRISMA 2020 guidance states that authors should describe how automation tools were used in the review process, including: search strategy design or translation, study selection process, data collection process, study risk of bias assessment, etc. See the PRISMA 2020 Expanded Checklist and Explanation and Elaboration (E&E) documents for more details.
Automated tools for systematic reviews
- Covidence Cochrane RCT Classifier
Learn more about using the Cochrane RCT classifier when screening records in Covidence. - Covidence Relevancy Ranking
Learn more about how Covidence uses machine learning to speed up the title/abstract screening process through relevancy ranking. - Covidence Text Highlighting
Learn how to create and manage text highlighting in Covidence to speed up title/abstract screening. - PubMed PubReMiner
Use PubMed PubReMiner to harvest common keywords and subject headings (MeSH) terms. Copy and paste PubMed unique identifiers (PMIDs) and click 'Start PubMed PubReMiner' to see how frequently keywords and MeSH terms are used in those articles. This tool can also be used to identify potential journals for publication or key authors in the field. - SR Accelerator
The SR Accelerator is a collection of automation tools to help make the process of evidence synthesis projects easier and faster. Tools include: The Methods Wizard, Word Frequency Analysis, a Deduplicator and more.
Articles about the use of AI for systematic reviews:
- Using artificial intelligence methods for systematic review in health sciences: A systematic review
"This review delineated automated tools and platforms that employ artificial intelligence (AI) approaches and evaluated the reported benefits and challenges in using such methods." - Semi-automated Tools for Systematic Searches
Provides an overview of semi-automated tools used in the systematic review search process. - Can ChatGPT Write a Good Boolean Query for Systematic Review Literature Search?
"In this paper, we investigate ChatGPT as a means for automatically formulating and refining complex Boolean queries for systematic review literature search."
For more information on systematic reviews and other evidence synthesis projects, see Ebling's Systematic Review guide.