Evidence Synthesis, Systematic Review Services : Assess Risk of Bias
Risk of Bias Assessment / Critical Appraisal of Study Validity
Your research team will conduct its risk of bias assessment, also referred to as "critical appraisal" or "quality assessment", on each research study that the team selects for its review. This is a process for recognizing the likelihood for bias and discussing the limitations of the evidence and how that may manifest in a study's methodology, missing data, or how outcomes are measured or selectively reported, among other variables.
Several checklist tools have been developed to guide this process. These should be determined in advance (a priori) and disclosed in the team's protocol. Again, and similarly to the initial screening process, two to three team members will conduct the risk-of-bias assessment. The level of this assessment and its reporting is determined by review methodology (a hallmark of systematic reviews, for instance, but not necessarily other evidence synthesis review types).
When reported in the team's review manuscript, the risk of bias assessment may be depicted by a table or graph, showing how well each included study satisfies selected, predetermined criteria. For an example, see the introductory content to the robvis application.
Tools for Evaluating Risk of Bias
-
Collaboration for Environmental Evidence Critical Appraisal Tool Version 0.3 (Prototype)Critical appraisal tool for evaluating ‘risk of bias’ (or threats to internal validity) of primary studies assessing effectiveness of interventions or impacts of exposures in environmental management.
-
Critical Appraisal Checklists, CASP (Critical Appraisal Skills Programme)Free downloadable checklists to assist with performing critical appraisals across a number of different study types.
-
GRADE-CERQualAn approach for assessing how much confidence to place in the findings of a qualitative evidence synthesis. The overall assessment of confidence (high, moderate, low, very low) is made on the basis of an assessment of four components: methodological limitations, coherence, adequacy, and relevance.
-
GRADE Approach (Cochrane)Systematic approach to rating the certainty of evidence in systematic reviews and other evidence syntheses.
-
Latitudes NetworkLibrary of validity assessment tools for use in evidence syntheses. Researchers can also register tools under development. Website also provides access to training resources.
-
Mixed Methods Appraisal Tool (MMAT)Critical appraisal tool that was developed for use in systematic mixed studies reviews (i.e., reviews combining qualitative, quantitative and/or mixed methods studies).
-
Risk of Bias Tools for Use in Systematic Review (Cochrane; health sciences)RoB2; ROBINS-I; ROBINS-E; ROB-ME; robvis