Revised JBI Critical Appraisal Tool for cohort studies released

A new and updated critical appraisal tool for cohort studies has been released by the JBI Effectiveness Methodology Group, as detailed in a recent publication in JBI Evidence Synthesis.  

This revised critical appraisal tool aims to provide a more robust framework for evaluating the risk of bias in cohort studies, reflecting advancements in methodological research and aligning with current guidance from the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) Working Group and PRISMA 2020 reporting guidelines.

What are cohort studies?

Cohort studies are recognised as a valuable analytical observational study design that investigates differences in outcomes between groups based on their exposure status. While randomised controlled trials (RCTs) are typically preferred for assessing intervention effectiveness, cohort studies play a vital role in systematic reviews, particularly when RCTs are not feasible, or when exploring the association of prognostic or risk factors with health outcomes. Cohort studies can also provide insights into intervention effectiveness, especially regarding adverse events, rare outcomes, and longer follow-up periods.

What has been revised in the JBI critical appraisal tool for cohort studies?

The revised JBI critical appraisal tool for cohort studies has undergone significant updates to enhance its relevance and longevity. A key focus of the revision was to align the tool with recent advancements in risk-of-bias science. The process involved organising existing questions into constructs of validity (internal, statistical conclusion, comprehensiveness of reporting, external) and further categorising questions related to internal validity into specific domains of bias.

The revised tool comprises eleven questions that assess various aspects of study design, conduct, and analysis to determine the risk of bias. These questions are now aligned to specific domains of bias, including:

• Bias related to selection and allocation: Assessing the similarity of the exposed and unexposed groups and their recruitment from the same population
• Bias related to classification of the exposure: Evaluating whether exposures were measured similarly and in a valid and reliable way for both groups
• Bias related to confounding factors: Examining the identification of potential confounders and the strategies employed to address them.
• Bias related to temporal precedence: Ensuring that participants were free of the outcome at the start of the study or at the time of exposure.
• Bias related to assessment, detection, and measurement of the outcome: Determining if outcomes were measured in a valid and reliable way and if the follow-up time was sufficient.
• Bias related to participant retention: Assessing the completeness of follow-up and the handling of loss to follow-up.
• Statistical conclusion validity: Evaluating the appropriateness of the statistical analysis used.

The revised JBI critical appraisal tool aims to enhance the rigor and transparency of the critical appraisal process for cohort studies included in systematic reviews. It provides a robust framework aligned with current methodological standards, so that the tool will assist reviewers in understanding the validity and relevance of the evidence.  

Resources:

Access the revised JBI critical appraisal tool

Read details about the updates and the domains of bias in the methodology paper released for #JBImethodology month: The revised JBI critical appraisal tool for the assessment of risk of bias for cohort studies

Barker, Timothy H.; Hasanoff, Sabira; Aromataris, Edoardo; Stone, Jennifer C.; Leonardi-Bee, Jo; Sears, Kim; Habibi, Nahal; Klugar, Miloslav; Tufanaru, Catalin; Moola, Sandeep; Liu, Xian-Liang; Munn, Zachary

JBI Evidence Synthesis. 23(3):441-453, March 2025.

Want to read more? Go to the Risk of Bias series in the journal, JBI Evidence Synthesis.