Last updated: 14th November 2023

Project summary

We are currently developing a critical appraisal tool for evaluating ‘risk of bias’ (or threats to internal validity) of primary studies assessing effectiveness of interventions or impacts of exposures in environmental management. There are well-known, widely applied risk of bias assessment tools in the health sector known as ‘RoB 2’ and ‘ROBINS-I’ (www.riskofbias.info), but there are currently no such critical appraisal tools in environmental management. Here we provide a third draft of the tool. The tool is still under development and requires initial testing, but it may help environmental evidence synthesists conduct critical appraisal. In the latest version, more descriptions are provided in Part A, and revisions and clarifications are made in Part B. We have merged two risk-of-bias criteria and so the total number of risk-of-bias criteria is reduced to seven in version 0.3.

If you are not familiar with critical appraisal step, we recommend reading Section 7 of CEE Guidelines before you download the tool. The tool can be used or tested in evidence syntheses addressing ‘PICO’ or ‘PECO’ type questions. If you are not familiar with these question types, we recommend reading Section 2 of CEE Guidelines before you download the tool. Since the current prototype tool is designed to evaluate risk of bias (or internal validity), external validity cannot be evaluated with this prototype tool.

The latest prototype tool is available in PDF and Excel files. Once you have gone through the process, you are welcome to provide feedback via our questionnaire, or you can directly contact us as we wish to publish the tool in an academic journal article in the future.

Download the Collaboration for Environmental Evidence Critical Appraisal Tool version 0.3 (prototype):