I just came across a recent article published in PLoS Medicine, Challenges in Developing Evidence-Based Recommendations Using the GRADE Approach: The Case of Mental, Neurological, and Substance Use Disorders (Barbui et al., 2010, PLoS Med 7(8): e1000322), where the WHO describes the route they took to develop their model intervention guide in the context of the mental health Gap Action Programme (mhGAP).

The idea is to provide recommendations for mental, neurological, and substance use disorders, which are responsible for 14% of the global burden of disease worldwide. They rely on the GRADE approach.⁽¹⁾

The GRADE methodology is fully described on http://www.gradeworkinggroup.org. It is used to summarize the evidence extracted from systematic reviews and meta-analyses, and grading the quality of evidence and strength of treatment recommendations gathered from them. It is distinct from a meta-analysis as it does not focus on the estimation of treatment effects or other statistical related stuff.

The GRADE approach considers that the following factors should be met before claiming any evidence regarding treatment effect in clinical guidelines (Box 3, from Barbui et al., 2010):

• Study limitations: Limitations in the study design that may bias the overall estimates of the treatment effect.
• Inconsistency: Unexplained differing estimates of the treatment effect size (i.e., heterogeneity or variability in results) across studies.
• Indirectness: The question being addressed by the guideline panel is different from the available evidence regarding the population, intervention, comparator, or outcome.
• Imprecision: Results are imprecise when studies include relatively few patients and few events and thus have wide confidence intervals around the estimate of the effect.
• Publication bias: Systematic underestimate or overestimate of the underlying beneficial or harmful effect due to the selective publication (or reporting) of studies.

Ansari et al.⁽²⁾ argue, however, that “Recommendations based on the GRADE approach specifically apply to clinical and not research settings.”

Anyway, this is a great series of checkpoints for reviewing purpose (in the wide sense).

References

1. Kavanagh, B.P. (2009). The GRADE system for rating clinical guidelines. PLoS Medicine, 6(9): e10000094.
2. Ansari, M.T., Tsertsvadze, A., and Moher, D. (2009). Grading Quality of Evidence and Strength of Recommendations: A Perspective. PLoS Medicine, 6(9): e1000151.