Qualidade metodológica e risco de viés em revisões sistemáticas de intervenções em saúde: uma revisão de métodos
DOI:
https://doi.org/10.17267/2675-021Xevidence.2022.e4067Palavras-chave:
Systematic Review, Healthcare, Methodologic Quality, Risk of bias, Critical appraisalResumo
OBJECTIVE: To compare the characteristics of systematic reviews of healthcare interventions that assessed or did not methodologic quality/risk-of-bias of included studies. Additionally, to analyze methodologic features of those assessing the methodologic quality/risk-of-bias. METHODS: PubMed database was searched. From 25,571 systematic reviews retrieved, a random sample of 1,025 was screened. Frequencies were used to describe outcomes. Unadjusted and adjusted logistic regressions were performed to test the associations with methodologic quality/risk-of-bias results assessment. In a second analysis, systematic reviews that assessed methodologic quality/risk of bias were dichotomized according to the design of included studies (randomized clinical trials-only versus non-randomized studies of intervention or a combination of both). RESULTS: 303 systematic reviews were fully analyzed. Methodologic quality/risk of bias was assessed by 278 (92%). Methodologic quality/risk-of-bias assessment was associated with a higher number of databases searched (>4, P= 0.008), the presence of meta-analysis (P= 0.005), and the design of included studies (randomized clinical trials-only, P= 0.042). The chance of using a suitable tool and a tool designed for risk-of-bias assessment rather than methodologic quality was higher for randomized clinical trials-only systematic reviews (P< 0.05). The most used tool was Cochrane’s RoB Tool without a clear studies’ overall risk classification system. CONCLUSION: methodologic quality/risk-of-bias assessment was associated with included studies’ design (randomized clinical trials-only), a meta-analysis of data, and the number of databases searched (>4). The most used tool was Cochrane’s RoB Tool, with no clearly defined rating system. Methodologic quality/risk-of-bias assessment methods description, results, and impacts on meta-analysis, the certainty of evidence, and systematic reviews’ results are still to be consistently addressed.
Downloads
Referências
Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS. Evidence based medicine: what it is and what it isn't. BMJ. 1996;312(7023):71-2. https://doi.org/10.1136/bmj.312.7023.71
Moher D, Liberati A, Tetzlaff J, Altman DG; PRISMA Group. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097
Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358:j4008. https://doi.org/10.1136/bmj.j4008
Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-6. https://doi.org/10.1136/bmj.39489.470347.ad
Lasserson TJ, Thomas J, Higgins JPT. Chapter 1: Starting a review [Internet]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). London: Cochrane; 2020. Available from: www.training.cochrane.org/handbook
Boutron I, Page MJ, Higgins JPT, Altman DG, Lundh A, Hróbjartsson A. Chapter 7: Considering bias and conflicts of interest among the included studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). London: Cochrane; 2020. Available from: www.training.cochrane.org/handbook
Page MJ, Higgins JP, Clayton G, Sterne JA, Hrobjartsson A, Savovic J. Empirical Evidence of Study Design Biases in Randomized Trials: Systematic Review of Meta-Epidemiological Studies. PloS one. 2016;11(7):e0159267. https://doi.org/10.1371/journal.pone.0159267
Page MJ, McKenzie JE, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CD, et al. The PRISMA 2020 statement: an updated guideline for reporting systematic reviews. BMJ. 2021;372:n71. https://doi.org/10.1136/bmj.n71
Pussegoda K, Turner L, Garritty C, Mayhew A, Skidmore B, Stevens A, et al. Systematic review adherence to methodological or reporting quality. Syst Rev. 2017;6(1):131. https://doi.org/10.1186/s13643-017-0527-2
Saltaji H, Ospina MB, Armijo-Olivo S, Agarwal S, Cummings GG, Amin M, et al. Evaluation of risk-of-bias assessment of trials in systematic reviews of oral health interventions, 1991-2014: A methodology study. J Am Dent Assoc. 2016;147(9):720-8.e1. https://doi.org/10.1016/j.adaj.2016.03.017
Munn Z, Stern C, Aromataris E, Lockwood C, Jordan Z. What kind of systematic review should I conduct? A proposed typology and guidance for systematic reviewers in the medical and health sciences. BMC Med Res Methodol. 2018;18(1):5. https://doi.org/10.1186/s12874-017-0468-4
Higgins JPT, Green S, editors. Cochrane Handbook for Systematic Reviews of Interventions Version 5.1.0 [updated March 2011] [Internet]. London: The Cochrane Collaboration; 2011. Available from: http://handbook-5-1.cochrane.org/
Martinic MK, Pieper D, Glatt A, Puljak L. Definition of a systematic review used in overviews of systematic reviews, meta-epidemiological studies and textbooks. BMC Med Res Methodol. 2019;19(1):203. https://doi.org/10.1186/s12874-019-0855-0
Stefani CM, Rezende LVML, Honnef LR, Oliveira LB, Lopes NC, Oliveira JMD, et al. Methodologic quality and risk of bias in Systematic Reviews of Healthcare Interventions: a methodologic review. 2021. https://doi.org/10.17605/OSF.IO/CH5T9
Green S, Higgins JPT, Alderson P, et al. Cochrane handbook: cochrane reviews: Ch 8: assessing risk of bias in included studies. Cochr Handbook Syst Rev Intervent. 2011:3–10. https://training.cochrane.org/handbook/archive/v5.1/
Wells G, Shea B, O'Connell D, Peterson J, Welch V, Losos M, et al. The Newcastle-Ottawa Scale (NOS) for assessing the quality of nonrandomized studies in meta-analysis, 2000 [Internet]. Available from: http://www.ohri.ca/programs/clinical_epidemiology/oxford.asp
Guyatt GH, Oxman AD, Montori V, Vist G, Kunz R, Brozek J, et al. GRADE guidelines: 5. Rating the quality of evidence--publication bias. J Clin Epidemiol. 2011;64(12):1277-82. https://doi.org/10.1016/j.jclinepi.2011.01.011
Santos MBF, Agostini BA, Bassani R, Pereira GKR, Sarkis-Onofre R. Protocol registration improves reporting quality of systematic reviews in dentistry. BMC Med Res Methodol. 2020;20(1):57. https://doi.org/10.1186/s12874-020-00939-7
Higgins JPT, Sterne JAC, Savovi? J, Page MJ, Hróbjartsson A, Boutron I, et al. A revised tool for assessing risk of bias in randomized trials In: Chandler J, McKenzie J, Boutron I, Welch V, editors. [Internet] Cochrane Methods. Cochrane Database of Systematic Reviews. 2016. https://doi.org/10.1002/14651858.CD201601
Sterne JAC, Savovi? J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: a revised tool for assessing risk of bias in randomised trials. BMJ. 2019;366:4898. https://doi.org/10.1136/bmj.l4898
Higgins JPT, Savovi? J, Page MJ, Elbers RG, Sterne JAC. Chapter 8: Assessing risk of bias in a randomized trial [Internet]. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). London: Cochrane; 2022. Available from: www.training.cochrane.org/handbook
Reeves BC, Deeks JJ, Higgins JPT, Shea B, Tugwell P, Wells GA. Chapter 24: Including non-randomized studies on intervention effects. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, et al., editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). London: Cochrane; 2020. Available from: www.training.cochrane.org/handbook
Sterne JAC, Hernán MA, Reeves BC, Savovi? J, Berkman ND, Viswanathan M, et al. ROBINS-I: a tool for assessing risk of bias in non-randomized studies of interventions. BMJ 2016;355:i4919. https://doi.org/10.1136/bmj.i4919
Sterne JAC, Hernán MA, McAleenan A, Reeves BC, Higgins JPT. Chapter 25: Assessing risk of bias in a non-randomized study. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA, editors. Cochrane Handbook for Systematic Reviews of Interventions version 6.1 (updated September 2020). London: Cochrane; 2020. Available from: www.training.cochrane.org/handbook
Sterne J, Higgins J, Reeves B, Savovi? J, Turner L. Cochrane Scientific Committee. Recommendation statement/report. Review of the development of the risk of bias tool for nonrandomised studies for interventions – ROBINS-I [Internet]. Available from: https://methods.cochrane.org/sites/default/files/public/uploads/scientific_committee_statement_report_robins_i_fin.pdf
Coleman BD, Khan KM, Maffulli N, Cook JL, Wark JD. Studies of surgical outcome after patellar tendinopathy: clinical significance of methodological deficiencies and guidelines for future studies. Victorian Institute of Sport Tendon Study Group. Scand J Med Sci Sports 2000;10(1):2–11. https://doi.org/10.1034/j.1600-0838.2000.010001002.x
Downs SH, Black N. The feasibility of creating a checklist for the assessment of the methodological quality both of randomised and non-randomised studies of health care interventions. J Epidemiol Community Health. 1998;52(6):377-84. https://doi.org/10.1136/jech.52.6.377
Tufanaru C, Munn Z, Aromataris E, Campbell J, Hopp L. Chapter 3: Systematic reviews of effectiveness. In: Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. https://doi.org/10.46658/JBIMES-20-04
Aromataris E, Munn Z, editors. JBI Manual for Evidence Synthesis. JBI; 2020. https://doi.org/10.46658/JBIMES-20-01
Downloads
Publicado
Edição
Seção
Licença
Copyright (c) 2022 Cristine Miron Stefani, Liliana Vicente Melo de Lucas Rezende , Lia Rosana Honnef, Luciana Butini Oliveira , Nathan da Cruz Lopes , Carla Massignan, Graziela De Luca Canto, Júlia Meller Dias de Oliveira
Este trabalho está licenciado sob uma licença Creative Commons Attribution 4.0 International License.
The authors retain copyrights, transferring to the Journal of Evidence-Based Healthcare only the right of first publication. This work is licensed under a Creative Commons Attribution 4.0 International License.
