Methodological Quality of Systematic Reviews Addressing Orthodontic Interventions: Methodological Study

Sarah Queiroz Notaro; Ana Paula Hermont; Poliana Valdelice Cruz; Raiane Machado Maia; Walesca Melo Avila; Tina Poklepovic Pericic; Lucas Guimarães Abreu; Ruimin Jiao; Carolina Castro Martins-Pfeifer

Authors

Sarah Queiroz Notaro
Ana Paula Hermont
Poliana Valdelice Cruz
Raiane Machado Maia
Walesca Melo Avila
Tina Poklepovic Pericic
Lucas Guimarães Abreu
Ruimin Jiao
Carolina Castro Martins-Pfeifer

Keywords:

Systematic Review, Orthodontics, Malocclusion, Clinical Trial, Orthodontic Appliances

Abstract

Objective: To assess the methodological quality and characteristics of systematic reviews (SRs) of interventional studies in orthodontics and assess how the certainty of the evidence is reported using the GRADE approach. Material and Methods: Six electronic databases were searched, followed by a hand search of the reference lists of eligible studies (PROSPERO #CRD42020180852). The required study design was randomized and nonrandomized studies of interventions published between January 2019 and May 2020. The Assessing the Methodological Quality of Systematic Reviews (AMSTAR 2) tool was used for the quality appraisal of the included SRs. Paired reviewers independently screened the studies, extracted data, and appraised the methodological quality. Results: The study included 46 SRs; 19.5% had moderate to high methodological quality, and the remaining had low to critically low methodological quality. Fifty-four percent of the reviews assessed the certainty of evidence using the GRADE approach, and 34.8% followed all GRADE criteria. Conclusion: Most reviews had a good judgment of the AMSTAR2 items, although some critical items contributed to decreased overall quality. Half of the reviews used the GRADE approach to assess the certainty of the evidence, and this approach should be included in future systematic reviews of interventions.

References

Ioannidis J. The mass production of redundant, misleading, and conflicted systematic reviews and meta-analyses. Milbank Q 2016; 94(3):485-514. https://doi.org/10.1111/1468-0009.12210

Papageorgiou SN, Papadopoulos MA, AE Athanasiou. Evaluation of methodology and quality characteristics of systematic reviews in orthodontics. Orthod Craniofac Res 2011; 14(3):116-137. https://doi.org/10.1111/j.1601-6343.2011.01522.x

Papageorgiou SN, Papadopoulos MA, Athanasiou AE. Reporting characteristics of meta-analyses in orthodontics: Methodological assessment and statistical recommendations. Eur J Orthod 2013; 36(1):74-85. https://doi.org/10.1093/ejo/cjt008

Shea BJ, Grimshaw JM, Wells GA, Boers M, Aderssson N, Hamel C, et al. Development of AMSTAR: A measurement tool to assess the methodological quality of systematic reviews. BMC Med Res Methodol 2007; 7:10. https://doi.org/10.1186/1471-2288-7-10

Hooper EJ, Pandis N, Cobourne MT, Seehra J. Methodological quality and risk of bias in orthodontic systematic reviews using AMSTAR and ROBIS. Eur J Orthod 2021; 43(5):544-550. https://doi.org/10.1093/ejo/cjaa074

Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: A critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ 2017; 358:j4008. https://doi.org/10.1136/bmj.j4008

Cuello-Garcia CA, Morgan RL, Brozek J, Santesso N, Verbeek J, Thayer K, et al. A scoping review and survey provides the rationale, perceptions, and preferences for the integration of randomized and nonrandomized studies in evidence syntheses and GRADE assessments. J Clin Epidemiol 2018; 98:33-40. https://doi.org/10.1016/j.jclinepi.2018.01.010

Zhang Y, Akl EA, Schunemann HJ. Using systematic reviews in guideline development: The GRADE approach. Res Synth Methods 2018. https://doi.org/10.1002/jrsm.1313

Koletsi D, Fleming PS, Eliades T, Pandis N. The evidence from systematic reviews and meta-analyses published in orthodontic literature. Where do we stand? Eur J Orthod 2015; 37(6):603-609. https://doi.org/10.1093/ejo/cju087

Page MJ, Moher D, Bossuyt PM, Boutron I, Hoffmann TC, Mulrow CK, et al. PRISMA 2020 explanation and elaboration: updated guidance and exemplars for reporting systematic reviews. BMJ 2021; 372:n160. https://doi.org/10.1136/bmj.n160

Beller EM, Chen JK, Wang UL, Glasziou PP. Are systematic reviews up-to-date at the time of publication? Syst Rev 2013; 2:36. https://doi.org/10.1186/2046-4053-2-36

Sampson M, Shojania KG, Garritty C, Horsley T, Ocampo M, Moher D. Systematic reviews can be produced and published faster. J Clin Epidemiol 2008; 61(6):531-536. https://doi.org/10.1016/j.jclinepi.2008.02.004

Andersen MZ, Fonnes S, Andresen K, Rosenberg J. Most published meta-analyses were made available within two years of protocol registration. Eur J Int Med 2021; 44(101342). https://doi.org/10.1016/j.eujim.2021.101342

Sterne JA, Hernán MA, Reeves BC, Savovic J, Berkman ND, Viswanathan M, et al. ROBINS-I: A tool for assessing risk of bias in non-randomised studies of interventions. BMJ 2016; 355:i4919. https://doi.org/10.1136/bmj.i4919

Schünemann HJ, Cuello C, Akl EA, Mustafa RA, Meerpohl JJ, Thayer K, et al. GRADE guidelines: 18. How ROBINS-I and other tools to assess risk of bias in nonrandomized studies should be used to rate the certainty of a body of evidence. J Clin Epidemiol 2019; 111:105-114. https://doi.org/10.1016/j.jclinepi.2018.01.012

Guyatt GH, Oxman AD, Vist GE, Kunz R, Falck-Ytter Y, Alonso-Coello P, et al. GRADE Working Group. GRADE: An emerging consensus on rating quality of evidence and strength of recommendations. BMJ 2008; 336(7650):924-926. https://doi.org/10.1136/bmj.39489.470347.AD

Higgins JP, Altman DG, Gotzche PC, Jüni P, Moher D, Oxman AD, et al. The Cochrane Collaboration's tool for assessing risk of bias in randomised trials. BMJ 2011; 343:d5928. https://doi.org/10.1136/bmj.d5928

Sterne JAC, Savovic J, Page MJ, Elbers RG, Blencowe NS, Boutron I, et al. RoB 2: A revised tool for assessing risk of bias in randomised trials. BMJ 2019; 366:l4898. https://doi.org/10.1136/bmj.l4898

Tufanaru C, Munn Z, Aromataris E, Campbell J, Hopp L. Chapter 3: Systematic reviews of effectiveness. Aromataris E, Munn Z. JBI Manual for Evidence Synthesis. JBI; 2020.

Slim K, Nini E, Forestier D, Kwiatkowski F, Panis Y, Chipponi J. Methodological index for nonrandomized studies (minors): Development and validation of a new instrument. ANZ J Surg 2003; 73(9):12-16. https://doi.org/10.1046/j.1445-2197.2003.02748.x

Fleming PS, Seehra J, Polychronopoulou A, Fedorowicz Z, Pandis. Cochrane and non-Cochrane systematic reviews in leading orthodontic journals: A quality paradigm? Eur J Orthod 2012; 35(2):244-248. https://doi.org/10.1093/ejo/cjs016

Sideri S, Papageorgiou SN, Eliades T. Registration in the international prospective register of systematic reviews (PROSPERO) of systematic review protocols was associated with increased review quality. J Clin Epidemiol 2018; 100:103-110. https://doi.org/10.1016/j.jclinepi.2018.01.003

Torres D, Normando D. Biostatistics: Essential concepts for the clinician. Dental Press J Orthod 2021; 26(1):e21spe1. https://doi.org/10.1590/2177-6709.26.1.E21SPE1

Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 2011; 64(4):383-394. https://doi.org/10.1016/j.jclinepi.2010.04.026

Bin Bahar BSK, Alkhalidy SR, Kaklamanos EG, Athanasiou AE. Do orthodontic patients develop more gingival recession in anterior teeth compared to untreated individuals? A systematic review of controlled studies. Int Orthod 2020; 18(1):1-9. https://doi.org/10.1016/j.ortho.2019.08.025

Khalaf K, Mando M. Removable appliances to correct anterior crossbites in the mixed dentition: A systematic review. Acta Odontol Scand 2020; 78(2):118-125. https://doi.org/10.1080/00016357.2019.1657178

Khalaf K, Mando M. Effect of drugs on orthodontic tooth movement in human beings: A systematic review of randomized clinical trials. Open Dent 2019; 13(1):22-32. https://doi.org/10.2174/1874210601913010022

Bondemark L, Kallunki J, Paulsson L. An updated systematic review regarding early Class II malocclusion correction. JWFO 2019; 8(3):89-94. https://doi.org/10.1016/j.ejwf.2019.06.002

Langendam MW, Akl EA, Dahm P, Glasziou P, Guyatt G, Schünemann HJ. Assessing and presenting summaries of evidence in Cochrane Reviews. Syst Rev 2013; 2:81. https://doi.org/10.1186/2046-4053-2-81