High certainty evidence is stable and trustworthy, whereas evidence of moderate or lower certainty may be equally prone to being unstable.

Autor: Djulbegovic B; Division of Hematology/Oncology, Department of Medicine, Medical University of South Carolina, Charleston, SC, USA., Koletsi D; Clinic of Orthodontics and Pediatric Dentistry, Center of Dental Medicine, University of Zurich, Zurich, Switzerland; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA. Electronic address: despoina.koletsi@zzm.uzh.ch., Hozo I; Department of Mathematics, Indiana University Northwest, Gary, IN, USA., Price A; Institute for Health Policy and Clinical Practice, Geisel School of Medicine, Dartmouth College, Hanover, NH, USA., Martimbianco ALC; Centre of Health Technology Assessment, Hospital Sírio-Libanês, São Paulo, Brazil; Postgraduate Program of Health and Environment, Universidade Metropolitana de Santos, São Paulo, Brazil., Riera R; Centre of Health Technology Assessment, Hospital Sírio-Libanês, São Paulo, Brazil; Universidade Federal de São Paulo, Escola Paulista de Medicina, Brazil (Unifesp), São Paulo, Brazil., Nadanovsky P; Department of Epidemiology and Quantitative Methods in Health, National School of Public Health, Fundação Oswaldo Cruz, Rio de Janeiro, Brazil; Department of Epidemiology, Institute of Social Medicine, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, Brazil., Dos Santos APP; Faculty of Dentistry, Department of Community and Preventive Dentistry, Universidade do Estado do Rio de Janeiro, Rio de Janeiro, Brazil., Pandis N; Department of Orthodontics and Dentofacial Orthopedics, Dental School/Medical Faculty, University of Bern, Bern, Switzerland., Pacheco RL; Centre of Health Technology Assessment, Hospital Sírio-Libanês, São Paulo, Brazil; Universidade Federal de São Paulo, Escola Paulista de Medicina, Brazil (Unifesp), São Paulo, Brazil., Fontes LE; Department of Intensive Care, and Emergency Medicine at Faculdade de Medicina de Petrópolis, in Petrópolis, Rio de Janeiro, Brazil., Seehra J; Centre for Craniofacial Development & Regeneration, Faculty of Dentistry, Oral & Craniofacial Sciences, King's College London, Floor 25, Guy's Hospital, London, SE1 9RT, United Kingdom., Ahmed M; Department of Medicine, Temerty Faculty of Medicine, University of Toronto, Toronto, Canada., Yao L; Department of Health Research Methods, Evidence, and Impact, McMaster University, Hamilton, Ontario, Canada., Nunan D; Kellogg College, University of Oxford, Oxford, UK; Centre for Evidence-Based Medicine, Nuffield Department of Primary Care Health Sciences, Oxford University, Oxford, UK., Hemkens LG; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, Stanford, CA, USA; Department of Clinical Research, University of Basel, University Hospital Basel, Basel, Switzerland; Pragmatic Evidence Lab, Research Center for Clinical Neuroimmunology and Neuroscience Basel (RC2NB), University Hospital Basel and University of Basel, Basel, Switzerland; Meta-Research Innovation Center Berlin (METRIC-B), Berlin Institute of Health, Berlin, Germany.
Jazyk: angličtina
Zdroj: Journal of clinical epidemiology [J Clin Epidemiol] 2024 Jul; Vol. 171, pp. 111392. Date of Electronic Publication: 2024 May 11.
DOI: 10.1016/j.jclinepi.2024.111392
Abstrakt: Objectives: To assess to what extent the overall quality of evidence indicates changes to observe intervention effect estimates when new data become available.
Methods: We conducted a meta-epidemiological study. We obtained evidence from meta-analyses of randomized trials of Cochrane reviews addressing the same health-care question that was updated with inclusion of additional data between January 2016 and May 2021. We extracted the reported effect estimates with 95% confidence intervals (CIs) from meta-analyses and corresponding GRADE (Grading of Recommendations Assessment, Development, and Evaluation) assessments of any intervention comparison for the primary outcome in the first and the last updated review version. We considered the reported overall quality (certainty) of evidence (CoE) and specific evidence limitations (no, serious or very serious for risk of bias, imprecision, inconsistency, and/or indirectness). We assessed the change in pooled effect estimates between the original and updated evidence using the ratio of odds ratio (ROR), absolute ratio of odds ratio (aROR), ratio of standard errors (RoSE), direction of effects, and level of statistical significance.
Results: High CoE without limitations characterized 19.3% (n = 29) out of 150 included original Cochrane reviews. The update with additional data did not systematically change the effect estimates (mean ROR 1.00; 95% CI 0.99-1.02), which deviated 1.06-fold from the older estimates (median aROR; interquartile range [IQR]: 1.01-1.15), gained precision (median RoSE 0.87; IQR 0.76-1.00), and maintained the same direction with the same level of statistical significance in 93% (27 of 29) of cases. Lower CoE with limitations characterized 121 original reviews and graded as moderate CoE in 30.0% (45 of 150), low CoE in 32.0% (48 of 150), and very low CoE in 18.7% (28 of 150) reviews. Their update had larger absolute deviations (median aROR 1.12 to 1.33) and larger gains in precision (median RoSE 0.78-0.86) without clear and consistent differences between these categories of CoE. Changes in effect direction or statistical significance were also more common in the lower quality evidence, again with a similar extent across categories (without change in 75.6%, 64.6%, and 75.0% for moderate, low, very low CoE). As limitations increased, effect estimates deviated more (aROR 1.05 with zero, 1.11 with one, 1.25 with two, 1.24 with three limitations) and changes in direction or significance became more frequent (93.2% stable with no limitations, 74.5% with one, 68.2% with two, and 61.5% with three limitations).
Conclusion: High-quality evidence without methodological deficiencies is trustworthy and stable, providing reliable intervention effect estimates when updated with new data. Evidence of moderate and lower quality may be equally prone to being unstable and cannot indicate if available effect estimates are true, exaggerated, or underestimated.
Competing Interests: Declaration of competing interest I.H.: has one-year consulting contract for research projects at the Medical University of South Carolina, Charleston, SC, USA. D.N.: declares having led/coauthored/coauthoring Cochrane Reviews and conducted GRADE assessments as part of completing some of these reviews. He also declares having been part of the Cochrane Convenes initiative organized by Cochrane to consider the issue of misinformation, its impact on the health evidence ecosystem and solutions to address it. L.G.H.’s Institution (RC2NB) is supported by Foundation Clinical Neuroimmunology and Neuroscience Basel. There are no competing interests for any other author.
(Copyright © 2024 The Author(s). Published by Elsevier Inc. All rights reserved.)
Databáze: MEDLINE
Externí odkaz: