Empirical evidence of study design biases in randomized trials: Systematic review of meta-epidemiological studies

Matthew J. Page, Julian P. T. Higgins, Gemma Clayton, Jonathan A C Sterne, A. Hróbjartsson, Jelena Savović

Publikation: Bidrag til tidsskriftReviewForskningpeer review

129 Downloads (Pure)

Resumé

Objective: To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials. Design: Systematic review of meta-epidemiological studies. Methods: We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR <1 and dSMD <0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic. Results: We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear. Conclusions: Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed. © 2016 Page et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
OriginalsprogEngelsk
Artikelnummer0159267
TidsskriftP L o S One
Vol/bind11
Udgave nummer7
Antal sider26
ISSN1932-6203
DOI
StatusUdgivet - 2016

Bibliografisk note

Cited By :2 Export Date: 22 March 2017 CODEN: POLNC

Citer dette

Page, Matthew J. ; Higgins, Julian P. T. ; Clayton, Gemma ; Sterne, Jonathan A C ; Hróbjartsson, A. ; Savović, Jelena. / Empirical evidence of study design biases in randomized trials : Systematic review of meta-epidemiological studies. I: P L o S One. 2016 ; Bind 11, Nr. 7.
@article{43a768f2b9eb40c393321676c2a29c3f,
title = "Empirical evidence of study design biases in randomized trials: Systematic review of meta-epidemiological studies",
abstract = "Objective: To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials. Design: Systematic review of meta-epidemiological studies. Methods: We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ({"}mortality{"} versus {"}other objective{"} versus {"}subjective{"}). Direction of effect was standardised so that ROR <1 and dSMD <0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic. Results: We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95{\%} CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95{\%} CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95{\%} CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95{\%} CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95{\%} CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear. Conclusions: Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed. {\circledC} 2016 Page et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.",
author = "Page, {Matthew J.} and Higgins, {Julian P. T.} and Gemma Clayton and Sterne, {Jonathan A C} and A. Hr{\'o}bjartsson and Jelena Savović",
note = "Cited By :2 Export Date: 22 March 2017 CODEN: POLNC",
year = "2016",
doi = "10.1371/journal.pone.0159267",
language = "English",
volume = "11",
journal = "P L o S One",
issn = "1932-6203",
publisher = "Public Library of Science",
number = "7",

}

Empirical evidence of study design biases in randomized trials : Systematic review of meta-epidemiological studies. / Page, Matthew J.; Higgins, Julian P. T.; Clayton, Gemma; Sterne, Jonathan A C; Hróbjartsson, A.; Savović, Jelena.

I: P L o S One, Bind 11, Nr. 7, 0159267, 2016.

Publikation: Bidrag til tidsskriftReviewForskningpeer review

TY - JOUR

T1 - Empirical evidence of study design biases in randomized trials

T2 - Systematic review of meta-epidemiological studies

AU - Page, Matthew J.

AU - Higgins, Julian P. T.

AU - Clayton, Gemma

AU - Sterne, Jonathan A C

AU - Hróbjartsson, A.

AU - Savović, Jelena

N1 - Cited By :2 Export Date: 22 March 2017 CODEN: POLNC

PY - 2016

Y1 - 2016

N2 - Objective: To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials. Design: Systematic review of meta-epidemiological studies. Methods: We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR <1 and dSMD <0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic. Results: We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear. Conclusions: Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed. © 2016 Page et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

AB - Objective: To synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials. Design: Systematic review of meta-epidemiological studies. Methods: We retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR <1 and dSMD <0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic. Results: We included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear. Conclusions: Certain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed. © 2016 Page et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

U2 - 10.1371/journal.pone.0159267

DO - 10.1371/journal.pone.0159267

M3 - Review

C2 - 27398997

VL - 11

JO - P L o S One

JF - P L o S One

SN - 1932-6203

IS - 7

M1 - 0159267

ER -