Identifying approaches for assessing methodological and reporting quality of systematic reviews

a descriptive study

Kusala Pussegoda, Lucy Turner, Chantelle Garritty, Alain Mayhew, Becky Skidmore, Adrienne Stevens, Isabelle Boutron, Rafael Sarkis-Onofre, Lise M Bjerre, Asbjørn Hróbjartsson, Douglas G Altman, David Moher

Publikation: Bidrag til tidsskriftReviewForskningpeer review

113 Downloads (Pure)

Resumé

BACKGROUND: The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.

METHODS: The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.

RESULTS: Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ.

CONCLUSIONS: The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.

OriginalsprogEngelsk
Artikelnummer117
TidsskriftSystematic Reviews
Vol/bind6
Antal sider12
ISSN2046-4053
DOI
StatusUdgivet - 2017

Fingeraftryk

Guidelines
Evidence-Based Practice
Health Services Research
Proxy
Practice Guidelines
MEDLINE
Libraries
Research

Citer dette

Pussegoda, Kusala ; Turner, Lucy ; Garritty, Chantelle ; Mayhew, Alain ; Skidmore, Becky ; Stevens, Adrienne ; Boutron, Isabelle ; Sarkis-Onofre, Rafael ; Bjerre, Lise M ; Hróbjartsson, Asbjørn ; Altman, Douglas G ; Moher, David. / Identifying approaches for assessing methodological and reporting quality of systematic reviews : a descriptive study. I: Systematic Reviews. 2017 ; Bind 6.
@article{c74d47c535f74d689b5baf3c52d37cb0,
title = "Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study",
abstract = "BACKGROUND: The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.METHODS: The Cochrane Library, MEDLINE{\circledR}, and EMBASE{\circledR} were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.RESULTS: Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80{\%} (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24{\%} (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29{\%} (22/76) of reports, with 36{\%} (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26{\%} (6/22) using QUOROM and OQAQ.CONCLUSIONS: The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.",
keywords = "Journal Article",
author = "Kusala Pussegoda and Lucy Turner and Chantelle Garritty and Alain Mayhew and Becky Skidmore and Adrienne Stevens and Isabelle Boutron and Rafael Sarkis-Onofre and Bjerre, {Lise M} and Asbj{\o}rn Hr{\'o}bjartsson and Altman, {Douglas G} and David Moher",
year = "2017",
doi = "10.1186/s13643-017-0507-6",
language = "English",
volume = "6",
journal = "Systematic Reviews",
issn = "2046-4053",
publisher = "BioMed Central",

}

Pussegoda, K, Turner, L, Garritty, C, Mayhew, A, Skidmore, B, Stevens, A, Boutron, I, Sarkis-Onofre, R, Bjerre, LM, Hróbjartsson, A, Altman, DG & Moher, D 2017, 'Identifying approaches for assessing methodological and reporting quality of systematic reviews: a descriptive study', Systematic Reviews, bind 6, 117. https://doi.org/10.1186/s13643-017-0507-6

Identifying approaches for assessing methodological and reporting quality of systematic reviews : a descriptive study. / Pussegoda, Kusala; Turner, Lucy; Garritty, Chantelle; Mayhew, Alain; Skidmore, Becky; Stevens, Adrienne; Boutron, Isabelle; Sarkis-Onofre, Rafael; Bjerre, Lise M; Hróbjartsson, Asbjørn; Altman, Douglas G; Moher, David.

I: Systematic Reviews, Bind 6, 117, 2017.

Publikation: Bidrag til tidsskriftReviewForskningpeer review

TY - JOUR

T1 - Identifying approaches for assessing methodological and reporting quality of systematic reviews

T2 - a descriptive study

AU - Pussegoda, Kusala

AU - Turner, Lucy

AU - Garritty, Chantelle

AU - Mayhew, Alain

AU - Skidmore, Becky

AU - Stevens, Adrienne

AU - Boutron, Isabelle

AU - Sarkis-Onofre, Rafael

AU - Bjerre, Lise M

AU - Hróbjartsson, Asbjørn

AU - Altman, Douglas G

AU - Moher, David

PY - 2017

Y1 - 2017

N2 - BACKGROUND: The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.METHODS: The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.RESULTS: Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ.CONCLUSIONS: The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.

AB - BACKGROUND: The methodological quality and completeness of reporting of the systematic reviews (SRs) is fundamental to optimal implementation of evidence-based health care and the reduction of research waste. Methods exist to appraise SRs yet little is known about how they are used in SRs or where there are potential gaps in research best-practice guidance materials. The aims of this study are to identify reports assessing the methodological quality (MQ) and/or reporting quality (RQ) of a cohort of SRs and to assess their number, general characteristics, and approaches to 'quality' assessment over time.METHODS: The Cochrane Library, MEDLINE®, and EMBASE® were searched from January 1990 to October 16, 2014, for reports assessing MQ and/or RQ of SRs. Title, abstract, and full-text screening of all reports were conducted independently by two reviewers. Reports assessing the MQ and/or RQ of a cohort of ten or more SRs of interventions were included. All results are reported as frequencies and percentages of reports.RESULTS: Of 20,765 unique records retrieved, 1189 of them were reviewed for full-text review, of which 76 reports were included. Eight previously published approaches to assessing MQ or reporting guidelines used as proxy to assess RQ were used in 80% (61/76) of identified reports. These included two reporting guidelines (PRISMA and QUOROM) and five quality assessment tools (AMSTAR, R-AMSTAR, OQAQ, Mulrow, Sacks) and GRADE criteria. The remaining 24% (18/76) of reports developed their own criteria. PRISMA, OQAQ, and AMSTAR were the most commonly used published tools to assess MQ or RQ. In conjunction with other approaches, published tools were used in 29% (22/76) of reports, with 36% (8/22) assessing adherence to both PRISMA and AMSTAR criteria and 26% (6/22) using QUOROM and OQAQ.CONCLUSIONS: The methods used to assess quality of SRs are diverse, and none has become universally accepted. The most commonly used quality assessment tools are AMSTAR, OQAQ, and PRISMA. As new tools and guidelines are developed to improve both the MQ and RQ of SRs, authors of methodological studies are encouraged to put thoughtful consideration into the use of appropriate tools to assess quality and reporting.

KW - Journal Article

U2 - 10.1186/s13643-017-0507-6

DO - 10.1186/s13643-017-0507-6

M3 - Review

VL - 6

JO - Systematic Reviews

JF - Systematic Reviews

SN - 2046-4053

M1 - 117

ER -