Assessment of the global trigger tool to measure, monitor and evaluate pateint safety in cancer patients

Reliability concerns are raised

Thea Otto Mattsson, Janne Lehmann-Knudsen, Jens M Lauritsen, Kim Brixen, Jørn Herrstedt

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

BACKGROUND: Countries around the world are currently aiming to improve patient safety by means of the Institute for Healthcare Improvement global trigger tool (GTT), which is considered a valid tool for evaluating and measuring patient safety within organisations. So far, only few data on the measurement properties and utility of the GTT have been published. AIMS: To determine and evaluate the effect of interrater variation between review teams on the standard outcome measures of the GTT and to assess and quantify measurement error of the GTT. METHODS: Retrospective chart reviews were conducted on identical charts by two independent review teams in 2010 at a department of oncology in a university hospital. Standard GTT outcome measurements were obtained and compared between teams using statistical process control (SPC) charts. A Bland-Altman plot assessed measurement error and limits of agreement. RESULTS: Only 31% of adverse events (AE) were identified by both teams, and further differences in categorisation of identical events was found. Moderate interrater agreement (κ=0.45) between teams gave rise to different conclusions on the patient safety process when monitoring using SPC charts. The Bland-Altman plot suggests little systematic error but large random error. CONCLUSIONS: Review teams may identify different AE and reach different conclusions on the safety process when using the GTT on identical charts. Tracking true change in the safety level is difficult due to measurement error of the GTT. The results do not encourage further use of the GTT until additional evaluation studies on the measurement properties of the GTT have been conducted.
Original languageEnglish
JournalB M J Quality and Safety
Volume22
Issue number7
Pages (from-to)571-579
ISSN2044-5415
DOIs
Publication statusPublished - 2013

Fingerprint

Patient Safety
Safety
Neoplasms
Health Care Quality Indicators
Outcome Assessment (Health Care)
Organizations

Cite this

@article{cf1c73c75e7f469d8687d87c8a961994,
title = "Assessment of the global trigger tool to measure, monitor and evaluate pateint safety in cancer patients: Reliability concerns are raised",
abstract = "BACKGROUND: Countries around the world are currently aiming to improve patient safety by means of the Institute for Healthcare Improvement global trigger tool (GTT), which is considered a valid tool for evaluating and measuring patient safety within organisations. So far, only few data on the measurement properties and utility of the GTT have been published. AIMS: To determine and evaluate the effect of interrater variation between review teams on the standard outcome measures of the GTT and to assess and quantify measurement error of the GTT. METHODS: Retrospective chart reviews were conducted on identical charts by two independent review teams in 2010 at a department of oncology in a university hospital. Standard GTT outcome measurements were obtained and compared between teams using statistical process control (SPC) charts. A Bland-Altman plot assessed measurement error and limits of agreement. RESULTS: Only 31{\%} of adverse events (AE) were identified by both teams, and further differences in categorisation of identical events was found. Moderate interrater agreement (κ=0.45) between teams gave rise to different conclusions on the patient safety process when monitoring using SPC charts. The Bland-Altman plot suggests little systematic error but large random error. CONCLUSIONS: Review teams may identify different AE and reach different conclusions on the safety process when using the GTT on identical charts. Tracking true change in the safety level is difficult due to measurement error of the GTT. The results do not encourage further use of the GTT until additional evaluation studies on the measurement properties of the GTT have been conducted.",
author = "{Otto Mattsson}, Thea and Janne Lehmann-Knudsen and Lauritsen, {Jens M} and Kim Brixen and J{\o}rn Herrstedt",
year = "2013",
doi = "10.1136/bmjqs-2012-001219",
language = "English",
volume = "22",
pages = "571--579",
journal = "B M J Quality and Safety",
issn = "2044-5415",
publisher = "BMJ Group",
number = "7",

}

Assessment of the global trigger tool to measure, monitor and evaluate pateint safety in cancer patients : Reliability concerns are raised. / Otto Mattsson, Thea; Lehmann-Knudsen, Janne; Lauritsen, Jens M; Brixen, Kim; Herrstedt, Jørn.

In: B M J Quality and Safety, Vol. 22, No. 7, 2013, p. 571-579.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - Assessment of the global trigger tool to measure, monitor and evaluate pateint safety in cancer patients

T2 - Reliability concerns are raised

AU - Otto Mattsson, Thea

AU - Lehmann-Knudsen, Janne

AU - Lauritsen, Jens M

AU - Brixen, Kim

AU - Herrstedt, Jørn

PY - 2013

Y1 - 2013

N2 - BACKGROUND: Countries around the world are currently aiming to improve patient safety by means of the Institute for Healthcare Improvement global trigger tool (GTT), which is considered a valid tool for evaluating and measuring patient safety within organisations. So far, only few data on the measurement properties and utility of the GTT have been published. AIMS: To determine and evaluate the effect of interrater variation between review teams on the standard outcome measures of the GTT and to assess and quantify measurement error of the GTT. METHODS: Retrospective chart reviews were conducted on identical charts by two independent review teams in 2010 at a department of oncology in a university hospital. Standard GTT outcome measurements were obtained and compared between teams using statistical process control (SPC) charts. A Bland-Altman plot assessed measurement error and limits of agreement. RESULTS: Only 31% of adverse events (AE) were identified by both teams, and further differences in categorisation of identical events was found. Moderate interrater agreement (κ=0.45) between teams gave rise to different conclusions on the patient safety process when monitoring using SPC charts. The Bland-Altman plot suggests little systematic error but large random error. CONCLUSIONS: Review teams may identify different AE and reach different conclusions on the safety process when using the GTT on identical charts. Tracking true change in the safety level is difficult due to measurement error of the GTT. The results do not encourage further use of the GTT until additional evaluation studies on the measurement properties of the GTT have been conducted.

AB - BACKGROUND: Countries around the world are currently aiming to improve patient safety by means of the Institute for Healthcare Improvement global trigger tool (GTT), which is considered a valid tool for evaluating and measuring patient safety within organisations. So far, only few data on the measurement properties and utility of the GTT have been published. AIMS: To determine and evaluate the effect of interrater variation between review teams on the standard outcome measures of the GTT and to assess and quantify measurement error of the GTT. METHODS: Retrospective chart reviews were conducted on identical charts by two independent review teams in 2010 at a department of oncology in a university hospital. Standard GTT outcome measurements were obtained and compared between teams using statistical process control (SPC) charts. A Bland-Altman plot assessed measurement error and limits of agreement. RESULTS: Only 31% of adverse events (AE) were identified by both teams, and further differences in categorisation of identical events was found. Moderate interrater agreement (κ=0.45) between teams gave rise to different conclusions on the patient safety process when monitoring using SPC charts. The Bland-Altman plot suggests little systematic error but large random error. CONCLUSIONS: Review teams may identify different AE and reach different conclusions on the safety process when using the GTT on identical charts. Tracking true change in the safety level is difficult due to measurement error of the GTT. The results do not encourage further use of the GTT until additional evaluation studies on the measurement properties of the GTT have been conducted.

U2 - 10.1136/bmjqs-2012-001219

DO - 10.1136/bmjqs-2012-001219

M3 - Journal article

VL - 22

SP - 571

EP - 579

JO - B M J Quality and Safety

JF - B M J Quality and Safety

SN - 2044-5415

IS - 7

ER -