A framework for quantifying net benefits of alternative prognostic models

Eleni Rapsomaniki, Ian R White, Angela M Wood, Simon G Thompson, Emerging Risk Factors Collaboration, Else-Marie Bladbjerg, Jørgen Jespersen

Research output: Contribution to journalJournal articleResearchpeer-review

Abstract

New prognostic models are traditionally evaluated using measures of discrimination and risk reclassification, but these do not take full account of the clinical and health economic context. We propose a framework for comparing prognostic models by quantifying the public health impact (net benefit) of the treatment decisions they support, assuming a set of predetermined clinical treatment guidelines. The change in net benefit is more clinically interpretable than changes in traditional measures and can be used in full health economic evaluations of prognostic models used for screening and allocating risk reduction interventions. We extend previous work in this area by quantifying net benefits in life years, thus linking prognostic performance to health economic measures; by taking full account of the occurrence of events over time; and by considering estimation and cross-validation in a multiple-study setting. The method is illustrated in the context of cardiovascular disease risk prediction using an individual participant data meta-analysis. We estimate the number of cardiovascular-disease-free life years gained when statin treatment is allocated based on a risk prediction model with five established risk factors instead of a model with just age, gender and region. We explore methodological issues associated with the multistudy design and show that cost-effectiveness comparisons based on the proposed methodology are robust against a range of modelling assumptions, including adjusting for competing risks.
Original languageEnglish
JournalStatistics in Medicine
Volume31
Issue number2
Pages (from-to)114-30
Number of pages17
ISSN0277-6715
DOIs
Publication statusPublished - 2012

Fingerprint

Health
Alternatives
Economics
Cost-Benefit Analysis
Hydroxymethylglutaryl-CoA Reductase Inhibitors
Competing Risks
Cost-effectiveness
Public Health
Risk Reduction Behavior
Risk Factors
Decision Support
Cross-validation
Model
Prediction Model
Linking
Discrimination
Screening
Meta-Analysis
Guidelines
Framework

Cite this

Rapsomaniki, E., White, I. R., Wood, A. M., Thompson, S. G., Emerging Risk Factors Collaboration, Bladbjerg, E-M., & Jespersen, J. (2012). A framework for quantifying net benefits of alternative prognostic models. Statistics in Medicine, 31(2), 114-30. https://doi.org/10.1002/sim.4362
Rapsomaniki, Eleni ; White, Ian R ; Wood, Angela M ; Thompson, Simon G ; Emerging Risk Factors Collaboration ; Bladbjerg, Else-Marie ; Jespersen, Jørgen. / A framework for quantifying net benefits of alternative prognostic models. In: Statistics in Medicine. 2012 ; Vol. 31, No. 2. pp. 114-30.
@article{b38ae23914154769970e473a755e8447,
title = "A framework for quantifying net benefits of alternative prognostic models",
abstract = "New prognostic models are traditionally evaluated using measures of discrimination and risk reclassification, but these do not take full account of the clinical and health economic context. We propose a framework for comparing prognostic models by quantifying the public health impact (net benefit) of the treatment decisions they support, assuming a set of predetermined clinical treatment guidelines. The change in net benefit is more clinically interpretable than changes in traditional measures and can be used in full health economic evaluations of prognostic models used for screening and allocating risk reduction interventions. We extend previous work in this area by quantifying net benefits in life years, thus linking prognostic performance to health economic measures; by taking full account of the occurrence of events over time; and by considering estimation and cross-validation in a multiple-study setting. The method is illustrated in the context of cardiovascular disease risk prediction using an individual participant data meta-analysis. We estimate the number of cardiovascular-disease-free life years gained when statin treatment is allocated based on a risk prediction model with five established risk factors instead of a model with just age, gender and region. We explore methodological issues associated with the multistudy design and show that cost-effectiveness comparisons based on the proposed methodology are robust against a range of modelling assumptions, including adjusting for competing risks.",
author = "Eleni Rapsomaniki and White, {Ian R} and Wood, {Angela M} and Thompson, {Simon G} and {Emerging Risk Factors Collaboration} and Else-Marie Bladbjerg and J{\o}rgen Jespersen",
note = "Kun udvalgte forfattere er indberettet. En fuldst{\ae}ndig forfatterliste findes p{\aa} PubMed",
year = "2012",
doi = "10.1002/sim.4362",
language = "English",
volume = "31",
pages = "114--30",
journal = "Statistics in Medicine",
issn = "0277-6715",
publisher = "JohnWiley & Sons Ltd.",
number = "2",

}

Rapsomaniki, E, White, IR, Wood, AM, Thompson, SG, Emerging Risk Factors Collaboration, Bladbjerg, E-M & Jespersen, J 2012, 'A framework for quantifying net benefits of alternative prognostic models', Statistics in Medicine, vol. 31, no. 2, pp. 114-30. https://doi.org/10.1002/sim.4362

A framework for quantifying net benefits of alternative prognostic models. / Rapsomaniki, Eleni; White, Ian R; Wood, Angela M; Thompson, Simon G; Emerging Risk Factors Collaboration ; Bladbjerg, Else-Marie; Jespersen, Jørgen.

In: Statistics in Medicine, Vol. 31, No. 2, 2012, p. 114-30.

Research output: Contribution to journalJournal articleResearchpeer-review

TY - JOUR

T1 - A framework for quantifying net benefits of alternative prognostic models

AU - Rapsomaniki, Eleni

AU - White, Ian R

AU - Wood, Angela M

AU - Thompson, Simon G

AU - Emerging Risk Factors Collaboration

AU - Bladbjerg, Else-Marie

AU - Jespersen, Jørgen

N1 - Kun udvalgte forfattere er indberettet. En fuldstændig forfatterliste findes på PubMed

PY - 2012

Y1 - 2012

N2 - New prognostic models are traditionally evaluated using measures of discrimination and risk reclassification, but these do not take full account of the clinical and health economic context. We propose a framework for comparing prognostic models by quantifying the public health impact (net benefit) of the treatment decisions they support, assuming a set of predetermined clinical treatment guidelines. The change in net benefit is more clinically interpretable than changes in traditional measures and can be used in full health economic evaluations of prognostic models used for screening and allocating risk reduction interventions. We extend previous work in this area by quantifying net benefits in life years, thus linking prognostic performance to health economic measures; by taking full account of the occurrence of events over time; and by considering estimation and cross-validation in a multiple-study setting. The method is illustrated in the context of cardiovascular disease risk prediction using an individual participant data meta-analysis. We estimate the number of cardiovascular-disease-free life years gained when statin treatment is allocated based on a risk prediction model with five established risk factors instead of a model with just age, gender and region. We explore methodological issues associated with the multistudy design and show that cost-effectiveness comparisons based on the proposed methodology are robust against a range of modelling assumptions, including adjusting for competing risks.

AB - New prognostic models are traditionally evaluated using measures of discrimination and risk reclassification, but these do not take full account of the clinical and health economic context. We propose a framework for comparing prognostic models by quantifying the public health impact (net benefit) of the treatment decisions they support, assuming a set of predetermined clinical treatment guidelines. The change in net benefit is more clinically interpretable than changes in traditional measures and can be used in full health economic evaluations of prognostic models used for screening and allocating risk reduction interventions. We extend previous work in this area by quantifying net benefits in life years, thus linking prognostic performance to health economic measures; by taking full account of the occurrence of events over time; and by considering estimation and cross-validation in a multiple-study setting. The method is illustrated in the context of cardiovascular disease risk prediction using an individual participant data meta-analysis. We estimate the number of cardiovascular-disease-free life years gained when statin treatment is allocated based on a risk prediction model with five established risk factors instead of a model with just age, gender and region. We explore methodological issues associated with the multistudy design and show that cost-effectiveness comparisons based on the proposed methodology are robust against a range of modelling assumptions, including adjusting for competing risks.

U2 - 10.1002/sim.4362

DO - 10.1002/sim.4362

M3 - Journal article

VL - 31

SP - 114

EP - 130

JO - Statistics in Medicine

JF - Statistics in Medicine

SN - 0277-6715

IS - 2

ER -

Rapsomaniki E, White IR, Wood AM, Thompson SG, Emerging Risk Factors Collaboration, Bladbjerg E-M et al. A framework for quantifying net benefits of alternative prognostic models. Statistics in Medicine. 2012;31(2):114-30. https://doi.org/10.1002/sim.4362