Intra- and intergrader reliability of semiautomatic measurements of fundus fluorescein angiography leakage in proliferative diabetic retinopathy

Design of study: Cross-sectional image grading study. Purpose: Panretinal photocoagulation (PRP) is the gold-standard treatment for proliferative diabetic retinopathy (PDR). Fundus fluorescein angiography leakage (FFAL) is often used as a marker of disease activity, but evaluation of the difference between pre- and post-treatment FFAL is often difficult and not standardized. We propose a new model to quantify FFAL as a marker of PDR-activity. The purpose of the study was to evaluate the intra- and intergrader reliability of the model. Method: Two substudies were performed. In Substudy A the area of FFAL of 50 images of PDR was measured by two independent graders, and intra- and intergrader intraclass correlation coefficients (ICC) were calculated. In Substudy B the same two graders evaluated the area of FFAL from 14 sets of images taken at two different times. The amount of FFAL was determined and each grader categorized if there had been regression, steady state or progression. Regression/progression was defined as at least 10% change in area of FFAL. In both substudies all images were masked for each grader and between the graders. Intra- and intergrader kappa was calculated. OPTOS widefield FFA (Optomap; Optos PLC., Dunfermline, Scotland, UK) was used and images 5 minutes after injection were chosen. Images were measured for FFAL using ImageJ (National Institute of Health, Bethesda, Maryland, USA), allowing for auto-marking of areas with FFAL. The grader then verified and/or adjusted the boundaries and measured the FFAL area in number of pixels. Results: For the 50 images evaluated in Substudy A, intragrader ICC was >0.99 for both graders, and intergrader ICC was >0.99. For the categorization of the 14 sets of images in substudy B, intragrader kappa was 86.7% and 61.8% for grader 1 and 2, respectively, and intergrader kappa was 50.8%. Conclusion: Measuring FFAL on Optomap images reliably has been a challenge historically. With our method, we have been able to achieve high intra- and intergrader reliability for area size, and moderate for progression.