Interrater variation of vascular classifications used in enhanced laryngeal contact endoscopy

Purpose: Combined use of contact endoscopy (CE) and Narrow Band Imaging (NBI, Olympus^®) is suggested for the visualization of specific vascular changes indicative of glottic neoplasia. We investigated the interrater reliability and agreement in 3 recognized classification systems of vascular changes applied to images from CE + NBI in patients suspected for glottic neoplasia. Methods: Six experienced head and neck surgeons familiar with NBI rated 120 images obtained by CE + NBI by 3 classification systems of vascular changes as suggested by Ni et al. (N-C), Puxeddu et al. (P-C), and the European Laryngological Society (ELS-C). Three raters were experienced in CE, and three raters had only limited experience with CE. Crude agreement and Fleiss’ kappa with 95% confidence interval were estimated for all 6 raters, and for the 2 levels of expertise for each original classification system and for dichotomized versions of the N-C and the P-C based on suggested neoplastic potential. Results: The interrater crude agreement and the corresponding kappa values for the ELS-C were good and significantly higher than those for the N-C and P-C for all raters, irrespective of the level of experience with CE (p < 0.0001). There were no significant differences between the N-C and the P-C (p = 0.16). Kappa was considerably improved for both the N-C and the P-C to a level not different from the ELS-C (p = 0.21–0.71) when their 5 original categories were pooled into dichotomized classifications. Conclusion: Difficulties in reliably classifying vascular changes in CE + NBI are evident. Two-tier classification systems are the most reliable.