External validation of AIBx, an artificial intelligence model for risk stratification, in thyroid nodules

Background: Artificial intelligence algorithms could be used to risk-stratify thyroid nodules and may reduce the subjectivity of ultrasonography. One such algorithm is AIBx which has shown good performance. However, external validation is crucial prior to clinical implementation.

Materials and methods: Patients harboring thyroid nodules 1-4 cm in size, undergoing thyroid surgery from 2014 to 2016 in a single institution, were included. A histological diagnosis was obtained in all cases. Medullary thyroid cancer, metastasis from other cancers, thyroid lymphomas, and purely cystic nodules were excluded. Retrospectively, transverse ultrasound images of the nodules were analyzed by AIBx, and the results were compared with histopathology and Thyroid Imaging Reporting and Data System (TIRADS), calculated by experienced physicians.

Results: Out of 329 patients, 257 nodules from 209 individuals met the eligibility criteria. Fifty-one nodules (20%) were malignant. AIBx had a negative predictive value (NPV) of 89.2%. Sensitivity, specificity, and positive predictive values (PPV) were 78.4, 44.2, and 25.8%, respectively. Considering both TIRADS 4 and TIRADS 5 nodules as malignant lesions resulted in an NPV of 93.0%, while PPV and specificity were only 22.4 and 19.4%, respectively. By combining AIBx with TIRADS, no malignant nodules were overlooked.

Conclusion: When applied to ultrasound images obtained in a different setting than used for training, AIBx had comparable NPVs to TIRADS. AIBx performed even better when combined with TIRADS, thus reducing false negative assessments. These data support the concept of AIBx for thyroid nodules, and this tool may help less experienced operators by reducing the subjectivity inherent to thyroid ultrasound interpretation.