Automated Machine Learning Models Can Identify Eyes With Diabetic Retinopathy Progression

In a recent prospective development and validation study, researchers found that automated machine learning models can accurately identified diabetic retinopathy progression from the use of ultra-widefield retinal images.

“The performance of the algorithms to identify diabetic retinopathy progression matched or exceeded previously published performance of bespoke artificial intelligence models,” the authors wrote.

The study included 1179 deidentified ultra-widefield retinal images taken with a California retinal imager. The images included those with mild (n = 380) or moderate (n = 799) nonproliferative diabetic retinopathy (NPDR) and 3 years of follow-up. The images were collected from a tertiary medical center from July to September 2022.

The primary outcome of the study results was the area under the precision-recall curve (AUPRC), sensitivity, specificity, and accuracy. The model’s AUPRC was 0.717 for baseline mild NPDR and 0.863 for moderate NPDR. Diabetic retinopathy progression was present in half of the deidentified images (n = 590).

In the validation set, among eyes with mild NPDR, the model’s sensitivity was 0.72 (95% CI, 0.57 to 0.83), specificity was 0.63 (95% CI, 0.57 to 0.69), prevalence was 0.15 (95% CI, 0.12 to 0.20), and accuracy was 64.3%. For eyes with moderate NPDR, sensitivity was 0.80 (95% CI, 0.70 to 0.87), specificity was 0.72 (95% CI, 0.66 to 0.76), prevalence was 0.22 (95% CI, 0.19 to 0.27), and accuracy was 73.8%.

Additionally, the model identified all eyes (n = 4) with mild NPDR that progressed within 6 months and 1 year. The model also identified eight of nine eyes with moderate NPDR that progressed within 6 months and 17 of 20 eyes that progressed within 1 year.

This study had limitations. For example, the authors noted that the size of the dataset, the omission of visual acuity outcomes, and the models only identifying progression among eyes with mild and moderate NPDR are all limitations that constrain the study results’ potential generalizability.

“This study demonstrates the accuracy and feasibility of automated machine learning (ML) models for identifying diabetic retinopathy (DR) progression developed using [ultra-widefield retinal] images, especially for prediction of 2-step or greater DR progression within 1 year,” the authors concluded. “Potentially, the use of ML algorithms may refine the risk of disease progression and identify those at highest short-term risk, thus reducing costs and improving vision-related outcomes.”

Reference:
Silva PS, Zhang D, Jacoba CMP, et al. Automated machine learning for predicting diabetic retinopathy progression from ultra-widefield retinal images. JAMA Ophthalmol. Published online February 8, 2024. doi:10.1001/jamaophthalmol.2023.6318