Nomogram to predict severe retinopathy of prematurity in Southeast China

Dan Liu, Xing-Yong Li, Hong-Wu He, Ka-Lu Jin, Ling-Xia Zhang, Yang Zhou,Zhi-Min Zhu, Chen-Chen Jiang, Hai-Jian Wu, Sui-Lian Zheng

1Department of Ophthalmology, the Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University, Wenzhou 325027, Zhejiang Province, China

2Eye Hospital and School of Ophthalmology and Optometry,Wenzhou Medical University, Wenzhou 325027, Zhejiang Province, China

3Taizhou Optometry Hospital, Taizhou 318001, Zhejiang Province, China

4Department of Ophthalmology, Jiaxing Maternity and Child Health Care Hospital, Jiaxing 314009, Zhejiang Province, China 5Department of Ophthalmology, Taizhou Women and Children’s Hospital of Wenzhou Medical University, Taizhou 318001, Zhejiang Province, China

6Clinical Medical Research Center, the Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University, Wenzhou 325027, Zhejiang Province, China

7Department of Ophthalmology, Taizhou Municipal Hospital,Taizhou 318099, Zhejiang Province, China

Abstract

● KEYWORDS: retinopathy of prematurity; nomogram;predictive factor; birth weight; multiple births; non-invasive ventilation

INTRODUCTION

Retinopathy of prematurity (ROP) is a fundus disease of prematurity with pathological proliferation of neovascularization and fibrosis, which is an important cause of blindness in children in both developed and developing countries[1].At present, most countries and regions mainly set screening standards according to gestational age and birth weight[2].For example, in the United States, ROP screening criteria are birth weight (BW)≤1500 g or gestational age(GA)≤30wk, while in the United Kingdom, screening criteria are BW≤1500 g or GA≤31wk.In China, ROP screening criteria are extensive, including BW＜2000 g or GA＜32wk.Extensive screening criteria will improve detection rate but bring huge screening burden.Most ROP eventually regress spontaneously, studies in numerous countries reported that only about 10% or less of screened preterm infants progress to severe ROP requiring treatment[3-5].ROP screening is not only stressful for infants, but also a huge drain on medical resources.Therefore, the development of new effective screening programs or models that can reduce the frequency of screening will contribute to the accuracy and effectiveness of ROP diagnosis and treatment.

At present, methods to predict the risk of severe ROP by establishing risk models have been widely studied[6], such as WINROP model[7], CHOP ROP model, ROPScore, PINT ROP,G-ROP[8], CO-ROP model[9], and DIGIROP-Birth model[10].

Some risk models have been extensively validated and have significantly reduced ROP screening.However, some wellperformed risk models, based on population data from developed countries in Europe or North America, have not been performed as well as expected in some developing countries or some Asian populations.For example, the WINROP model was 100% sensitive to severe ROP and reduced the number of infants requiring screening by 76%in a retrospective study in Sweden[11].But a study in Mexico showed that WINROP identified 84.7% of severe ROP in very preterm infants, but only 5.3% in moderate preterm infants[12].Similar results were seen in southern China, WINROP only identified 56% of type 1 ROP in a study involving 432 ROP preterm infants[13].

This study aims to explore the predictive factors for severe ROP and to establish a nomogram for predicting severe ROP in southeast China.

SUBJECTS AND METHODS

Ethical ApprovalThis study was approved by the Ethics Committee of the Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University and Taizhou Women and Children’s Hospital (LCKY2019-288),and was in accordance with the Declaration of Helsinki.Patient informed consent was waived due to the retrospective nature of the study.

SubjectsInfants diagnosed with ROP after fundus screening,hospitalized in the Neonatal intensive care Unit of the Second Affiliated Hospital and Yuying Children’s Hospital of Wenzhou Medical University from January 1, 2015 to December 31,2020 were included as training group (January 1, 2015 to December 31, 2019) and internal validation group (January 1, 2020 to December 31, 2020).ROP infants hospitalized in Taizhou Women and Children’s Hospital from January 1,2020 to August 1, 2021 were included as external validation cohort.Fundus screening was performed by pediatric fundus ophthalmologist with sufficient experience and knowledge in accordance with the Screening Guidelines for ROP in China[14].The International Classification of Retinopathy of Prematurity, Third Edition (ICROP3)[15]was used for diagnosis and staging of ROP.Pre-threshold disease type I, threshold disease stage 3, aggressive posterior ROP, stage 4, and stage 5 require treatment, which were defined as severe ROP, and the remaining stages were defined as mild ROP.

Inclusion criteria: 1) diagnosis of ROP; 2) GA＜32wk or BW＜2000 g; 3) complete medical information.Exclusion criteria: 1) fundus image blur caused by refractive medium turbidity; 2) incomplete clinical data.

Data CollectionThe information of ROP stages and 43 candidate predictive factors were collected retrospectively.Candidate predictive factors including gender, BW, GA,weight gain in first week after birth, weight gain in second week after birth, weight gain in third week after birth,delivery method, Apgar score at 1min, Apgar score at 5min,multiple births, assisted conception, neonatal respiratory distress syndrome, patent ductus arteriosus, neonatal hyperbilirubinemia, intracranial hemorrhage, neonatal asphyxia, congenital heart disease, neonatal anemia, sepsis,bronchopulmonary dysplasia, blood transfusion, oxygen inhalation, nasal cannula oxygen inhalation, face mask oxygen inhalation, non-invasive ventilation, endotracheal intubation, infant incubator, premature rupture of membrane,intrauterine infection, placental abstention, abnormal amniotic fluid, umbilical cord around the neck or prolapse, and intrauterine distress.Meanwhile, maternal factors were also collected, including age, gestational hypertension, gestational diabetes, and prenatal use of antibiotics, dexamethasone,antihypertensive drugs, ritodrine hydrochloride, hypoglycemicdrugs, magnesium sulfate, and vitamin K.

Table 1 Clinical features of the training, internal validation and external validation groups

Statistical AnalysisQuantitative variables were described by medians with interquartile ranges (IQRs), and qualitative variables were described as whole numbers and proportions.The associations between candidate predictive factors and severe ROP were assessed by univariate logistic regression analysis and multivariate logistic regression analysis in training group.A nomogram was established using the rms package in R statistical software (version 4.2.0) based on the independent predictive factors identified by multivariate logistic regression.The model was evaluated using receiver operating characteristic curve (ROC), calibration plot, and decision curve analysis.The area under curve (AUC) of ROC curve is equivalent to C statistic.The nomogram was verified in internal and external validation group and compared with WINROP model and DIGIROP model.All statistical analysis and nomogram establishing was performed using the R statistical software (version 4.2.0) and SPSS (version 22).Pvalue was calculated and a value less than 0.05 was considered significant.

WINROP Algorithm ScreeningWINROP was an online prediction model (www.winrop.com) used to predict the risk of severe ROP in premature infants based on GA, BW and postnatal longitudinal weight data analysis.Prediction results were classified as high risk (red alert) and low risk (green alert).Data of each infant were entered into the WINROP model according to the official steps until the “red alert”appeared or the corrected GA reached 34wk.

DIGIROP-Birth ScreeningDigirop-Birth model was an online prediction model (www.digirop.com) used to predict the risk of severe ROP for premature infants with GA between 24 and 30wk, based on gender, GA, and BW.Predicted outcomes were the probability of developing severe ROP (0 to 100%).Data of each infant were input into its online model following the official steps to obtain results.

RESULTS

Clinical FeaturesA total of 554 infants were diagnosed based on ICROP3 with 483 mild ROP and 71 severe ROP.Among these infants, 402 infants were included in the training group,76 infants were included in the internal validation group, and 76 infants were included in the external validation group.The clinical features were listed in Table 1.

Predictive Factors for Severe ROPUnivariate analysis was performed for all 43 candidate predictive factors in training group, and 8 of them associated with severe ROP(P＜0.05), including BW, GA, weight gain in second week after birth, weight gain in third week after birth, multiple births,Apgar score at 1min, Apgar score at 5min, and non-invasive ventilation (Table 2).Using multivariable analysis, BW [odds ratio (OR), 0.997; 95% confidence interval (CI), 0.996-0.999;P＜0.001], multiple births (OR, 1.885; 95%CI, 1.013-3.506;P=0.045), or non-invasive ventilation (OR, 0.288; 95%CI,0.146-0.570;P＜0.001) were all independently associated with severe ROP (Table 3).Although numerous other candidate predictors were associated with severe ROP in univariate analyses, none was significant in the multivariate model (data not shown).

Nomogram for Severe ROP and PerformanceWe then established a nomogram to predict severe ROP based on BW, multiple births, and non-invasive ventilation as we demonstrated in Table 3.The points for each predictor were defined as showed in Figure 1A and added to obtain the total points.The occurrence probability of severe ROP can be obtained according to total points.The ROC curve of this model (Figure 1B; Table 4) had an AUC of 72.5% (95%CI,64.8%-80.1%) in training group, and had well-fitted calibration curves (Figure 1C).

Internal ValidationInternal validation group was predicted by new nomogram, WINROP model, and Digirop-Birth model, respectively.According to the inclusion criteria of WINROP model, 1 case of mild ROP was excluded due to GA≥32wk, and 6 severe ROP and 6 mild ROP were excluded because of missing longitudinal weight data after birth in internal validation group.According to the inclusion criteria of Digirop-Birth model, 7 case of mild ROP were excluded due to GA＞30wk.The AUC of the new nomogram was larger than that of WINROP and Digirop-Birth, which means the new model had better predictive performance than WINROP model and Digirop-Birth model in internal validation.In decision curve analysis, the new model outperforms WINROP and Digirop-Birth under most risk thresholds.However, the calibration curve of the internal validation group fluctuated greatly, which means the predictive stability of the new model is not perfect.This may be related to the small sample size(Figure 2; Table 4).

Table 2 Univariate analysis of the association of candidate predictors with severe ROP

External ValidationExternal validation group were predicted by nomogram, WINROP model and Digirop-Birth model,respectively.In WINROP model, 15 case of mild ROP was excluded due to GA≥32wk.In Digirop-Birth model, 28 case of mild ROP were excluded due to GA＞30wk, 1 case of severe ROP were excluded due to GA＜24wk.The AUC of the new nomogram was also larger than that of WINROP and Digirop-Birth.In decision curve analysis, the new model also outperforms WINROP and Digirop-Birth.However, the calibration curve of the internal validation group fluctuated greatly, which may be related to the small sample size (Figure 2;Table 4).

Table 3 Multivariate analysis of the association of candidate predictors with severe ROP

Figure 1 New nomogram for prediction of severe ROP A: New nomogram for prediction of severe ROP.To estimate the probability of severe ROP, mark patient values at each axis, draw a straight line perpendicular to the point axis, and sum the points for all variables.Next, mark the sum on the total point axis and draw a straight line perpendicular to the probability axis.B: ROC curve of nomogram in training group; C: Calibration plot of nomogram in training group.ROP: Retinopathy of prematurity; ROC: Receiver operating characteristic curve.

Mixed ValidationIn order to minimize the impact of sample size, we combined all the data from the internal validation combination and external validation group into a mixed validation group, which was verified in the same way.The results showed that the new nomogram has good efficiency and the fluctuation of calibration curve is reduced (Figure 2; Table 4).

Figure 2 Validation of the nomogram A: ROC curve of nomogram in internal validation group (left), external validation group (middle) and mixed validation group (right); B: Calibration plot of nomogram in internal validation group (left), external validation group (middle) and mixed validation group (right); C: Decision curve of nomogram in internal validation group (left), external validation group (middle) and mixed validation group (right).ROC: Receiver operating characteristic curve.

DISCUSSION

Most ROP will regress naturally, severe ROP which requiring treatment tends to be less than 10% of the screened population[5].This study investigated the predictive factors for severe ROP in southeast China, and found that BW, multiple births, and non-invasive ventilation are risk factors for severe ROP.We established a nomogram for assessment of the risk of severe ROP.

BW is a recognized predictive factor for ROP.Low BW increases the risk of incident ROP and severe ROP[1].Low BW is associated with a greater likelihood of retinal underdevelopment.GA was not included in new nomogram because it was not statistically significant in multivariate analysis.We hypothesized that GA may be associated with the occurrence of ROP, but the association between mild ROP and severe ROP was not significant, which may require larger sample size to verify.

Numerous studies have found that multiple birth is associated with ROP and severe ROP[1].However, other studies reported lower incidence of ROP in multiple births[16], or no differences between singleton and multiple births[17].But the sample sizes of these studies were relatively small.A population-based retrospective study[18]in Taiwan, China, included insurance claims data from 27 830 twins and 111 080 singleton cohort.The study found that the overall incidence of ROP was 13.6 times higher in the twin group than in the singleton group.Multiple gestation, which tend to be associated with many perinatal problems, include low GA, low BW, less mature lung function, and longer duration of oxygen use, which may increase the probability of severe ROP in preterm multiple births.With the development of assisted reproductive technology, the incidence of multiple births has further increased, so the impact of multiple births on ROP deserves our attention.

Numerous studies have identified long-term mechanical ventilation as a risk factor for ROP and severe ROP[18-20].In this study, non-invasive ventilation was associated with severe ROP (P＜0.05) and was a protective factor.The protective effect of non-invasive ventilation on severe ROP may be related to reasonable and stable oxygen concentration and reduced mechanical ventilation time.

At present, there are many excellent prediction models in the field, with advanced algorithms and large modeling sample sizes.For example, the WINROP model was 100% sensitive to severe ROP of 353 infants in Sweden[11], and 98.6% of 1706 infants in another multicenter study involving the United States and Canada[21].In a retrospective study with 445 infants in Italy, WINROP showed 93.2% sensitivity to identify severe ROP, while ROPScore and CHOP ROP showed 100%sensitivity, respectively[22].

But in several studies, models based on population from developed countries in Europe and North America tend to produce mediocre results in other developed countries in Asia and Africa or developing countries.A WINROP study in South Korea showed a sensitivity of 90.0% and specificity of 52.6%[23].In a study of 352 preterm infants in the developing country of Mexico, WINROP correctly identified 84.7% of type 1 ROP in very preterm infants, but only 5.3% in moderate preterm infants[12].Similar results were seen in a study of 432 ROP preterm infants in southern China, WINROP correctly identified 56% of type 1 ROP (28/50) in all infants[13].Digirop-Birth model, based on Swedish data, performed less satisfactorily in identifying infants with severe ROP in a retrospective study of 442 Chinese premature infants than previously reported.The AUC was 0.634 and sensitivity was 51.6% when threshold was 0.0084[24].

In this study, new nomogram model was established based on the analysis results of risk factors for severe ROP.Compared with WINROP model and Digirop-Birth model, new nomogram has better performance in validation groups.The model can be transformed to a nomogram for easy clinical use.In the future, we will collect more data from clinical centers in southeast China and other regions of China to further refine the new model.

In summary, we found that BW, multiple births, and noninvasive ventilation were significantly associated with severe ROP, of which low BW and multiple births were risk factors and non-invasive ventilation was protective factor.We established a new nomogram model for predicting of severe ROP, which is more effective than WINROP model and DIGIROP-Birth model in southeast China.

There are several limitations to this study.Although we analyzed a large number of predictors, the sample size was still small, which may affect the accuracy of the analysis.Although internal and external validation were used, the sample size of the validation group was also small and some analyses may be limited.Our new nomogram still needs a larger sample size for validation and improvement.

ACKNOWLEDGEMENTS

Authors’ contributions:Zheng SL designed and supervised whole study; Li XY, He HW, Jin KL, Zhang LX, Zhou Y, Zhu ZM and Wu HJ collected data; Liu D, Li XY, Zhang LX, and Jiang CC analyzed data; Li XY and Jiang CC designed and verified model; Liu D and Li XY wrote the manuscript.

Foundation:Supported by Wenzhou Science and Technology Project (No.Y20190173).

Conflicts of Interest:Liu D,None;Li XY,None;He HW,None;Jin KL,None;Zhang LX,None;Zhou Y,None;Zhu ZM,None;Jiang CC,None;Wu HJ,None;Zheng SL,

None.