Development and multicenter validation of a nomogram for preoperative prediction of lymph node positivity in pancreatic cancer(NeoPangram)

Jie Hua ,Xue-Min Chen ,Yun-Jie Chen ,Bao-Chun Lu f ,Jin Xu ,Wei Wang ,Si Shi,Xian-Jun Yu ,∗

a Department of Pancreatic Surgery, Fudan University Shanghai Cancer Center, Shanghai 20 0 032, China

b Department of Oncology, Shanghai Medical College of Fudan University, Shanghai 20 0 032, China

c Shanghai Pancreatic Cancer Institute, Shanghai 20 0 032, China

d Department of Hepatobiliary and Pancreatic Surgery, The First People’s Hospital of Changzhou, Changzhou 213004, China

e Department of Minimally Invasive Hepatobiliary and Pancreatic Surgery, Ningbo No. 2 Hospital, Ningbo 315010, China

f Department of Hepatobiliary and Pancreatic Surgery, Shaoxing People’s Hospital, Shaoxing 3120 0 0, China

Keywords:Lymph node metastasis Pancreatic ductal adenocarcinoma Nomogram Neoadjuvant therapy

ABSTRACT Background: Neoadjuvant therapy is associated with nodal downstaging and improved oncological outcomes in patients with lymph node (LN)-positive pancreatic cancer.This study aimed to develop and validate a nomogram to preoperatively predict LN-positive disease.Methods: A total of 558 patients with resected pancreatic cancer were randomly and equally divided into development and internal validation cohorts.Multivariate logistic regression analysis was used to construct the nomogram.Model performance was evaluated by discrimination,calibration,and clinical usefulness.An independent multicenter cohort consisting of 250 patients was used for external validation.Results: A four-marker signature was built consisting of carbohydrate antigen 19–9 (CA19–9),CA125,CA50,and CA242.A nomogram was constructed to predict LN metastasis using three predictors identified by multivariate analysis:risk score of the four-marker signature,computed tomography-reported LN status,and clinical tumor stage.The prediction model exhibited good discrimination ability,with C-indexes of 0.806,0.742 and 0.763 for the development,internal validation,and external validation cohorts,respectively.The model also showed good calibration and clinical usefulness.A cut-off value (0.72) for the probability of LN metastasis was determined to separate low-risk and high-risk patients.Kaplan-Meier survival analysis revealed a good agreement of the survival curves between the nomogram-predicted status and the true LN status.Conclusions: This nomogram enables the identification of pancreatic cancer patients at high risk for LN positivity who may have more advanced disease and thus could potentially benefit from neoadjuvant therapy.

Introduction

Pancreatic cancer has the poorest prognosis among digestive malignancies with a 5-year relative survival of 7% −8% [ 1,2 ].Surgical resection remains the only potentially curative treatment.However,only 15% −20% of patients are surgical candidates,and most still suffer from recurrence and metastasis after surgery and have relatively short survival time [ 3,4 ].

Over the past decade,neoadjuvant therapy has attracted increasing attention regarding potentially curable pancreatic cancer.A recent study has shown that neoadjuvant therapy followed by resection is associated with a significant survival benefit compared with upfront surgery in patients with resectable disease [5].This supports using neoadjuvant therapy as a favorable alternative strategy for managing resectable pancreatic cancer.Additionally,onethird of patients with positive lymph nodes (LNs) may be downstaged after neoadjuvant therapy [6].The current National Comprehensive Cancer Network (NCCN) guidelines suggest that resectable pancreatic cancer patients with high-risk features,such as large regional LNs,should be considered candidates for neoadjuvant therapy [7].This implies that patients with positive LNs may be potential beneficiaries of neoadjuvant therapy.

LN metastasis is an independent prognostic indicator.The latest American Joint Committee on Cancer (AJCC) staging system has redefined the N category for patients with metastasis in four or more LNs as N2,thus classifying some stage IIB patients as stage III [8].This change highlights the importance of metastatic LN counts in staging and prognosis.However,the current clinical staging for nodal involvement is inaccurate.Approximately 70% of patients who are clinically node negative present nodal involvement upon pathological evaluation [9].To improve the accuracy of clinical nodal staging and identify patients with a high risk of LN metastasis who may potentially benefit from neoadjuvant therapy,we developed and validated a prediction model for individualized preoperative prediction of pathological LN positivity in resectable pancreatic cancer patients.

Patients and methods

Patients

Patients with pancreatic cancer who underwent surgical resection between January 2010 and December 2017 at Fudan University Shanghai Cancer Center were identified in a prospectively maintained database.We excluded patients with distant metastases(liver metastases or peritoneal carcinomatosis) on surgical exploration,fewer than 15 LNs examined after surgical resection,preoperative anticancer therapy (chemotherapy,radiotherapy,or both),or incomplete clinicopathological data.A total of 558 patients with resected pancreatic ductal adenocarcinoma (PDAC) were finally included.Computer-generated random numbers were used to assign 279 of the patients to a development cohort and 279 patients to an internal validation cohort.Patients’ clinicopathological data,including age,sex,comorbidity,preoperative bilirubin,preoperative serum levels of eight serum tumor markers [carbohydrate antigen 19–9 (CA19–9) (U/mL),CA125 (U/mL),CA72–4 (U/mL),CA15–3(U/mL),CA50 (IU/mL),CA242 (U/mL),carcinoembryonic antigen(CEA) (ng/mL),and alpha-fetoprotein (AFP) (ng/mL)],tumor location,clinical primary tumor (T) stage,computed tomography (CT)-reported LN status,type of surgery,pathological T stage,number of examined LNs,number of positive LNs,and histological grade,were collected.T and N stages were categorized according to the 8th edition of the AJCC Cancer Staging Manual [8].CT-reported LN status [negative (N-) or positive (N +)]was derived from a radiologic report that was completed by two experienced radiologists.Each CT scan was independently re-evaluated and cross-checked by two authors (Hua J and Shi S) who were unaware of the pathological LN status.Positive LN status should be in accordance with the suspicious LN features (Fig.S1) defined by the Pancreatic Ductal Adenocarcinoma Radiology Reporting Template [10].Any inconsistencies were resolved through discussion with a third author (Xu J) until a consensus was reached.An independent multicenter cohort consisting of 250 patients with the same inclusion criteria as described above from The First People’s Hospital of Changzhou (2010–2017),Ningbo No.2 Hospital (2015–2017),and Shaoxing People’s Hospital (2016–2017) was used for external validation.The study was approved by the Institutional Review Board of Fudan University Shanghai Cancer Center,and the requirement for written informed consent was waived due to the retrospective nature of this study.

Nomogram construction

Least absolute shrinkage and selection operator (LASSO) is a popular variable selection method that improves prediction accuracy by shrinking large regression coefficients to reduce overfitting [11].We used a LASSO regression model to select the most useful predictive tumor markers and constructed a multimarkerbased signature for predicting pathological LN positivity in pancreatic cancer patients in the development cohort.A risk scoring system was then developed based on the LASSO coefficients of the selected tumor markers.Each patient was then assigned a risk score,of which,scores above zero were recognized as high risk for LN metastasis,while scores below zero were considered low risk for LN metastasis.

A logistic regression model was used to examine the associations between potential predictors and pathological LN positivity.In addition to the risk score of the tumor marker signature,preoperatively determined potential predictors,including age,sex,comorbidities,preoperative bilirubin,tumor location,clinical T stage,and CT-reported LN status,were selected.Predictors were entered into the multivariate logistic regression model using backward stepwise selection withP=0.10 as the entry and removal probability,and a nomogram for pathological LN positivity was constructed with the final model.

Model performance

Model performance was evaluated by discrimination,calibration,and clinical usefulness.

The discrimination performance of the model was measured by Harrell’s concordance index (C-index),which is also known as the area under the curve (AUC) of a receiver operating characteristic(ROC) curve.The C-indexes and respective 95% confidence intervals (CIs) of the nomogram for the development cohort and both the internal and external validation cohorts were computed.The bias-corrected C-index was calculated using bootstrapping (10 0 0 replicates) and cross-validation (10-fold).

Calibration was assessed by plotting the nomogram-predicted probabilities versus the actual probabilities.A 45-degree line represents the ideal prediction.The closer the curve is to the 45-degree line,the better the calibration.In addition,the Hosmer-Lemeshow test was also performed to assess goodness of fit,of which aPvalue of less than 0.05 indicated a significant lack of fit.

Clinical usefulness was estimated with decision curve analysis by quantifying the net benefit at different threshold probabilities.

Statistical analysis

Clinicopathological features in patients who were node positive versus node negative were compared using Chi-square test for categorical variables and Mann-WhitneyUtest for continuous variables.Statistical analyses for construction and validation of the nomogram were performed using R software (version 3.5.0;rms,glmnet,pROC,Hmisc,andrmdapackages).The AUCs were compared using DeLong’s test [12].The optimal cut-off value for the probability of LN metastasis was determined by Youden index and the closest-to-(0,1) criterion.If disagreement existed,the value identified by Youden index was regarded as the“optimal”cut-off value [13].Overall survival was estimated using Kaplan-Meier curves and log-rank test.All statistical tests were two-sided,andP<0.05 was considered statistically significant.This study was conducted and reported in accordance with the Transparent Reporting of a multivariable prediction model for Individual Prognosis or Diagnosis (TRIPOD) guidelines [14]and the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement(www.strobe-statement.org/).

Fig.1.Flowchart of patient selection.PDAC:pancreatic ductal adenocarcinoma;FUSCC:Fudan University Shanghai Cancer Center.

Results

Patient characteristics

The patient selection process is shown in Fig.1 .Patient characteristics stratified by pathological LN status are presented in Table 1 .LN positivity was comparable between the development and internal validation cohorts (60.6% and 63.4%,respectively;P=0.485).

Construction of a multimarker-based signature and risk scoring system

Of the eight examined tumor markers,four (CA19–9,CA125,CA50,and CA242) had nonzero coefficients corresponding to the optimal tuning parameter (λ=0.056) (Fig.2).The risk score formula of the four-marker-based signature was built using the coefficients:risk score=−0.2238474+(serum level of CA19–9 ×0.0 0 0 04040021)+(serum level of CA125 × 0.001145052) +(serum level of CA50 × 0.001091296)+(serum level of CA242 ×0.0 0 03179741).Distributions of the risk scores and LN status in development and validation (internal and external) cohorts are shown in Fig.S2.Patients with high-risk scores had significantly higher proportions of LN positivity than those with low-risk scores(development cohort:80.4% vs.50.8%,P<0.001;internal validation cohort:73.8% vs.57.4%,P=0.006;external validation cohort:49.3% vs.31.1%,P=0.006;Fig.S3).

Predictors of LN positivity and nomogram construction

Initially,eight candidate variables were selected:age,sex,comorbidities,preoperative bilirubin,tumor location,clinical T stage,CT-reported LN status,and risk score of the four-marker signature(Table 2).Backward stepwise selection in binary logistic regression modeling identified three variables that were the most associated with LN metastasis.CT-reported LN status and risk score of the four-marker signature were independently associated with LN metastasis (P<0.05),while clinical T stage tended to be associated with LN metastasis (P=0.076).The equation for the final prediction model was as follows:y=−0.697+(2.080 × risk score of the four-marker signature)+2.114 (if CT-reported LN status was positive)+0.098 (if clinical T stage was T2) or 0.599 (if clinical T stage was T3).The combined prediction model showed significantly higher predictive accuracy than any single predictor (allP<0.001,DeLong’s test;Fig.S4).

A nomogram to predict the probability of pathological LN positivity of patients with PDAC was constructed based on multivariate logistic regression model (Fig.3 A).In the nomogram model,a score proportional to the estimated regression coefficient was assigned to each predictor.The maximum scores in the nomogram were 100,84.68,and 24.01 for risk score of the four-marker signature,CT-reported LN status,and clinical T stage,respectively.Prediction of the percentage probability of LN metastasis was determined as follows:Probability of LN metastasis(%)=A web-based calculator for the prediction of LN metastatic probability is available at www.pancreaticcancer.cn/NeoPangram.asp .

Model performance

The final model exhibited good discrimination ability in the development cohort (C-index=0.806,95% CI:0.753–0.859).The bias-corrected C-indexes using bootstrapping and 10-fold crossvalidation were 0.795 and 0.782,respectively.The good discrimination ability of the final model was confirmed in the internal validation cohort (C-index=0.742,95% CI:0.684–0.800),with bias-corrected estimates of 0.743 for bootstrapping and 0.751 for 10-fold cross-validation.Similarly,the external validation cohort yielded a C-index of 0.763 (95% CI:0.700–0.826),and bias-corrected estimates of the C-indexes were 0.761 and 0.758 using bootstrapping and 10-fold cross-validation methods,respectively.

Table 1 Patient characteristics stratified by lymph node status.

Table 2 Logistic regression model of the association between variables and lymph node metastasis.

Fig.2.Tumor marker selection using the least absolute shrinkage and selection operator (LASSO) regression model.A:Tuning parameter (λ) selection in the LASSO regression model using 10-fold cross-validation.The area under the curve (AUC) was plotted against log(λ).AUC values are shown as red dots,with error bars representing 95%confidence intervals (CIs).Dotted vertical lines indicate the optimal values by the minimum criteria and one-standard-error (1SE) criteria.B:LASSO coefficient profiles of the eight tumor markers.The dotted vertical line indicates the value chosen by 10-fold cross-validation,where optimal λ (λ=0.056;the 1SE criteria) resulted in 4 nonzero coefficients.L1 norm:the vector parameter of LASSO regression.

The calibration plots showed good agreement between model prediction and actual observation for predicting the probability of LN metastasis in all three cohorts (Fig.3 B).The Hosmer-Lemeshow test yielded a nonsignificant statistic (χ2=10.255,df=8,P=0.248),which indicated good model fit.

The clinical usefulness of the final model was confirmed by decision curve analyses in three cohorts (Fig.4).The decision curve showed that the net benefit changed with the threshold probability.The net benefit for decision-making based on our final model for threshold probabilities greater than 38% but not exceeding 77%was better than the net benefit for assuming that all or no patients had LN metastasis.

Optimal cut-off value for the probability of LN metastasis

Fig.3.Nomogram to predict individual lymph node (LN) metastasis probability (A) and calibration curves of the nomogram (B).A:To estimate the probability,a vertical line was drawn from each variable value to row 1 (points).Then,the three points were added to calculate the total points,and a vertical line was drawn from row 5 (total points)to row 6 (probability of LN metastasis).B:The Y-axis represents the observed probability of LN metastasis,while the X-axis represents the nomogram-predicted probability.The 45-degree dotted line (ideal line) represents a perfect prediction.The three calibration curves remained close to the ideal line,indicating that the nomogram-predicted probability of LN metastasis corresponds closely with the actual probability.

In addition to constructing the nomogram to predict the probability of pathological LN metastasis,we further distinguished two risk groups according to their probabilities,which were identified based on the optimal cut-off value of the ROC curve.The Youden index identified 0.72 as the optimal cut-off value,which was fully consistent with the closest-to-(0,1) criterion (Fig.5 A).Therefore,patients with probabilities equal to or greater than 0.72 were categorized as the high-risk group,and those with probabilities lower than 0.72 were categorized as the low-risk group.LN metastases were found in 22.1% (82/371) and 68.0% (297/437) of the low-risk and high-risk groups,respectively.The positive predictive value of this risk stratification using 0.72 as the cut-off value was 78.4%.Kaplan-Meier survival analysis showed that the curve of the low-risk group agreed with that of the LN-negative group,and the curve of the high-risk group also agreed with that of the LN-positive group (Fig.5 B and C).A significant difference in overall survival between the low-risk and high-risk groups was observed,which suggested the prognostic value of the nomogram(P=0.0 0 09 andP=0.0 0 01,Fig.5 B and C).

Fig.4.Decision curve analysis for the nomogram.At a threshold probability of > 25% and < 90% in the development cohort (A),> 38% and < 92% in the internal validation cohort (B),and > 18% and < 77% in the external validation cohort (C),the nomogram provided a larger net benefit than assuming all or no patients had lymph node (LN)metastasis.

Fig.5.A:Receiver operating characteristic curve showing the optimal cut-off value for the probability of LN metastasis.The Youden index identified 0.72 as the optimal cut-off point corresponding to J (the maximum vertical distance from the curve to the chance line).The closest-to-(0,1) criterion also identified 0.72 as the point that has the shortest distance from the curve to the (0,1) point (d=min { ) .B:Kaplan-Meier curves for overall survival using the true LN statusand nomogram-predicted LN status in the development cohort.C:Kaplan-Meier curves for overall survival using the true lymph node (LN) status and nomogram-predicted LN status in the internal validation cohort

Discussion

In this study,we developed and validated a novel tool for predicting pathological LN positivity in patients with resectable pancreatic cancer.The predictive tool was composed of three predictors:risk score of the four-marker signature,CT-reported LN status,and clinical T stage.By integrating these predictors,we developed a user-friendly nomogram that facilitated preoperative individualized prediction and stratification of patients with LN metastasis.The nomogram showed reasonably good discrimination,calibration,and clinical usefulness.Survival analysis demonstrated a good consistency of the survival curves between patients with the nomogram-predicted LN stratification and true LN status,indicating a high intrinsic accuracy of this model.

Clinical staging of pancreatic cancer is inaccurate,especially for clinical nodal staging.Approximately two-third of patients have nodal status incorrectly classified by clinical staging [4].In other words,preoperatively deemed early-stage disease may actually be more advanced disease (stage IIB or III;the 8th edition of the AJCC pancreatic cancer staging system) after pathological evaluation.Since normal-sized LNs may have micrometastases,and some enlarged LNs are confirmed to be reactive hyperplasia,the value of clinical nodal staging based solely on radiological imaging is limited.In an early study that evaluated combined CT and serum CA19–9 levels to predict LN metastasis in PDAC patients,the combination of positive CT findings and high CA19–9 levels(≥37 U/mL) had a higher positive rate for predicting node metastasis than did CT or serum CA19–9 levels alone [15].However,the study was limited by the relatively small number of patients included,and by the use of LNs with a diameter greater than 1 cm as the only criterion for positive LNs on CT.Our current study introduced LASSO regression to select the most relevant serum tumor markers for predicting LN metastasis.By shrinking the magnitude of all the coefficients to reduce overfitting and improve the prediction accuracy,LASSO regression identified four most predictive tumor markers:CA19–9,CA125,CA50,and CA242.This four-marker signature reduced the limitations of a single tumor marker;for example,CA19–9 may be undetectable in Lewis antigen-negative patients.In addition,we applied suspicious LN features (short axis>1 cm,abnormal round morphology,heterogeneity,or central necrosis) defined by the standardized imaging reporting template for PDAC [10],which facilitated more precise detection of metastatic LNs on CT.Furthermore,this study was not only internally validated but also externally validated using multicenter data from 250 patients with resected PDAC,indicating that the nomogram has good reliability and general applicability.

The use of neoadjuvant therapy for the treatment of PDAC remains controversial.Although neoadjuvant therapy is increasingly advocated in borderline resectable disease and is currently the preferred initial approach,it remains in the exploratory stage for resectable disease.The potential benefit of the neoadjuvant approach in this population included the identification of optimal surgical candidates [16],downstaging effect [6],improved R0 resection rate [17],decreased incidence of pancreatic fistula [18],and increased completion rate of multimodality therapy [19].In addition,a potential survival benefit of neoadjuvant therapy has also been reported.In a propensity score-matched analysis of 15237 patients with resectable PDAC,those who received neoadjuvant therapy had better overall survival than those who received upfront resection(median survival:26 vs.21 months;log-rankP<0.01) [5].However,this study was biased,and prospective evidence is needed to confirm these findings.Currently,the NCCN guidelines for PDAC do not recommend neoadjuvant therapy for clearly resectable patients without high-risk features,except in clinical trials [7].For those with high-risk features (i.e.,markedly elevated CA19–9 levels,large primary tumors,large regional LNs,excessive weight loss,or extreme pain),which are all likely to be signs of more advanced disease,consideration can be given to neoadjuvant therapy after biopsy confirmation.Nevertheless,there is no clear criterion for these high-risk features,which precludes the use of this strategy in clinical practice.Thus,we constructed a nomogram to predict LN metastasis probability by integrating predictive tumor markers,LN status on CT,and preoperative tumor stage,which covered three of the five high-risk features.We further set a probability threshold to separate low-risk and high-risk patient groups,producing a positive predictive value of 78.4%.This model might potentially be helpful to identify which patients with resectable disease will benefit from neoadjuvant therapy,namely,those at high risk of pathological LN positivity.

Adjuvant therapy has been proven to improve the outcomes of patients with resected pancreatic cancer [ 20,21 ].However,in realworld clinical practice,roughly half of patients do not receive or complete all intended cycles of adjuvant therapy after upfront resection [22].Given the low rate of multimodality therapy in the postoperative setting and the high degree of clinical understaging,an important question regarding the optimal management of PDAC patients presenting with resectable disease emerges.Neoadjuvant therapy,as an alternative treatment strategy,has been included in the American Society of Clinical Oncology management guidelines for patients with resectable disease [23].In this regard,neoadjuvant therapy tailored to increasing the proportion of patients who complete multimodality therapy and identifying more advanced disease based on our predictive nomogram could be a step further.Future randomized trials regarding neoadjuvant therapy versus upfront resection may use this online prediction model (www.pancreaticcancer.cn/NeoPangram.asp) to select appropriate patients and thus validate its usefulness.

The limitations of the present study include its retrospective design.In addition,differences may exist when detecting serum levels of tumor markers using different methods or devices.This might limit the generalizability of the nomogram,although multicenter external validation confirmed the robustness of the model(C-index:0.763).In addition,our current study may be limited by the absence of node-to-node correlation with CT findings and surgical pathology.However,the inclusion of node-to-node correlation only compromises the accuracy of the current model and complicates its use.

In conclusion,we constructed and validated a nomogram model based on the risk scores of a four-marker signature,CT-reported LN status,and clinical T stage for predicting LN metastasis in patients with resectable PDAC.Because no definitive criteria exist for identifying patients at high risk for LN metastasis,this prediction model may be useful in clinical practice because it may facilitate more accurately selecting candidates who may benefit from neoadjuvant therapy.

Acknowledgments

We thank Dr.Wei-Bo Chen (The First People’s Hospital of Changzhou),Dr.Wu-Ke Wang (Ningbo No.2 Hospital),and Drs.Jing-Song Yin and Fei-Zhuan Lin (Shaoxing People’s Hospital) for their assistance with data collection.

CRediT authorship contribution statement

Jie Hua:Data curation,Formal analysis,Methodology,Software,Validation,Visualization,Writing -original draft.Xue-Min Chen:Data curation,Methodology,Validation,Visualization,Writing -original draft.Yun-Jie Chen:Data curation,Methodology,Validation,Visualization,Writing -original draft.Bao-Chun Lu:Data curation,Methodology,Validation,Visualization,Writing -original draft.Jin Xu:Conceptualization,Funding acquisition,Supervision,Writing -review &editing.Wei Wang:Investigation,Resources,Supervision,Writing -review &editing.Si Shi:Conceptualization,Funding acquisition,Methodology,Resources,Supervision,Validation,Writing -review &editing.Xian-Jun Yu:Conceptualization,Funding acquisition,Investigation,Project administration,Supervision,Writing -review &editing.

Funding

This study was supported by grants from the National Natural Science Foundation of China (81772555,81802352 and 81902428),the National Science Foundation for Distinguished Young Scholars of China (81625016),the Shanghai Sailing Program (19YF1409400 and 20YF14090 0 0),the Shanghai Rising-Star Program (20QA1402100),the Shanghai Anticancer Association Young Eagle Program (SACA-CY19A06),the Clinical and Scientific Innovation Project of Shanghai Hospital Development Center (SHDC12018109 and SHDC12019109) and the Scientific Innovation Project of Shanghai Education Committee (2019-01-07-00-07-E0 0 057).

Ethical approval

The study was approved by the Institutional Review Board of Fudan University Shanghai Cancer Center,and the requirement for written informed consent was waived due to the retrospective nature of this study.

Competing interest

No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.

Supplementary materials

Supplementary material associated with this article can be found,in the online version,at doi:10.1016/j.hbpd.2020.12.020 .