Development and validation of a novel model to predict liver-related mortality in patients with idiosyncratic drug-induced liver injury

Yn Wng ,Ci-Lun Zou ,Jing Zhng ,Li-Xi Qiu ,Yong-F Hung ,Xin-Yn Zho ,Zhng-Shng Zou ,,Ji-Dong Ji ,

a Liver Research Center,Beijing Friendship Hospital,Capital Medical University;Key Laboratory on Translational Medicine on Cirrhosis;National Clinical Research Center for Digestive Diseases,Beijing 100050,China

b The Third Unit,Department of Hepatology,Beijing You’an Hospital,Capital Medical University,Beijing 100069,China

c Liver Transplantation Center,Beijing Friendship Hospital,Capital Medical University,National Clinical Research Center for Digestive Diseases,Beijing 100050,China

d Clinical Center for Pediatric Liver Transplantation,Capital Medical University,Beijing 100050,China

e Senior Department of Hepatology,the Fifth Medical Center of Chinese PLA General Hospital,Beijing 100039,China

Keywords: Liver injury Prognostic score Risk stratification Mortality

ABSTRACT Background: Early identification of patients with high mortality risk is critical for optimizing the clinical management of drug-induced liver injury (DILI).We aimed to develop and validate a new prognostic model to predict death within 6 months in DILI patients.Methods: This multicenter study retrospectively reviewed the medical records of DILI patients admitted to three hospitals.A DILI mortality predictive score was developed using multivariate logistic regression and was validated with area under the receiver operating characteristic curve (AUC).A high-mortality-risk subgroup was identified according to the score.Results: Three independent DILI cohorts,including one derivation cohort (n=741) and two validation cohorts (n=650,n=617) were recruited.The DILI mortality predictive (DMP) score was calculated using parameters at disease onset as follows: 1.913 × international normalized ratio+0.060 × total bilirubin (mg/dL)+0.439 × aspartate aminotransferase/alanine aminotransferase– 1.579 × albumin (g/dL)–0.006 × platelet count (10 9/L)+9.662.The predictive performance for 6-month mortality of DMP score was desirable,with an AUC of 0.941 (95% CI: 0.922-0.957),0.931 (0.908-0.949) and 0.960 (0.942-0.974) in the derivation,validation cohorts 1 and 2,respectively.DILI patients with a DMP score ≥8.5 were stratified into high-risk group,whose mortality rates were 23-,36-,and 45-fold higher than those of other patients in the three cohorts.Conclusions: The novel model based on common laboratory findings can accurately predict mortality within 6 months in DILI patients,which should serve as an effective guidance for management of DILI in clinical practice.

Introduction

Idiosyncratic drug-induced liver injury (iDILI) is one of the leading causes of acute liver failure (ALF) and accounts for 11.9% of ALF in the USA [1,2].About 66.4% of iDILI-related ALF would die or need liver transplantation (LT),as reported by the ALF Study Group [2].Therefore,it is of paramount importance to promptly identify these patients who are at high risk of mortality and mount timely intensive care.

To date,several predictive models have been proposed for the prediction of ALF associated with DILI,including Hy’s law [3],new Hy’s law (nHy’s Law) [4],an algorithm proposed by Robles-Diaz et al.(named Robles-Diaz model in our study) [4],and druginduced liver toxicity ALF score (DrILTox ALF score) [5].Recently,a model based on model for end-stage liver disease score (MELD),serum albumin (ALB) and comorbidity burden,was proposed by Ghabril et al.[6],which was investigated for the mortality within 6 months (named Ghabril Model in our study).

However,these prognostic models proposed were all mainly derived on Western populations,where the offending drugs,spectrum of comorbidities and the genetic background are generally different from patients from the Eastern countries.For example,much higher percentage of herbal-based remedies were found to be the implicated agents for DILI patients from the Asian countries [7–11].What’s more,our previous data show that the comorbidity burden of Chinese DILI patients was lower than that in the developed countries: the Charlson comorbidity index (CCI)was lower and renal insufficiency or renal failure at DILI onset was rare [11].Thus,this may preclude the direct extrapolation of the previously proposed models to Asian patients.

Therefore,in the present study,we developed and validated a novel death/LT prediction score from easily available biochemical measurements in Asian DILI patients which composed high percentage of herbal and dietary supplements (HDS),aiming to improve their clinical outcomes through better stratification and individualized care.

Methods

Study design

This is a multicenter retrospective cohort study.The protocol was reviewed and approved by the Institutional Review Board with a waiver for informed consent at the three participating centers(KY-2021-9-16-1,2020-P2-300-01 and LL-2018-131-K).

The derivative cohort included patients diagnosed with DILI and hospitalized at the Fifth Medical Center of Chinese PLA General Hospital,Beijing,China,between January 1,2016,and June 30,2019.

The external validation cohorts were from the other two hospitals: one from Beijing Friendship Hospital,Capital Medical University,Beijing,China,between January 1,2009,and June 30,2021;the other from Beijing You’an Hospital,Capital Medical University,Beijing,China,between January 1,2011,and December 31,2014.

Screening and review process of DILI patients

The electronic medical record databases for hospitalized patients of the three hospitals were searched using Beijing version of International Classification of Disease (ICD)-10 codes of K71.102,K71.601,K71.901,and the diagnosis terms of “drug induced liver injury”,“drug induced liver failure”,and “drug-induced hepatitis”.

All the cases were reviewed and independently evaluated by three hepatologists according to the same inclusion and exclusion criteria to ensure the validity of the data.Medical history,clinical,laboratory,and radiographic data of all the patients were systematically reviewed and follow-up data were captured from the hospital database and through telephone interviews.

Inclusion and exclusion criteria of DILI patients

Inclusion criteria: (1) the liver function tests should fulfill one of the following criteria: (i) alanine aminotransferase (ALT) level ≥5 × upper limit of normal (ULN),(ii) alkaline phosphatase (ALP)level ≥2 × ULN,or (iii) ALT level ≥3 × ULN and total bilirubin(TB) >2 × ULN.(2) The duration from DILI onset was less than 3 months.(3) Causality between the candidate drug and liver injury was assessed with Roussel Uclaf Causality Assessment Method(RUCAM) [12,13].Patients with RUCAM score equal and higher than 6 would be evaluated by at least two of three hepatologists,and patients scoring between 3 and 5 should be evaluated by all three hepatologists.

Exclusion criteria: (1) positive results in anti-hepatitis A virus IgM,anti-hepatitis E virus IgM,hepatitis B surface antigen,antihepatitis C virus IgG and/or hepatitis C virus RNA,Epstein-Barr virus DNA,and cytomegalovirus DNA;(2) pre-existing autoimmune hepatitis,primary biliary cholangitis or primary sclerosing cholangitis;(3) alcoholic liver disease and non-alcoholic steatohepatitis;(4) Wilson disease and hemochromatosis;(5) biliary obstruction based on ultrasound or magnetic resonance cholangiopancreatography (included when a cholestatic pattern was presented);(6) a history of LT prior to DILI onset;(7) direct hepatotoxicity or indirect hepatotoxicity;and (8) incomplete key data or lost to followup.

Definition of DILI onset,clinical type,severity and comorbidity

Data at DILI onset were defined as first available laboratory test fulfilling DILI criteria,which was the accessible data either of primary care,secondary or tertiary hospitals.DILI was classified as hepatocellular,cholestatic,or mixed liver injury according to R value,defined as (ALT/ULN)/(ALP/ULN) and calculated based on the liver function tests at DILI onset [14].The severity of liver injury was graded as mild,moderate,severe,fatal/LT,according to International DILI Expert Working Group [14].The comorbidity of patients was calculated by CCI [15].

Definition of study endpoints

Liver-related death was recorded if DILI was regarded as primary or contributory cause of death [16].The indication of LT was according to King’s College criteria [17].The composite endpoint was defined as liver-related death or LT,also called mortality in this study,since those patients receiving LT should have been dead without the procedure [18].

Development of a new prognostic score

Firstly,demographics and laboratory data which may be related to death/LT according to previous reports or clinical practice were chosen as candidates for univariate logistic regression analysis.Those factors with two-sidedPvalue <0.10 or with empirical significance in clinical practice were reserved for the next step.Secondly,forward stepwise multivariate logistic regression analysis,with two-sidedPvalue <0.05 by likelihood ratio test as entry criterion and two-sidedPvalue >0.10 as removal criterion,were conducted to develop the prognostic model.Thirdly,the performance of the new score was compared with the Hy’s law,nHy’s law,Robles-Diaz model,DrILTox ALF score,and Ghabril model.

External validation of a new prognostic score

Patients from validation cohorts 1 and 2 were applied to the external validation of the new model independently.

Risk stratification

Considering that near 10% of DILI patients died or underwent LT according to the previous study [16],we assume that patients over 90th percentile score of the new prognostic model would be at high risk of mortality.Therefore,the patients were divided into low-risk and high-risk of death/LT based on the 90th percentile score of the new prognostic model.

Statistical analysis

Continuous variables were presented as median (interquartile range) and analyzed by Kruskal-Wallis test.Categorical variables were presented as numbers (%) and analyzed using Chi-square test or Fisher’s exact test.

Univariate logistic regression analysis was performed to estimate the effect of various variables on the 6-month death/LT.Odds ratio (OR) and their 95% confidence interval (CI) along with correspondingPvalues were estimated simultaneously.Forward stepwise multivariate logistic regression analysis was used to develop a new model to predict 6-month death/LT,named DILI mortality predictive score (DMP score).

The predictive accuracy of DMP score for 6-month death/LT was displayed with a calibration plot,and the overall performance across the range of DMP score was assessed with the Hosmer-Lemeshow test,McFadden’s pseudo-R2and scaled Brier score [19].

A nomogram to predict 6-month mortality risk was constructed according to DMP score,and the discriminative ability of DMP score was visualized by Kaplan-Meier curves with log-rank test,in which the two groups were stratified by the 90th percentile of DMP score.

The area under the receiver operating characteristic curve(AUC),sensitivity and specificity of the new model for the prediction of 28-day,90-day and 6-month mortality was compared with Hy’s law,nHy’s law,Robles-Diaz model,DrILTox ALF score,and Ghabril model using DeLong’s test.Integrated discrimination improvement (IDI) was calculated to compare the discriminative accuracy of the DMP score and Ghabril model furthermore.

The external validation of the performance of the prognostic score used the same methods applied to the derivation data.A two-sidedPvalue <0.05 was considered statistically significant.

SPSS 22.0 (IBM Corporation,Armonk,NY,USA) and R Statistical Software (version 4.2.0,R Core Team 2022)were adopted to perform the above statistical analysis and graphics.

Results

Clinical characteristics and categories of implicated medications in the derivation cohort

There was a total of 741 patients in the derivation cohort(Fig.1).Patients were grouped into liver-related death/LT within 6 months group and transplant-free survival (TFS) group based on their outcomes.Cases with non-liver-related death (two cases in the derivation cohort,one in the validation cohort 1,and none in the validation cohort 2) were excluded from the current study.

As shown in Table 1,the median age of the derivation cohort was 48.0 years and the percentage of female was 61.9%.During a median follow-up of 45.4 months,55 patients died due to liverrelated cause within 6 months (detailed characteristics are described in Table S1).Eighteen patients died within 28 days,28 between 29 days and 90 days,and 9 between 91 days and 6 months.Most of the patients have a low comorbidity burden and only 11(1.5%) patients had CCI higher than 2.

The categories of implicated drugs were divided into HDS,conventional agents,and HDS plus conventional agents.A total of 45.1% cases were attributed to HDS,22.5% to conventional agents,and 32.4% to HDS plus conventional agents (Fig.2).The clinical characteristics and mortality in patients who take HDS and those taking conventional agents only were described in Table S2.No difference in the categories of implicated agents and injury patterns was found between the death/LT and TFS groups.

Fig.2.Implicated agents and HDS categories in the derivation cohort.DILI: drug-induced liver injury;HDS: herbal and dietary supplements;NSAIDs: nonsteroidal antiinflammatory drugs.

Levels of aspartate aminotransferase (AST),AST/ALT,TB,total bile acid (TBA),white blood cell (WBC) count,international normalized ratio (INR),and serum creatinine (Cr) in the liver-related death/LT group were significantly higher than those in the TFS group,whereas levels of ALB,platelet (PLT) count,hemoglobin and cholinesterase were significantly lower in the liver-related death/LT group.

Development of a new DILI mortality prediction score

After univariate logistic stepwise analysis,hemoglobin,WBC,PLT,INR,AST/ALT,TB,TBA,ALB and Cr were all associated with liver-related death/LT.A new DILI DMP score comprising INR,TB,ALB,AST/ALT and PLT,was developed using multivariate logistic analysis (Table 2).

Table 2Factors at DILI onset associated with 6-month liver-related death/LT in DILI patients on univariate and multivariate analyses.

The new score was calculated as follows: The risk score=1.913 × INR+0.060 × TB (mg/dL)+0.439 × AST/ALT–1.579 × ALB (g/dL)– 0.006 × PLT (× 109/L)+9.662.The predictive ability of the DMP score for 6-month death/LT in the derivation cohort,calculated as AUC was 0.941 (95% CI: 0.922-0.957,Table 3).The nomogram of the DMP score was shown in Fig.S1.

The sensitivity,specificity,positive predictive value and negative predictive value of the DMP score for 6-month dealth/LT in the derivation cohort were 90.9%,87.9%,37.6% and 99.2%,respectively(Table 3).

External validation of the new DMP score

Among a total of 650 and 617 patients in the validation cohorts 1 and 2,respectively,20 and 41 cases died or underwent LT due to liver-related causes within 6 months (Fig.1).The clinical characteristics of the two validation cohorts were described in Table S3.

A comparable overall predictive ability for 6-month mortality was found in the two validation cohorts as in the derivation cohort(validation cohort 1: AUC=0.931,95% CI: 0.908-0.949;validation cohort 2: AUC=0.960,95% CI: 0.942-0.974;Table 3).

Calibration of the DMP score

Calibration of the DMP score was estimated with a calibration plot and summarized across the full range of the score (Fig.3).The observed mortality and predicted probabilities of death/LT of derivation group were similar in 6 months (Hosmer-Lemeshowχ2=5.469,P=0.706;R2=0.484;Brier score=0.039) (Fig.3 B).The observed mortality and predicted probabilities of death/LT of the two validation groups were also similar in 6 months (validation cohort 1: Hosmer-Lemeshowχ2=4.103,P=0.848,R2=0.376,Brier score=0.022;validation cohort 2: Hosmer-Lemeshowχ2=7.528,P=0.481,R2=0.574,Brier score=0.031)(Fig.3 D and F).

Fig.3.Calibration of the DMP score for predicting 6-month death/liver transplantation in the derivation and two validation cohorts.A and B: The derivation group;C and D: the validation cohort 1;E and F: the validation cohort 2.Observed vs.predicted mortality rates according to the approximate deciles of the scores.Patients were grouped into quintiles sorted by the predicted risk score.

The predicted mortality of death/LT less than 90th percentile of the DMP score had accuracy of >85% in derivation group (Fig.3 A)and >90% in the two validation groups (Fig.3 C and E).The above results showed the DMP score exhibited good predictive accuracy of 6-month death/LT,especially in the score less than 90th percentile.

Risk stratification by the DMP score

The discriminative ability of the DMP score was visualized by plotting the transplant-free survival curves for two risk groups according to the 90th percentile of the score.Patients were separated into low-risk (<8.5) and high-risk (≥8.5) groups.The 6-month mortality rates of the high-risk group were significantly higher than those of the low-risk group in the derivation and two validation cohorts (low-risk,2.3%,1.1% and 1.1%;high-risk,52.6%,39.4%and 50.0%,P<0.001,for derivation,validation cohorts 1 and 2,respectively) (Fig.4).

Fig.4.Risk stratification of the DMP score.A: The derivation group;B: the validation group 1;C: the validation group 2.Cumulative incidence of death/LT within 6 months according to the DMP score classification rule (low-risk: DMP score <8.5;high-risk: DMP score ≥8.5).CI: confidence interval;LT: liver transplantation.

The performance of the DMP score for the prediction of 28-day and 90-day mortalities

Though we aimed to develop the DMP score to predict 6-month mortality,we also assessed the predictive performance of the DMP score for 28-day and 90-day death/LT in the derivation and two validation cohorts.The DMP score showed excellent predictive ability of 28-day and 90-day death/LT in the derivation and two validation cohorts with AUC all higher than 0.949 (Table 3).

Comparison of the discriminative accuracy of the DMP score with the previously proposed predictive models

As shown in Fig.5,the predictive performance of the DMP score was superior in comparison with previously proposed predictive scores.The AUCs of the DMP score were significantly higher than those of Hy’s law,nHy’s law,Robles-Diaz model,DrILTox ALF score and Ghabril model for the prediction of 28-day,90-day and 6-month mortality.Though the AUCs of the DMP score and the Ghabril model were significantly different,the values of the AUCs of the two models were close [0.966 (0.950-0.978) vs.0.915 (0.893-0.934),P=0.003;0.971 (0.956-0.982) vs.0.937 (0.917-0.953),P=0.002;0.941 (0.921-0.957) vs.0.912 (0.889-0.931),P=0.030,for 28-day,90-day,and 6-month mortality,respectively].

Fig.5.Comparison of the discriminative accuracy of DMP score and the previous proposed predictive models for the prediction of 28-day,90-day and 6-month liver-related death/LT of DILI patients in the derivation cohort.A: AUC of the six models for the prediction of death/LT within 28 days.B: AUC of the six models for the prediction of death/LT within 90 days.C: AUC of the six models for the prediction of death/LT within 6 months.D: Comparison of AUC of the six models for the prediction of 28-day,90-day and 6-month liver-related death/LT.

To further compare the improvement in model discrimination,we further calculated and compared IDIs of the DMP score and the Ghabril model.The results showed that the DMP score significantly outperformed the Ghabril model (IDI=0.174,z=2.502,P=0.012).

Discussion

In this study,we developed and externally validated a new model (the DMP score) among a total of 2008 patients.The DMP score based on INR,TB,ALB,AST/ALT and PLT count at DILI onset,can accurately predict the 28-day,90-day and 6-month liverrelated death/LT of DILI patients.The DMP score can accurately identify the high-risk DILI patients,who are at 20-fold higher risk of death/LT than the low-risk group.

The DMP score comprises five simple,objective,non-invasive and readily available variables: INR,TB,ALB,PLT count,and AST/ALT.They are not only relevant to the severity of liver injury (TB,INR,AST/ALT),but also crude measures of hepatocellular synthetic function (ALB) [20].PLT count can be affected by ALFinduced liver injury and inflammation [21].Our study had similar results with the USA Drug-Induced Liver Injury Network that patients with lower PLT counts at DILI onset were more likely to experience a liver-related death or undergo LT [22].Each of the above five variables had been shown in previously proposed models to predict serious outcomes (ALF and death) in DILI patients.Furthermore,the simple calculation of the score could also facilitate its implementation in clinical practice.

Of note,the DMP score was proposed based on a population whose genetic background and implicated medication for DILI were different from the Western populations.Firstly,a greater percentage of HDS in the causative agents was found in our cohorts.This is consistent with previous reports that a much higher percentage of herbal-based remedies was found in the East(Japan [23],China [7,8,11],India [9],and Korea [10]and so on)than in the West [24].Secondly,genetic factors are believed to contribute to DILI susceptibility and various polymorphisms have been identified to be associated with drug metabolism and liver injury [25–30].Thirdly,the ratio of patients with CCI >2 in the derivation cohort was obviously lower than that of the cohort of Ghabril model [1.5% (11/741) vs.17.3% (53/306)][6].It indicated that the overall burden of comorbid diseases of our cohort was obviously lower.Thus,a model comprising no CCI would therefore simplify the process of evaluation and may promote the discriminative accuracy of liver-related mortality.

This multicenter cohort study was derived in a big cohort and externally validated in two independent cohorts,with sizeable subjects in the death/LT group within 6 months.This made our study qualified to develop a reliable prognostic model.To ensure the validity of the data,all the medical records and follow-up data,including prognosis,were independently evaluated by three hepatologists according to the same inclusion and exclusion criteria.Although patients in the two validation cohorts had milder severity of liver injury (lower AST,ALT and TB,and higher ALB) than the derivation cohort,the predictabilities are similar in the two groups,further strengthening the reliability of the DMP score.

The calibration performance of the DMP score also showed good agreement between the predicted and the observed 6-month mortality in both the derivation and two validation cohorts.The calibration across the full range exhibited excellent accuracy for those with DMP score less than 90th percentile,higher than 85%in the derivation group and 90% in the two validation groups.Of note is that the negative predictive value of the DMP score was 99.2%.These results indicated that the DMP score is a useful tool for discriminating low-and high-risk mortality patients,especially in the exclusion of low-risk patients whom further intensive care is unnecessary.

Considering that most predictive models of ALF or death predict mortality of 28-day and 90-day,we also assessed the predictive performance of the DMP score for 28-day and 90-day death/LT in the derivation and two validation cohorts.Indeed,the DMP score also showed excellent predictive ability of both 28-day and 90-day mortality in all three cohorts,even better than that of 6-month mortality.

Finally,we also compared the predictive performance of the DMP score with other predictive models proposed earlier,including Hy’s law,nHy’s law,Robles-Diaz model,DrILTox ALF score and Ghabril model.The AUC of the DMP score for 6-month death/LT was significantly higher than all the above models.In Ghabril model,medical comorbidity was independently associated with all-cause mortality in DILI patients.However,in our study,the very low percentage of patients (only 11 cases in 741 cases) with high comorbidity burden (as measured by CCI >2) [15]made it reasonable to construct a model without CCI in the setting with lower comorbidity burden.Actually,the DMP score had better discriminative accuracy of high-mortality-risk patients (IDI=0.174,P=0.012),although Ghabril model can accurately predict 6-month mortality (AUC up to 0.912).However,considering the different endpoints (liver-related mortality vs.all-cause mortality) and comorbidity burdens of our study and that of the Ghabril model,whether the DMP score is applicable to predict all-cause mortality in DILI patients which composed of high burden of medical comorbidity remained to be validated.

Our study has several limitations.Firstly,this is a retrospective cohort study.But we hope that well documented data,stringent and consistent inclusion and exclusion criteria,and three independent derivation and external validation cohorts with large sample size would offset some of this downside.Secondly,all subjects were hospitalized patients;therefore,our results and conclusion would not be directly extrapolated to the outpatients whose liver injury might be milder.However,the high negative predictive value of our model can guarantee the accuracy when applied to outpatients with milder severity.We plan to include both inpatient and outpatient DILI patients in a prospective cohort study in the future.Thirdly,patients with underlying liver diseases,such as viral hepatitis B and C and steatohepatitis,were excluded.Therefore,whether our model is applicable for DILI patients with underlying liver diseases requires further investigation.

In conclusion,our study showed that DMP score composed of five objecti ve and commonly used variables can accurately predict 28-day,90-day,and 6-month death/LT in DILI patients.It could be a useful tool for clinicians to identify high-risk patients who need intensive care,thereby implementing individualized management.

Acknowledgments

The authors would like to thank Aileen Wee and Ji-Min Liu for review &editing on this manuscript.The authors would like to thank Zi-Kun Ma,Tian-Tian Guo,Yu Wang,Jing-Yi Liu,Li-Wei Liu,Ying Sun,Bin-Xia Chang,Lin Han,Ang Huang for data curation on this study.The authors are grateful to Wei Chen for help with the preparation of figures.The authors are grateful to Min Li and Yuan-Yuan Kong for help with the methodology.

CRediTauthorshipcontributionstatement

YanWang:Conceptualization,Data curation,Formal analysis,Investigation,Methodology,Visualization,Writing– original draft,Writing– review &editing.Cai-LunZou:Data curation,Formal analysis,Investigation,Methodology,Visualization,Writing– original draft,Writing– review &editing.JingZhang:Data curation,Validation.Li-XiaQiu:Data curation,Validation.Yong-FaHuang:Methodology,Visualization,Writing– review &editing.Xin-Yan Zhao:Conceptualization,Funding acquisition,Resources,Supervision,Writing– review &editing.Zheng-ShengZou:Conceptualization,Resources,Supervision,Writing– review &editing.Ji-Dong Jia:Conceptualization,Funding acquisition,Resources,Supervision,Writing– review &editing.

Funding

This study was supported by grants from the National Key R&D Program of China (2021ZD0113200) and the National Natural Science Foundation of China (81900526).

Ethicalapproval

The protocol was reviewed and approved by the Institutional Review Board with a waiver for informed consent at the three participating centers (KY-2021-9-16-1,2020-P2-300-01 and LL-2018-131-K).

Competing interest

No benefits in any form have been received or will be received from a commercial party related directly or indirectly to the subject of this article.

Supplementary materials

Supplementary material associated with this article can be found,in the online version,at doi:10.1016/j.hbpd.2023.06.002.