Development and validation of a model integrating clinical and coronary lesion-based functional assessment for longterm risk prediction in PCI patients

Shao-Yu WU,Rui ZHANG,Sheng YUAN,Zhong-Xing CAI,Chang-Dong GUAN,Tong-Qiang ZOU,Li-Hua XIE,Ke-Fei DOU,4,✉

1. State Key Laboratory of Cardiovascular Disease,Beijing,China;2. Cardiometabolic Medicine Center,Fu Wai Hospital,National Center for Cardiovascular Diseases,Chinese Academy of Medical Sciences and Peking Union Medical College,Beijing,China;3. Catheterization Laboratories,Fu Wai Hospital,National Center for Cardiovascular Diseases,Chinese Academy of Medical Sciences and Peking Union Medical College,Beijing,China;4. Department of Cardiology,Fu Wai Hospital,National Center for Cardiovascular Diseases,Chinese Academy of Medical Sciences and Peking Union Medical College,Beijing,China

ABSTRACT OBJECTIVES To establish a scoring system combining the ACEF score and the quantitative blood flow ratio (QFR) to improve the long-term risk prediction of patients undergoing percutaneous coronary intervention (PCI).METHODS In this population-based cohort study,a total of 46 features,including patient clinical and coronary lesion characteristics,were assessed for analysis through machine learning models.The ACEF-QFR scoring system was developed using 1263 consecutive cases of CAD patients after PCI in PANDA III trial database.The newly developed score was then validated on the other remaining 542 patients in the cohort.RESULTS In both the Random Forest Model and the DeepSurv Model,age,renal function (creatinine),cardiac function (LVEF)and post-PCI coronary physiological index (QFR) were identified and confirmed to be significant predictive factors for 2-year adverse cardiac events.The ACEF-QFR score was constructed based on the developmental dataset and computed as age (years)/EF(%)+1 (if creatinine ≥ 2.0 mg/dL)+1 (if post-PCI QFR ≤ 0.92).The performance of the ACEF-QFR scoring system was preliminarily evaluated in the developmental dataset,and then further explored in the validation dataset.The ACEF-QFR score showed superior discrimination (C-statistic=0.651;95% CI: 0.611-0.691,P ＜ 0.05 versus post-PCI physiological index and other commonly used risk scores) and excellent calibration (Hosmer-Lemeshow χ2=7.070;P=0.529) for predicting 2-year patient-oriented composite endpoint (POCE).The good prognostic value of the ACEF-QFR score was further validated by multivariable Cox regression and Kaplan-Meier analysis (adjusted HR=1.89;95% CI: 1.18-3.04;log-rank P ＜ 0.01) after stratified the patients into high-risk group and low-risk group.CONCLUSIONS An improved scoring system combining clinical and coronary lesion-based functional variables (ACEF-QFR)was developed,and its ability for prognostic prediction in patients with PCI was further validated to be significantly better than the post-PCI physiological index and other commonly used risk scores.

Achieving myocardial revascularization is the key to the treatment of coronary artery disease (CAD).With the widespread application of percutaneous coronary intervention (PCI)and the increasing clinical complexity of CAD patients,evaluating treatment effects and predicting prognosis after PCI is extremely important in clinical practice.At present,few accurate and easy-to-use score algorithms can assist physicians in predicting individual outcomes in clinical cardiology scenarios.The ACEF (age,serum creatinine,and ejection fraction) score was proposed by Ranucci,et al.[1]in 2009 to predict the risk of operative mortality in cardiac operations and has been validated in many procedures and clinical settings.Based on the law of parsimony,this scoring model consisted of only three factors (age,serum creatinine,and ejection fraction) but exhibited relatively accurate prognostic prediction ability.[2]However,for risk assessment in patients with left main or multivessel CAD,the ACEF model does not contain any coronary lesion-based parameters,which is an important prognostic indicator for those undergoing PCI.

The purpose of myocardial revascularization is to minimize residual ischemia.[3]Currently,physiologic assessment after PCI is an effective indicator to quantify residual ischemia,and multiple studies have confirmed the prognostic value of postoperative fractional flow reserve (FFR) assessment.[4]The Quantitative flow ratio (QFR) is a novel parameter for deriving FFR without hyperemia induction or the use of pressure wire.[4,5]Previous studies have validated that the QFR has good diagnostic accuracy in identifying coronary ischemic burden and quantifying the extent of residual stenosis,with FFR as the reference standard.[6-8]Our recent study also found that the post-PCI QFR was a strong independent predictor of cardiac mortality,and a QFR ≤ 0.92 after PCI was associated with significant increases in adverse events.[9,10]Therefore,whether combining clinical and coronary lesion-based physiological variables can improve the predictive power of clinical endpoints in patients undergoing PCI remains to be verified.

In the view of previous study,the ACEF score could be considered a skeleton,and different variables can be added depending on specific clinical scenarios.[11]In this study,we first established a scoring system combining the ACEF and post-PCI QFR after identifying features that significantly contributed to adverse outcomes through machine learning models.Second,we evaluated the performance of the ACEF-QFR score for long-term cardiac event prediction based on the validation dataset.

METHODS

Study Design and Population

The purpose of this study was to construct a scoring system integrating clinical and coronary lesionbased functional assessment and investigate its effect on predicting long-term adverse cardiac events.The population and research design process were presented in Figure 1.We selected patients from the PANDA III trial to develop and validate the scoring model.The PANDA III trial is a large multicenter population-wide study involving a cohort of 2348 consecutive patients who underwent PCI at 46 sites in China.The design and main results of the trial have been previously published.[12]Follow-up and examination of all patients were completed in August 2021.

Figure 1 Flowchart of the study. A hierarchical listing based on the processes for QFR assessment was used to identify the exclusion reasons per patient: (1) no calibration data in DICOM filet;(2) no analyzable 2 projections;(3) poor angiographic quality images affecting contour delineation;(4) ostial lesions less than 3mm from the aorta;(5) severe vessel overlap or tortuosity at the stenotic segments;and (6) unable to complete accurate frame count.DICOM: Digital Imaging and Communication in Medicine;DM: diabetes mellitus;PCI: percutaneous coronary intervention;QFR: quantitative flow ratio.

In the present study,the inclusion criteria were(1) patients who had qualifying ejection fraction (EF)and serum creatinine (Scr) measurements before PCI and (2) patients with analyzable postoperative QFR in all measurement-requiring vessels.The exclusion criteria were (1) patients with a history of CABG and (2) patients who underwent two different coronary revascularizations during hospitalization.

The present study was approved by the ethics committee of the Cardiovascular Institute of Fuwai Hospital,and followed the Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) reporting guideline.Written informed consent was obtained from all patients before the PCI procedure.

Measurements of QFR

All patients in the PANDA III trial underwent QFRmeasurements before and after PCI.Vessels with lesions with diameter stenosis ≥ 50% and reference luminal diameter ≥ 2.0 mm by visual assessment were required to be measured.Off-line QFR measurement was conducted by well-trained technicians using a QFR system (AngioPlus,Pulse Medical Imaging Technology,Shanghai,China).The details of QFR analysis were the same as the method previously reported.[12]

Given that one patient may have multiple vessels with measured QFR,we used the average QFR (the mean QFR value of three vessels) and the lowest QFR (the minimum QFR value of three vessels) to define the individual's coronary physiological index.According to previous studies,a post-PCI QFR ≤0.92 was confirmed to be independently associated with the risk of MACE at 2 years after PCI.[9,10]After analyzing 2227 lesion vessels in the development dataset,the best cutoff value for the post-PCI QFR for physiological significance was 0.916,close to the previous relevant studies.Therefore,we chose 0.92 as the cutoff value of the post-PCI QFR to define optimal/suboptimal functional results based on our present study.For those patients suffering from multivessel CAD,if the post-PCI QFR of any one vessel was no more than 0.92 (QFR ≤ 0.92),then the patients were considered to have a poor postoperative coronary functional evaluation.Thus,the post-PCI QFR of all patients was dichotomized by the cutoff value: post-PCI QFR ≤ 0.92 and post-PCI QFR＞ 0.92.

Data Collection and Classic Score Calculations

Clinical data were obtained via a review of the medical records.All baseline and procedural angiograms were reviewed and analyzed by reviewers in a blinded independent core laboratory (CCRF,Beijing,China).Score calculations were performed by experienced technicians blinded to the clinical outcomes in the core laboratory of Fuwai Hospital,Chinese Academy of Medical Sciences.If ratings were different between observers,consensus was reached after reexamination.

The ACEF score was calculated using the following formula: age (years)/ejection fraction (%)+1 (if the serum creatinine level ≥ 2.0 mg/dL).[1]The modified ACEF score was calculated using the following formula: age (years)/ejection fraction (%)+1 point for every 10 mL/min reduction in creatinine clearance below 60 mL/min/1.73 m2(up to a maximum of 6 points).[13]The SYNTAX score was calculated according to the SYNTAX score algorithm (http://syntaxscore.com/),determined as the sum of the points assigned to each individual lesion identified in the coronary artery with ＞ 50% diameter stenosis in vessels ＞ 1.5 mm diameter.[14]The residual SYNTAX score (rSS) was defined as the remaining SYNTAX score of the lesions after PCI.If a patient underwent PCI two or more times because of planned staged revascularization,the rSS was calculated after the final procedure.[15]

End Points and Follow-up

The primary end point was 2-year patient-oriented composite endpoint (POCE).defined as a patient-oriented composite endpoint including all-cause death,all myocardial infarction,or any ischemiadriven revascularization.The secondary end points included the individual components of POCE,which were defined according to standards by the Academic Research Consortium.[16]All adverse events were judged and determined by an independent clinical events committee.

All patients underwent clinical follow-up by clinic visit or telephone.The vital signs of patients were assessed during outpatient follow-up or through their family-reported data.If necessary,vital status was determined from the National Death Registry of China (Centers for Disease Control and Prevention’s Death Registry System).

Significant Feature Identification

Univariate and multivariate Cox regression analyses were performed on each variable,and the significant predictors for 2-year POCE were confirmed.We further apply machine learning technique to identify the clinical features of patients and the characteristics of their coronary vessels which have a significant impact on the outcome.

Random Forests,an ensemble algorithm based on classification and regression trees (CART) trained on bootstrapped samples and randomly selected features,is commonly used in various prediction problems in biological studies,and has been proved to have superior performance than other classification and regression models.[17]Significant features were selected by information gain attribute ranking,which is a measure of the effectiveness of an attribute in classifying the training data.Information gain is defined as the amount of entropy reduction of a class,which reflects additional information regarding the class provided by the feature.

Given that the recorded patients’ events in PANDA III trial is survival data,the learning survival neural network (DeepSurv) was then used to analyze the correlation between patient-individual features and their survival outcomes,which is a deep learning algorithm combining deep feed-forward neural network and the Cox proportional hazards model in survival analysis.The DeepSurv method was initially developed by Katzman,et al.[18]for survival analysis,and demonstrated to be able to learn the highly intricate and linear/nonlinear associations between prognostic clinical characteristics and an individual’s risk of death.A deep neural network is trained to model the log-risk function h(x) in the hazards function.The network takes patient's information as input.Several fully-connected layers followed by the dropout layers embed the input into a hidden vector.After that,the hidden vector is fed into the output layer which contains a linear activation function to estimate the log-risk function.Finally,feature component weightings of each variable for 2-year POCE prediction were derived.

Statistical Analysis

The scoring system was constructed based on the development dataset using the Cox proportional hazard regression model.The detailed derivation and calculation process would be introduced at the result part.Receiver operating characteristic (ROC) curves were used to estimate the prognostic value of post-PCI physiological index and all risk scores.The discrimination was determined based on the area under the receiver operating characteristic curve(AUC),[19]and AUCs were compared using Delong’s test.[20]The calibration of physiological index and risk scores was measured according to Hosmer-Lemeshow (HL) statistics.[21]A calibration curve was used to evaluate the agreement between the observed versus predicted outcome risk derived by the scoring model.Net reclassification improvement (NRI) and integrated discrimination improvement (IDI) were used to calculate the ability of the new risk score to reclassify the risk of POCE in contrast to that of other risk scores.[22]Decision curve analysis (DCA) and clinical impact curve (CIC) analysis were performed to further evaluate the clinical utility of the scoring system.Multivariable-adjusted 2-year clinical outcomes and cumulative event rates were calculated based on the Kaplan-Meier method and compared by log-rank test.

Continuous variables were described as the mean± SD.Student’sttest or nonparametric test was performed for comparisons of continuous variables.Categorical variables were expressed as counts and percentages and compared using the chi-squared or Fisher's exact test as appropriate.AllPvalues were 2-tailed,and aPvalue ＜ 0.05 was considered statistically significant.All analyses were performed with the R version 4.1.2 system.

RESULTS

Population and Rates of Adverse Events

A total of 2348 patients in the PANDA III trial were enrolled.Based on the inclusion and exclusion criteria,1805 patients with 2227 lesion vessels of PANDA III were included in the present data analysis.A total of 1263 (70%) patients were assigned to the developmental dataset and used to construct the score system,and 542 (30%) patients were assigned to the validation dataset (Figure 1).The mean follow-up duration was 2.4 years.For the development dataset,the mortality and MI rates were 2.3% and 5.1% respectively,and the cumulative POCE rate was 11.8% (149/1,263).For the validation dataset,the mortality and MI rates were 3.5% and 5.9% respectively,and the cumulative POCE rate was 12.7% (69/542).

Baseline Characteristics

Baseline characteristics of included population based on the developmental dataset are presented in Table 1.Patients who suffered POCE 2 years after PCI were older,more likely to have a history of stroke,more likely to be acute myocardial infarction,and had worse renal function and cardiac function,that is,higher creatinine and lower left ventricular ejection fraction.Physiological index including average baseline QFR,average post-PCI QFR and lowest post-PCI QFR was significantly lower in the POCE group than in the non-POCE group.Risk scores reflecting the complexity of the disease,including the ACEF score,modified ACEF score,and residual SYNTAX score (rSS),were significantly higher in the POCE group than in the non-POCE group.

Table 1 Baseline characteristics of included population based on the developmental dataset.

Lesion characteristics and procedure information of included vessels in the entire cohort are presented in Table S1.2-year VOCE following PCI was associated with undesirable pre and post PCI QCA,more implanted stents and longer total stent length.The lesion type more likely to be tandem lesion,bifurcation lesion and distortion in the VOCE group.Coronary physiological index pre-PCI and post-PCI QFR was significantly lower in the VOCE group than in the non-VOCE group.

Baseline characteristics of included population based on the validation dataset are presented in Table 2.2-year POCE following PCI was associated with older age and larger proportion of previous PCI.Similarly,coronary physiological index was significantly lower in the POCE group than in the non-POCE group.Classic clinical and angiographic risk scores were significantly higher in the POCE group than in the non-POCE group.

Table 2 Baseline characteristics of included population based on the validation dataset.

Identification of the Significant Features

Univariate and multivariate Cox regression analyses were performed on each variable to define the predictors of 2-year POCE following the PCI procedure based on the developmental dataset (Table 3).Age,body mass index,creatinine,creatinine clearance,previous stroke,left ventricular ejection fraction and coronary physiological index QFR were related to 2-year POCE in univariate analysis.Multicollinearity diagnostics was conducted to assess the existence of any significant relationships.If multicollinearity was an issue,we have either removed the less significant variables,or applied variable transformation to account for the correlations.And then a stepwise regression analysis was employed to include appropriate variables into the multivariate model.After multivariate adjustment,the independent predictors of 2-year POCE were age,sex,creatinine,smoking history,previous stroke,left ventricular ejection fraction and coronary physiological index QFR(allPvalue ＜ 0.05).

Table 3 Predictors for 2-year POCE following the PCI procedure based on the developmental dataset.

Then we further used Random Forest Model and DeepSurv Model to identify the clinical and angiographic features of patients which have a significant impact on the outcome based on the developmental dataset.Feature component weightings for 2-year POCE prediction derived by Random Forest and DeepSurv algorithm were listed in Table S2.Radar chart was applied to visualize the weight of each characteristic variable (Figure 2).In the Random Forest Model,post-PCI QFR had the highest weight for 2-year POCE (0.0860),followed by pre-PCI lesion length (0.0572),pre-PCI QFR (0.0524),creatinine clearance (0.0512),age (0.0508) and LVEF(0.0454).The DeepSurv Model showed the similar result.Creatinine has the highest weight for 2-year POCE (0.0836),with post-PCI QFR (0.0776),creatinine clearance (0.0728),age (0.0714) and LVEF(0.0676) occupying the top five significance.In general,post-PCI QFR and factors in the ACEF,including age,creatinine,and ejection fraction,were further confirmed to be significantly associated with the risk of 2-year POCE after the PCI procedure.Therefore,it is reasonable to use these variables at the least to predict long-term adverse cardiovascular events in PCI patients.

Figure 2 Radar map for feature selection based on the developmental dataset. (A): Identification of significant feature for 2-year POCE prediction by the Random Forest Model;(B): identification of significant feature for 2-year POCE prediction by the DeepSurv Model.Feature component weightings for 2-year POCE prediction were shown in Table S2.BMI: body mass index (kg/m2);CHD:coronary heart disease;LVEF: left ventricular ejection fraction;MI: myocardial infarction;QFR: quantitative flow ratio.

Development of the Scoring Model

A Cox proportional hazard regression model was performed to develop the ACEF-QFR score based on the original ACEF score formula.It estimated the risk h(xi) of the event occurring for patient i with features xiusing a linear function: h(Xi)=∑ xiβi(β=the coefficient of xi).We assumed that features that significantly contributed to adverse outcomes in the cohort mainly included factors of ACEF and post-PCI QFR,and the ACEF-QFR score was calculated according to the following equation: h(X)=β1×QFR+β2× ACEF+βn× xn.Given that the scoring model should be constructed at the individual level instead of the vascular level in the present study,we transferred post-PCI QFR of 2,227 vessels to a dichotomic variable according to the cutoff value of 0.92 as previously described.When the post-PCI QFR ＞ 0.92,then h(X)=β2× ACEF+βn× xn=β2×[ACEF+(βn× xn)/β2].When the post-PCI QFR ≤0.92,then h(X)=β1+β2× ACEF+βn× xn=β2×[ACEF+(βn× xn)/β2+β1/β2].Therefore,the new scoring formula is obtained by adding the quotient of their regression coefficients β1/β2to the original ACEF formula if the post-PCI QFR ≤ 0.92.The final ACEF-QFR system was calculated as follows: ACEFQFR=Age (years)/EF (%)+1 (if creatinine ≥ 2.0 mg/dL)+β1/β2(if post-PCI QFR ≤ 0.92)

A developmental dataset was used to construct the model.Collinearity was checked with a tolerance and variance inflation factor.Little apparent correlation was found between these two variables.Multicollinearity diagnostics presented 0.988 tolerance 1.012 VIF for the ACEF score,and 0.988 tolerance 1.012 VIF for the post-PCI QFR.The results showed an absence of collinearity between the two variables in the model.The post-PCI QFR of 1263 patients in the development dataset was dichotomized by the cutoff value of 0.92.The ACEF and post-PCI QFR were entered into a Cox regression model.The β coefficient of the post-PCI QFR≤0.92 was 1.006,and the β coefficient of the ACEF was 0.863 (Table 4).The quotient of their regression coefficients β1/β2round up approximately equals 1.According to the linear function and deduction process described above,the ACEF-QFR scoring system is expressed as follows:

Table 4 The prediction model based on the Cox proportional hazard regression model for 2-year POCE with the two identified predictors based on the developmental dataset.

ACEF-QFR=Age (years)/EF (%)+1 (if creatinine ≥ 2.0 mg/dL)+1 (if post-PCI QFR ≤ 0.92).

Distributions of ACEF-QFR and Other Risk Scores

Figure 3 showed the distributions of ACEF-QFR score,post-PCI physiological index and other risk scores in all patients based on the developmental dataset,including average post-PCI QFR,lowest post-PCI QFR,ACEF score,modified ACEF score,and residual SYNTAX score.All risk scores showed abnormal distribution (allPvalues for normality ＜0.01).It can be seen from the distribution map that the ACEF-QFR score has two peaks,concentrated between 0.5-1.5 and 1.5-2.5.However,other classic clinical and angiographic risk scores are only concentrated within one range.It indicated that ACEFQFR identified patients with suboptimal post-PCI physiological index while retaining those who had adverse clinical conditions but optimal post-PCI physiological index.

Discrimination and Calibration of the ACEF-QFR

We first preliminarily evaluate the effectiveness of the ACEF-QFR scoring model in the developmental dataset,and then further explore its performance in the validation dataset.For the developmental dataset,ROC curves accompanying the AUCs and H-L test of the ACEF-QFR versus post-PCI physiological index and other risk scores for 2-year POCE were shown (Figure S1 and Table S3).The ACEF-QFR had significantly better discrimination ability for 2-year POCE than post-PCI physiological index and risk scores (0.652;95% CI: 0.603-0.702;P＜0.05 versus the average post-PCI QFR and lowest post-PCI QFR;P ≤0.001 versus the ACEF score,modified ACEF score and residual SYNTAX score,respectively).The ACEF-QFR also showed excellent calibration between the predicted and observed 2-year POCE based on the Hosmer-Lemeshow test (Hosmer-Lemeshow χ2=12.170;P=0.144) and calibration curve (Figure S2).A similar result was found in the validation dataset (Figure 4 and Table 5).Compared with post-PCI physiological index and other commonly used risk scores,the ACEF-QFR showed the highest AUC (0.651;95% CI:0.611-0.691;P ＜0.05 versus the average post-PCI QFR and residual SYNTAX score;P ＜0.001 versus the ACEF score and modified ACEF score).A relatively higher AUC was discovered when compared with the lowest post-PCI QFR although statistically significant was not met.The ACEF-QFR also displayed relatively good calibration and no lack of fitting.The Hosmer-Lemeshow test and calibration curve demonstrated that the new model has a good ability to generate predictions that are close to observations for 2-year POCE (Hosmer-Lemeshowχ2=7.070;P=0.529) (Figure 5).

Table 5 Discrimination and calibration of the ACEF-QFR score versus post-PCI physiological index and other commonly used risk scores for 2-year POCE based on the validation dataset.

Figure 4 ROC curve analysis showing the performance of the ACEF-QFR score versus post-PCI physiological index and other commonly used risk scores for 2-year POCE based on the validation dataset. Performance of the ACEF-QFR score,Average post-PCI QFR,Lowest post-PCI QFR,ACEF Score,Modified ACEF Score,and Residual SYNTAX Score for 2-year POCE were shown.Area under the ROC curves are shown in Table 5.ACEF:age,creatinine,and ejection fraction;ACEF-QFR: age,creatinine,ejection fraction,and quantitative flow ratio;PCI: percutaneous coronary intervention;ROC: receiver operator characteristic.

Figure 5 Calibration plot showing the agreement between the observed and predicted risks derived by the ACEF-QFR score based on the validation dataset. A calibration curve was added to each calibration plot based on a Cox model that fitted outcomes to a restricted cubic spline of the predictions.Calibration is optimal when the calibration curve is close to the diagonal line,reflected by a calibration intercept close to 0 and a calibration slope close to 1.ACEF: age,creatinine,and ejection fraction; QFR: quantitative flow ratio.

Reclassification Improvement and Clinical Utility of the ACEF-QFR

Table S4 presents the results of the net reclassification improvement (NRI) and integrated discrimination improvement (IDI) of the ACEF-QFR versus post-PCI physiological index and other risk scores for 2-year POCE based on the developmental dataset.Compared with the post-PCI physiological index and other commonly used risk scores,the ACEFQFR showed significant improvement in both net reclassification and integrated discrimination (allP＜0.05).For the validation dataset,it was found that the NRI of the ACEF-QFR score over the ACEF score alone was 0.984 (95% CI: 0.311-1.657;P ＜0.01)in comparison with the ACEF score (Table 6).Additionally,the IDI of the ACEF-QFR score over the ACEF alone was 6.68% (95% CI: 1.70%-11.66%;P ＜0.01).Similar results were found compared to the post-PCI physiological index and other commonly used risk scores (P ＜0.05 versus the average post-PCI QFR,lowest post-PCI QFR,modified ACEF score and residual SYNTAX score,respectively),indicating that the predictive accuracy of the ACEFQFR is much better than that of other risk scores by appropriately reclassifying high-risk and low-risk patients.

Table 6 Reclassification improvement of 2-year POCE by ACEF-QFR versus post-PCI physiological index and other risk scores based on the validation dataset.

Additionally,decision curve analysis (DCA) and clinical impact curve (CIC) analysis were performed to further evaluate the clinical utility of the ACEF-QFR score based on the validation dataset.According to the DCA,the net benefit for the ACEFQFR scoring model was larger over the range of average and lowest post-PCI QFR and the other commonly used risk scores (Figure 6).CIC analysis was performed to evaluate the clinical applicability of the ACEF-QFR score.The CIC revealed a good costbenefit ratio and good consistency between predicted and actual probabilities when applying the scoring model to predictions (Figure 7).

Prognostic Value of ACEF-QFR

The best cutoff value for the ACEF-QFR according to the Youden index was 1.21.Therefore,the entire population was categorized into two groups: a low-risk group (ACEF-QFR ≤ 1.21) and a high-risk group (ACEF-QFR ＞ 1.21).

Table S5 shows the multivariable-adjusted Cox regression analysis for 2-year clinical outcomes stratified by the ACEF-QFR cutoff value based on the developmental dataset.The covariables in the model included the clinical variables withP ＜0.05 in the univariate analysis and any other baseline variables judged to be of clinical relevance from previously published literature.The event rate for 2-year POCE were 19.0% and 7.6% in the high-risk group and low-risk group,respectively (adjusted HR=2.65;95% CI: 1.91-3.68;P ＜0.001).The event rate and hazard ratio of other secondary endpoints between the two groups are listed below,which showed similar results.For validation dataset,the clinical outcomes for 2-year POCE and its individual components between the two groups were showed in Table 7.Patients with suboptimal ACEF-QFR scores(ACEF-QFR ＞1.21) showed a significantly higher risk of 2-year POCE in the cohorts (17.4%vs.9.6%;adjusted HR=1.89;95% CI: 1.18-3.04;P ＜0.01).

Table 7 Multivariable-adjusted clinical outcomes at 2 years in cohorts according to ACEF-QFR based on the validation dataset.

Figure S3 presents the cumulative risk curves for 2-year major adverse cardiac events between the low-risk group and high-risk group based on the developmental dataset.The high-risk group showed significantly higher 2-year POCE compared with the low-risk group (log-rankP ＜0.0001,Figure S3A).The results of the Kaplan-Meier estimates for other secondary endpoints showed similar result between the two groups (all log-rankP ＜0.001,Figure S3BS3G).The analysis of prognostic value for ACEFQFR based on the validation dataset was shown in Figure 8.The significant relation of high-risk group with higher rate of 2-year POCE was found (logrankP ＜0.01,Figure 8A),while the result of the Kaplan-Meier estimates for other secondary endpoints were similar to that of 2-year POCE except for cardiac death and myocardial infarction probably due to limited sample size (Figure 8B-8G).These results indicated that the ACEF-QFR displayed good ability for 2-year adverse cardiac event prediction in the entire cohort.

Figure 8 Kaplan-Meier analysis showing cumulative events of primary and secondary endpoints stratified by the ACEF-QFR cutoff value based on the validation dataset. Cumulative risk curve and the number of patients at risk between the Low-risk group and High-risk group.(A): 2-year POCE;(B): all-cause death;(C): cardiac death;(D): myocardial infarction;(E): targeted vessel myocardial infarction;(F): ischemia-driven revascularization;and (G): ischemia-driven target vessel revascularization.Low-risk group: ACEFQFR ≤ 1.21 (n=324),high-risk group: ACEF-QFR ＞ 1.21 (n=218).

Patients over 65 years old or with underlying diseases,such as hypertension,diabetes,and previous MI,were considered high-risk patients,and for these patients,we performed a subgroup analysis to validate the predictive performance of the ACEF-QFR for 2-year POCE.The good-to-fair performance of the ACEF-QFR for 2-year POCE could also be maintained for different demographic characteristics(age,sex) and in patients with or without hypertension,diabetes,and previous myocardial infarction(MI) (Figure 9).Subgroup analyses are shown with hazard ratios and 95% CI for 2-year POCE.There were no significant interactions in any of the subgroups (interactionPvalue ＞ 0.05 for all comparisons).

Figure 9 Performance of ACEF-QFR for 2-year POCE in various subgroups. Receiver operating characteristic curve analysis demonstrates the prognostic values of the ACEF-QFR (A) for different age groups (＜ 65 years old or ≥ 65 years old),(B) for different sexes(male or female),(C) in patients with or without hypotension,(D) diabetes and (E) previous myocardial infarction.(F) Forest plots are shown with hazard ratios and 95% confidence intervals for 2-year POCE.There were no significant interactions in any of the subgroups (interaction P value ＞0.05 for all comparisons).ACEF-QFR: age,creatinine,ejection fraction,and quantitative flow ratio;MI:myocardial infarction;POCE: patient-oriented composite endpoint.

DISCUSSION

In this study,we used data from the PANDA III trial to develop a new risk model,the ACEF-QFR,to predict POCE at 2 years.The ACEF-QFR scoring system contains three clinical prognostic factors (age,serum creatinine,and ejection fraction) and one coronary lesion-based functional variable (QFR).The performance of the ACEF-QFR score was then verified in the validation cohort.We found that the newly developed ACEF-QFR score displayed good discrimination and calibration ability for 2-year POCE.Predictive accuracy and clinical utility were demonstrated to be much better than that of other commonly used risk scores.This model was also validated for its ability to predict long-term adverse cardiac events and showed similar effectiveness in different subgroups.

Several scoring systems have been shown to be effective in long-term risk prediction for patients undergoing cardiac operation,such as the ACEF score,modified ACEF score and residual SYNTAX score.The ACEF score was originally developed to predict longterm mortality in patients after cardiac surgery and has subsequently been introduced into numerous cardiology clinical conditions.[23-26]Based on the law of parsimony,the ACEF score consists of three variables (age,serum creatinine,and ejection fraction)but demonstrates superior predictive ability compared with other complicated score algorithms.[27-29]However,the predictive value of ACEF for the prognosis of CAD patients after PCI is limited.One potential reason is that despite the ACEF performing well in the prediction of long-term mortality,the poor prognosis of CAD patients undergoing PCI also includes myocardial infarction,ischemia-driven revascularization and stent thrombosis.High reversible ischemic burden was associated with adverse cardiovascular events,and reducing the coronary ischemic burden would undoubtedly improve longterm prognosis.[3]A key concern that limits the application of the ACEF score is that it only takes into account the patients’ clinical characteristics and does not encompass any coronary lesion-based parameter;thus,the predictive ability for CAD patients undergoing PCI might have been weakened.In a study by Zhang,et al.[30]validating the performance of 5 risk scores on 10,724 patients who achieved complete revascularization after PCI,ACEF could predict mortality significantly better than chance(AUC=0.68;95% CI: 0.62-0.74) but was not predictive for MACE (AUC=0.51;95% CI: 0.49-0.54).In the LEADERS trial,Wykrzykowska,et al.[26]found that the C-statistic of the ACEF score for predicting cardiac death was 0.727 and for myocardial infarction was 0.615 in all-comer PCI patients.In contrast,the ACEF score's ability to evaluate the risk of MACEs and TVR was relatively lower (0.577 and 0.527,respectively).This can be attributed to the fact that ACEF only contains clinical variables,and using it to predict events involving coronary variables is far from satisfactory.

In the view of Ranucci,the ACEF score could be considered a skeleton,and different variables can be added depending on specific clinical scenarios.[11]Palmerini,et al.[31]compared six scores in 2,094 patients with NSTEMI treated with PCI and demonstrated that risk scores incorporating clinical and angiographic variables had the highest predictive accuracy for a broad spectrum of ischemic end points.In the present study,both multivariate Cox regression analyses and the two machine learning models,Random Forest and DeepSurv model,confirmed that not only factors in ACEF presented a significant impact on 2-year POCE,but also post-PCI coronary physiological index.Therefore,to improve the precision of long-term cardiac event estimations,we established a scoring system integrating the ACEF and QFR through a Cox proportional hazard regression model based on the original ACEF score formula.

The quantitative flow ratio (QFR),a novel 3-D angiography-based tool for coronary physiology evaluation,can be used to derive FFR without the need for pressure wire or the induction of hyperemia.[5]The QFR is measured automatically using well-validated algorithms.Compared with the gold standard FFR,the QFR has been shown to have good diagnostic accuracy and reproducibility in identifying hemodynamically significant coronary stenosis.[6]A meta-analysis combining four clinical trials (FAVOR Pilot,FAVOR II China,FAVOR II E/J,and Wifi II) completed in China,Europe and Japan revealed the good correlation and overall diagnostic agreement of the QFR with FFR as the reference standard.[32]The QFR is an indicator for coronary physiology evaluation and can serve as a good predictor for vascular-related endpoints.In the HAWKEYE study,a prospective international multicenter clinical study,it was found that the post-PCI QFR in patients with complete revascularization was significantly associated with VOCE (vessel-oriented composite endpoint).[33]In the Favor III China,a large multicenter randomized controlled trial conducted in 26 hospitals,researchers revealed that using the QFR to guide surgery can reduce the MACE by 35%.[8]Our recent studies also showed that suboptimal functional outcomes (QFR ≤ 0.92) after PCI are significantly associated with poor prognosis and serve as a strong independent predictor of target vessel revascularization.[9,10]The significant importance of post-PCI QFR for 2-year POCE was further validated though machine learning models in the present study.It’s feature weight ranks first (0.0860)and second (0.0776) among all 46 features in the Random Forest model and DeepSurv Model respectively.As a physiological coronary test parameter,the QFR shows superior value in coronary stenosis evaluation and coronary ischemic burden assessment in contrast to other approaches focused only on anatomical characteristics,for example,coronary angiography.

The ACEF-QFR scoring system was considered a comprehensive assessment of patient outcomes and presented overall improved performance.Based on the test result in the validation dataset of the present study,the ACEF-QFR had an optimal discrimination ability for 2-year POCE following the PCI procedure (C-statistic=0.651;95% CI: 0.611-0.691),while the post-PCI physiological index presented a fine but suboptimal performance (C-statistic=0.616 for average post-PCI QFR and 0.623 for lowest post-PCI QFR).The ACEF score and the modified ACEF score showed a fair but relatively lower AUC (Cstatistics=0.579 and 0.589,respectively).

Our result was consistent with those from previous studies since the factors in the ACEF score contain the majority of the prognostic information,which is objectively measured.The ACEF-QFR was built on the ACEF score skeleton,and it retained the three factors representing essential prognostic dimensions,that is,age,renal function and cardiac function.The QFR parameter we added to the scoring system serves as a coronary lesion-based physiological index,making up for the bias of the original ACEF score.These features may be able to explain the appreciable ability of the ACEF-QFR for longterm cardiac event prediction.In our study,both multivariable Cox regression and Kaplan-Meier analysis further validated the predictive value of the scoring system (adjusted HR=1.89;95% CI:1.18-3.04;log-rankP＜ 0.01).In addition,the good performance of the ACEF-QFR score for 2-year POCE was consistent across different demographic characteristics (age,sex),and could be maintained in patients irrespective of various underlying disease (hypertension,diabetes,and previous myocardial infarction).

The ACEF-QFR scoring system not only takes into account patients’ clinical characteristics features but also encompasses coronary lesion-based physiological variables,enhancing the ability for longterm risk prediction in patients with PCI,so it is comprehensive.Moreover,each variable in the score is objectively measured without complicated manual operations or subjective visual inspection,so it is objective.In addition,it is also convenient.The formula still follows the law of parallelism,and the score can be easily calculated by mental arithmetic.Finally,it is generally applicable.Subgroup analysis showed that the predictive ability of the ACEFQFR score could also be maintained for patients with different clinical characteristics.All these features make it possible for the newly developed score to be widely used in clinical settings.

Limitations

The present study has some limitations that should be acknowledged.First,it was a retrospective analysis with a lack of accordant angiographic criteria and preset specific equipment,and unknown confounding factors and certain selection bias cannot be excluded.However,considering that the PANDA III trial was a multicenter population-wide study performed in a large and consecutive cohort of patients,we believe these results could reflect the real-world practice.Second,the relatively low event rate might give rise to a high probability of data overfitting.Interventional cardiologists in 46 cooperation centers involving in this trial have undergone standardized PCI training and got lots of practice.Patients who underwent PCI performed by experienced operators might have a better prognosis.This may explain the relatively low event rate in our study.Third,during the step of feature identification through DeepSurv Model,we examined 46 features of the patients and their lesions in the model.However,the process of feature weighting can be hard to interpret because the deep learning network is a highly nonlinear function,much like black boxes,which make it difficult to determine how the network processing massive amounts of data.We could only extract the feature component weightings obtained from the first layer of deep learning networks.Therefore,the DeepSurv Model was performed as a supplementary validation for the result from the Random Forest model which did not taking consider patients’ survival time information.Fourth,although the AUC of ACEF-QFR scoring model only reached 0.65 for 2-year POCE possibly due to the relatively low event rate based on the PANDA III dataset,there is still a significant improvement for ACEF-QFR when comparing with other commonly used risk scores.Moreover,although the substantial impact for long-term clinical outcome of post-PCI QFR and the factors in ACEF were confirmed through machine learning models,pre-PCI QFR and pre-PCI lesion length also occupies a significant weight of feature components.Therefore,the prognostic value of the scoring model combining pre-PCI lesion characteristics and pre-PCI physiological index should be further investigated in future studies.Finally,although the predictive ability of the ACEF score was confirmed in a separate cohort of patients,evaluation of the scoring model was limited to the PANDA III trial,thus the prognostic value of ACEFQFR drawn from this study may not be generalized to other populations at the moment unless further well-designed,large-scale,randomized trials are conducted.

Conclusion

The newly developed ACEF-QFR scoring system,which combines clinical and coronary lesion-based functional variables,enhanced the predictive ability for long-term risk prediction in patients undergoing PCI.This novel model may provide reliable prognostic information for cardiologists and patients undergoing PCI and help clinicians make informed decisions about follow-up treatment and further adjust treatment regimens for optimal patient outcomes.

DISCLOSURE

Acknowledgments

The PANDA III trial was sponsored by Sino Medical,Tianjin,China.The present study was supported by the Beijing Municipal Science and Technology Project [Z191100006619107 to B.X.] and Capital Health Development Research Project [2020-1-4032 to K.D.].

Conflict of Interest

The manuscript is approved by all authors for publication.All authors report no personal conflicts of interest regarding this manuscript.