AUTHORS
Siyao He, MS, Ke Yuan2, Ting MD, PhD Niu, MD, PhD3, Yuhuan Zheng, PhD4, Li Zhang, PhD5
AFFILIATIONS
1 West China Hospital, Sichuan University, Chengdu, Sichuan, China
2 West China Hospital, Sichuan University, Chengdu, Sichuan, China
3 West China Hospital, Sichuan University, Chengdu, Sichuan, China
4 West China Hospital, Sichuan University, Chengdu, Sichuan, China
5 West China Hospital, Sichuan University, Chengdu, Sichuan, China
ABSTRACT
Background: Functional high-risk multiple myeloma (FHRMM) is usually identified only after early relapse or progression has occurred. Identifying these patients earlier could guide more intensive monitoring and timely treatment adjustments. The cross-population performance of prediction models incorporating genetic information remains uncertain. We developed pretreatment models for predicting progression or death within 18 months (FHR18) and compared a genetics-integrated model with a model based on conventional clinical variables. Methods: The MMRF CoMMpass cohort (n=620) was used for model development, and an independent real-world cohort from West China Hospital (n=451) was used for external validation. FHR18 was defined as disease progression or death within 18 months after initiation of first-line treatment. Ten machine-learning algorithms were evaluated using baseline clinical, laboratory, and genetic variables. Repeated nested cross-validation was used for internal validation. All preprocessing, model fitting, parameter tuning, and model selection were completed within the MMRF cohort. The selected model was then locked and applied to the West China Hospital cohort without retraining. As a sensitivity analysis, we developed a simplified model based only on routine clinical and laboratory variables, following the same analytical pipeline. Results: FHR18 occurred in 184 of 620 patients (29.7%) in the MMRF cohort and 148 of 451 patients (32.8%) in the West China Hospital cohort. Random forest was selected as the final genetics-integrated model. Its internal AUC was 0.655 (95% CI, 0.610–0.702), with a precision-recall AUC of 0.425, a Brier score of 0.197, and a calibration slope of 1.304. In the external cohort, the AUC was 0.575 (95% CI, 0.520–0.628), with a precision-recall AUC of 0.388, a Brier score of 0.220, and a calibration slope of 0.561.The simplified stochastic gradient boosting model had a similar internal AUC of 0.652 (95% CI, 0.601–0.696). In the external cohort, its AUC was 0.609 (95% CI, 0.551–0.661), with a precision-recall AUC of 0.426, a Brier score of 0.214, and a calibration slope of 0.747. The simplified model showed similar internal performance but better calibration externally. Conclusions: When transferred from the predominantly White MMRF cohort to an independent Chinese cohort, the genetics-integrated model showed reduced performance, indicating limited cross-population generalizability. We found that a simplified model incorporating routinely available measures such as hemoglobin, platelet count, creatinine, and LDH achieved similar internal discrimination and numerically better external performance than the genetics-integrated model. The simplified model uses routine tests already available in most hospitals—no genetic profiling needed. This makes it more practical for real-world use. Further validation and model refinement are underway.
DISCLOSURES
Siyao He, MS: No relevant conflicts of interest to declare.