AUTHORS
Sydney Manning, MS, John Spanias, PhD2, William Blumentals3, Claire Brulle-Wohlhueter4, Helgi Van de Velde, MD, PhD5, Silvy Saltzman6, Mony Morisse, MD, MSc7, Jacob Aptekar, MD, PhD8, Francesco Maura, MD9
AFFILIATIONS
1 Medidata Solutions, a Dassault Systèmes company, New York, New York, United States
2 Medidata Solutions, a Dassault Systèmes company, New York, New York, United States
3 Sanofi, Cambridge, Massachusetts, United States
4 Sanofi, Cambridge, Massachusetts, United States
5 Sanofi, Cambridge, Massachusetts, United States
6 Medidata Solutions, a Dassault Systèmes company, New York, New York, United States
7 Ichnos Glenmark Innovation, New York, New York, United States
8 Structure Therapeutics, San Francisco, California, United States
9 Memorial Sloan Kettering Cancer Center, New York, New York, United States
ABSTRACT
Introduction
Bispecific T-cell engagers (BiTEs) provide an alternative treatment to anti-BCMA Chimeric Antibody Receptor T-cell (CAR-T) for Relapsed/Refractory Multiple Myeloma (RRMM). CAR-T can pose significant risk of adverse events (AEs) like cytokine release syndrome (CRS, Afrough et al. 2024), where grade 2+ events require intervention (Lee at al. 2014). Longer-term, these treatments can be associated with Infections (Jurgens et al. 2024). Predictors of these AEs are well-described for CAR-T, but not for TCE or RRMM specifically. Additionally, the few predictive tools developed using machine learning (ML) to predict AEs in a clinical trial setting use only pre-treatment data, rather than incorporating day 1 (first day of study treatment) information which limits their applicability.
Objective
To classify RRMM patients by risk (high, medium, low) of AEs (CRS and Infections) prior to and on day-1 of treatment with CAR-T and BiTE therapies using ML methods.
Methods
Data was sourced from the Medidata Enterprise Data Store (MEDS), an anonymized repository of completed clinical trials. Trials used were initiated from 2015-2019, predominantly in North America. The sample included 799 patients aged 18+ (median=63). Outcomes predicted were CRS grade 2+ (Lee et al. 2014) between days 2 and 10 following first dose of anti-BCMA TCE/CAR-T therapy, and Infection (System Organ Class level MedDRA V266) grade 3+ (CTCAE) between days 31 and 100, after first study treatment. Model features were selected through literature review, domain expert input, data availability, and model impact. Patients who experienced the outcome event of CRS grade 2+ on day 1 (n=59) were excluded from the CRS cohort.
The final model selected to predict CRS was a two-part residual model; Model 1 predicted the outcome and Model 2 predicted the error (residual) between the Model 1 prediction and the true outcome. This allowed greater granularity in determining the importance of specific features because their impact on the residual prediction would be distinct. Model 1 predicted the probability of CRS using less novel features (CAR-T or TCE treatment, CRS grade 1 on day 1, change in: Temperature, diastolic blood pressure [BP], systolic BP). Model 2 predicted the Model 1 residuals using more novel features, e.g., concomitant medications and AEs on day 1, labs, additional vital signs, and other patient characteristics. The sum of probabilities from Model 1 and Model 2 provided final model performance metrics.
The final model selected to predict Infection was a tuned random forest classifier with hyperparameters selected via grid search. The model included 24 features; such as number of prior lines of treatment, treatment arm, other AEs during days 1-30, and change in labs/vital signs on days 1-30.
Models were evaluated using stratified 5-fold-cross-validation (by outcome and CAR-T/TCE treatment arm). Risk category boundaries were defined by dividing the probabilities of events into three equal proportions.
Results
In the CRS cohort (n=740), 22% of patients experienced CRS grade 2+ on days 2-10. Model performance was fair; cross-validated ROC-AUC was 0.68+-0.05 (test) and 0.87+-0.03 (train). The residual Model 2 (with novel features) yielded minimal information gain; less novel features (CAR-T or TCE treatment, CRS grade 1 on day 1, change in: Temperature, diastolic blood BP, systolic BP) were stronger predictors. SHAP plots showed TCE treatment as a positive predictor for CRS when compared to CAR-T. In a single training fold, 73% of high-risk versus 2% of low risk patients experienced CRS grade 2+.
In the Infections cohort (n=799, median follow-up 34.9 months) 61 patients (7.6%) experienced a grade 3+ Infection on days 31-90. Performance of the model was fair; cross-validated mean ROC-AUC was 0.68+-0.03 (test) and 0.91+-0.01 (train), indicating overfitting which was resistant to standard methods of adjustment. The top features were change in urate, platelets, hematocrit, sodium, hemoglobin, and treatment arm.
Discussion
These findings offer novel risk categorization to improve RRMM care management using dynamic information rather than solely baseline data. For CRS, established features were supported as strong predictors and provide an opportunity for clinical use. Higher risk of CRS in TCE-treated patients than CAR-T-treated patients warrants further validation.
DISCLOSURES
John Spanias, PhD: No relevant conflicts of interest to declare.