2026 ASH®
Annual Meeting & Exposition December 12-15, 2026 ▪ New Orleans, Louisiana

Paper Details

Abstracts, authors, affiliations, and disclosures.

Monday, December 14, 2026
06:00 p.m. - 08:00 p.m.
MCCNO - Hall F
Technology and Procedures, Molecular Testing, Pathology, Treatment Considerations, Combination Therapy, Drug Development, Supportive Care, Biological Therapies, Monoclonal Antibody Therapy, Non-Biological Therapies, Anticoagulant Drugs, Pharmacology, Women’s Health, Adverse Events, Artificial Intelligence (AI), Generative AI, Machine Learning, Natural Language Processing, Biological Processes, Apoptosis, Genomics, Hematopoiesis, Molecular Biology, Clinical Practice (Health Services and Quality), Diseases, Bleeding and Clotting, Bleeding Disorders, Platelet Disorders, Thromboembolism, Thrombotic Disorders, VWD, Genetic Disorders, Immunology, Autoimmune Disorders, Lymphoid Malignancies, Myeloid Malignancies, Acute Myeloid Malignancies, Chronic Myeloid Malignancies, Research, AML, Translational Research, MDS, MPN, CMML, Clinical Research, Clinical Trials, Epidemiology, Health Outcomes Research, Patient-Reported Outcomes, Real-World Evidence, Survivorship, Study Population, Human, Adult, Elderly, Young Adult
PAPER

5535 Multicenter validation of a large language model framework for automated case ascertainment and longitudinal phenotyping in polycythemia vera and essential thrombocythemia

Muhammad Ali Khan, MBBS, Ahmed Ali Khan, MD2, Zuhair Abrar, MD3, Usama Cheema, MD4, Zahra Rubab Khakwani, MD5, Talha Tahir, MD6, Maryam Ilyas, MD7, Umar Afzal, MD8, Shan Khan9, Kanishka Uttam Chandani, M.B.B.S.10, Irbaz Riaz, MD, MS, MBI, PhD11, Jeanne Palmer, MD12
1 Mayo Clinic, Phoenix, Arizona, United States
2 Mayo Clinic, Phoenix, Arizona, United States
3 Mayo Clinic, Phoenix, Arizona, United States
4 Mayo Clinic, Phoenix, Arizona, United States
5 Mayo Clinic, Phoenix, Arizona, United States
6 Mayo Clinic, Phoenix, Arizona, United States
7 Mayo Clinic, Phoenix, Arizona, United States
8 Mayo Clinic, Phoenix, Arizona, United States
9 Mayo Clinic, Phoenix, Arizona, United States
10 Mayo Clinic, Phoenix, Arizona, United States
11 Mayo Clinic, Phoenix, Arizona, United States
12 Mayo Clinic, Phoenix, Arizona, United States

Background: Polycythemia vera (PV) and essential thrombocythemia (ET) registries commonly rely on International Classification of Diseases (ICD) codes for case identification and manual chart abstraction, often capturing only baseline information. ICD codes are designed for billing and may misclassify disease, while manual abstraction is slow, resource-intensive, and error-prone. These limitations hinder outcomes research in these chronic diseases, where progression to myelofibrosis or acute myeloid leukemia, thrombosis, and survival depend on longitudinal disease and treatment trajectories. We developed a large language model (LLM)-based framework for automated case ascertainment and longitudinal phenotyping in PV and ET.

Methods: Patients with ICD-9/10 codes for PV (238.4/D45) or ET (238.71/D47.3) across all Mayo Clinic sites through July 24, 2026, were identified. The framework used: (1) rule-based retrieval of relevant clinical documents and structured data; (2) GPT-4o, deployed in a HIPAA-compliant environment with zero-shot deterministic prompting, to extract 46 variables spanning symptoms, complications, treatments, marrow morphology, spleen measurements, mutations, and cytogenetics; (3) rule-based reconciliation of conflicting information; and (4) rule-based diagnostic classification according to 2017 WHO criteria or a documented hematologist diagnosis. For validation, 1,000 ICD-flagged patients were randomly sampled, stratified by workflow classification and coded disease: 500 workflow-confirmed and 500 unconfirmed, each including 250 PV-coded and 250 ET-coded patients. Trained clinicians, blinded to workflow output, independently reviewed each case; discordant or indeterminate cases underwent adjudication. Case-ascertainment sensitivity and specificity were calculated against the reconciled reference standard. Variable extraction was evaluated by exact agreement for binary and categorical variables and 10% relative tolerance for continuous variables.

Results: Among 38,250 patients with a PV or ET ICD code, the workflow confirmed 13,364 (35%) as true PV (n=7,041) or ET (n=6,323). In the 1,000-patient validation sample, initial workflow and clinician classifications were concordant in 807 cases (80.7%); 193 (19.3%) required adjudication. Case ascertainment achieved 96.5% sensitivity and 85.4% specificity. Applying stratum-specific reference rates to the full ICD-flagged cohort, the estimated positive predictive value of ICD codes alone was 31%; thus, 69% of coded patients did not have PV or ET. Among 172 patients undergoing extraction validation (388 documents), 92.3% of abstractions were concordant and 7.7% required adjudication. Extraction accuracy was 92.5% for binary variables (F1=0.90) and 90.5% for continuous variables within 10% relative tolerance. Across confirmed cases, the framework generated a median of 19 quarterly assessments per patient (IQR, 6–39) over a median follow-up of 4.89 years (IQR, 1.43–10.3). Progression to myelofibrosis occurred in 1,812 patients (14%; PV, 15%; ET, 12%) at a median of 8.51 years from diagnosis (IQR, 4.14–14.8), and 3,508 patients died (26%; PV, 27%; ET, 25%).

Conclusions: ICD codes alone substantially misclassified PV and ET, with an estimated positive predictive value of only 31%, limiting their reliability for registries and real-world evidence studies. Our multicenter LLM-based framework achieved high diagnostic sensitivity and specificity while generating longitudinal, trajectory-level phenotypes for more than 13,000 confirmed patients. These findings support automated case ascertainment and phenotyping as a scalable approach to building longitudinal registries in MPNs. Further studies should compare automated and manual curation with respect to time, cost, and longitudinal data completeness and assess their effects on downstream clinical inferences.

Muhammad Ali Khan, MBBS: No relevant conflicts of interest to declare.