2026 ASH®
Annual Meeting & Exposition December 12-15, 2026 ▪ New Orleans, Louisiana

Paper Details

Abstracts, authors, affiliations, and disclosures.

Sunday, December 13, 2026
06:00 p.m. - 08:00 p.m.
MCCNO - Hall F
Access, Representation, Population Health, Technology and Procedures, Imaging, Treatment Considerations, Supportive Care, Biological Therapies, Transplantation, Bispecific Antibody Therapy, (CAR)-T Cell Therapies, Workforce, Adverse Events, Artificial Intelligence (AI), Generative AI, Natural Language Processing, Biological Processes, Pathogenesis, Metabolism, Clinical Practice (Health Services and Quality), Clinical Guidelines, Health Economics, Patient Safety, Practice Models, Quality Improvement, Quality of Care, Value, Diseases, Immunology, Immunodeficiency, Lymphoid Malignancies, Lymphoid Leukemias, Lymphomas, Plasma Cell Disorders, Myeloid Malignancies, Acute Myeloid Malignancies, Chronic Myeloid Malignancies, CLL, Non-Hodgkin Lymphoma, B Cell Lymphoma, Indolent Lymphoma, Aggressive Lymphoma, Research, Translational Research, CML, Clinical Research, Clinical Trials, Epidemiology, Health Outcomes Research, Health Disparities Research, Patient-Reported Outcomes, Real-World Evidence, Registries, Survivorship, Study Population, Human, Adult, Elderly
PAPER

4392 Foundational abstraction of lymphoma clinical study-relevant features from medical notes using large language models

Emma Pranschke, MPH, Suchi Kamat, MBBS2, Philippe Armand, MD3, Matthew S Davids, MD, MMSc4, Reid W Merryman, MD5, Mimi Jiejing Zhang, MD, MS, MPH6, Kenneth Kehl, MD, MPH7, Brian Finn8, Mark Murakami, MD9
1 Dana Farber Cancer Institute, Boston, Massachusetts, United States
2 Dana Farber Cancer Institute, Boston, Massachusetts, United States
3 Mayo Clinic, Rochester, Minnesota, United States
4 Dana Farber Cancer Institute, Boston, Massachusetts, United States
5 Dana Farber Cancer Institute, Boston, Massachusetts, United States
6 Dana Farber Cancer Institute, Boston, Massachusetts, United States
7 Dana Farber Cancer Institute, Boston, Massachusetts, United States
8 Dana Farber Cancer Institute, Boston, Massachusetts, United States
9 Dana Farber Cancer Institute, Boston, Massachusetts, United States

Introduction: Identification of clinical study candidates relies on manual medical record review, which can limit accrual. The ability of large language models (LLMs) to efficiently summarize unstructured text (Van Veen Nat Med, 2024) has inspired use of artificial intelligence (AI) tools to streamline clinical workflows (Mazor JAMA Netw Open, 2025). We sought to evaluate the potential of LLMs to facilitate abstraction of patient features germane to study eligibility. Our primary objective was to assess the preliminary accuracy and efficiency of LLMs in deriving study-relevant features from clinical notes. Our secondary objective was to evaluate the ability of an unrelated LLM to identify and correct potentially erroneous abstractions by the first LLM (i.e., evaluating a ‘LLM as a judge’).

Methods: We randomly sampled 100 patients with a lymphoma clinic visit scheduled at Dana-Farber Cancer Institute (DFCI) between July and December 2026 who had at least one progress note by a DFCI lymphoma provider in the medical record after January 1, 2025. Two domain experts collaboratively abstracted the following data from the most recent note: (1) lymphoma diagnoses; (2) prior lymphoma therapies; and (3) total number of prior systemic therapies for each diagnosis. Manual abstraction (considered as ‘ground truth’) was blinded to LLM results.

LLM prompts were developed to extract the same features. Prompts incorporated role assignment, rules, examples, and instructions for output structure. Prompts were finalized with input from clinical domain experts and executed by a HIPAA-compliant instance of Claude Sonnet 4.5 (Anthropic, PBC). To assess the utility of ‘LLM as a judge’ (i.e., implementing an LLM to evaluate the performance of a separate LLM), a HIPAA-compliant instance of GPT 5.1 (OpenAI) was deployed twice, first to evaluate Claude Sonnet’s classifications, and then to correct them based on its evaluation.

LLM classifications were compared to ‘ground truth’ domain expert abstraction. The primary endpoint was accuracy of abstracted lymphoma diagnoses, most recent lymphoma therapy regimen and line (0, 1, 2, 3+), and prior exposure to selected therapies relevant to trial eligibility. Accuracy was measured as precision (positive predictive value), recall (sensitivity), and F1 scores (a summary metric comprising precision and recall). Precision, recall, and F1 were calculated for each classification level per category; weighted averages of these class-level metrics were computed based on the frequency of each classification level in the ‘ground truth’ dataset.

Abstraction efficiency was evaluated descriptively based on domain expert time vs. LLM end-to-end latency. A secondary analysis compared accuracy and efficiency of ‘judged/corrected’ versus ‘non-judged’ LLM output.

Results: Claude Sonnet achieved 86.0% precision and 91.0% recall for lymphoma diagnosis; F1 was 88.2%. It classified patients’ most recent treatment regimen with 90.8% precision and 91.0% recall, for an F1 of 90.7%. For line of therapy, Claude Sonnet achieved 87.5% precision, 86.0% recall, and 86.4% F1.

For therapies of special interest, Claude Sonnet’s performance varied. Abstraction of chimeric antigen receptor T-cells achieved 100% accuracy, precision, recall, and F1. Abstraction of BCL2 and BTK inhibitors each achieved 99.3% precision, 99.0% recall, and 99.1% F1. The occurrence of stem cell transplants in the cohort was low; however, Claude Sonnet correctly abstracted 1 of the 4 stem cell transplants documented in the ‘ground truth’ dataset.

Domain expert abstraction time vs. LLM end-to-end latency for the 100-patient cohort was 11.5 hours vs. 12 seconds. When acting as a judge, GPT 5.1 modestly improved accuracy of classification for diagnosis (precision, recall, and F1 were 94.2%, 96.0%, and 95.0%, respectively) but not other categories. End-to-end latency for judging and correcting responses was 14 seconds.

Conclusions: These preliminary findings support the potential of LLMs to abstract and classify complex study-relevant features from clinical notes with sufficient accuracy to justify evaluation in larger cohorts. Pending further prompt revisions to address errors revealed by this evaluation (e.g., identifying stem cell transplants) and more powerful LLMs, we seek to build on these efforts to develop study-centric applications to nominate candidates for further review by clinical research teams and enhance accrual efficiency.

Emma Pranschke, MPH: No relevant conflicts of interest to declare.