AUTHORS
Dang Nhu Hai Pham, PhD, Aidan Boyne2, Barbara Lam Jr, MD3, Jennifer La, PhD4, Siameck Ayandeh5, Mrinal Ranjan6, Juan Luis Bueno7, Joyce Tiong, MD8, Emma Kitchel9, Melkon Harutunian10, Gongliang Zhang11, Nathaniel Fillmore12, Omid Jafari, Senior Software Engineering and Programming Associate13, Ang Li, MD, MS14
AFFILIATIONS
1 Baylor College of Medicine, Houston, Texas, United States
2 Baylor College of Medicine, Houston, Texas, United States
3 University of Washington, Fred Hutch Cancer Center, Seattle, Washington, United States
4 VA Boston Healthcare System, Boston, Massachusetts, United States
5 University of Utah, Salt Lake City, Utah, United States
6 Baylor College of Medicine, Houston, Texas, United States
7 Baylor College of Medicine, Houston, Texas, United States
8 Baylor College of Medicine, Houston, Texas, United States
9 Baylor College of Medicine, Houston, Texas, United States
10 Baylor College of Medicine, Houston, Texas, United States
11 MedStar Health, Columbia, Maryland, United States
12 VA Boston Healthcare System, Boston, Massachusetts, United States
13 Baylor College of Medicine, Houston, Texas, United States
14 Baylor College of Medicine, Houston, Texas, United States
ABSTRACT
Background: Accurate and scalable identification of venous thromboembolism (VTE) outcomes remains a major challenge in thrombosis research. Clinical trials, registries, and observational studies frequently rely on labor-intensive manual endpoint adjudication or administrative billing codes, both of which have important limitations for identifying recurrent VTE. Existing NLP models classify individual clinical notes but cannot reliably distinguish repeated documentation of the same thrombosis from new events, limiting their utility for longitudinal outcome adjudication. The application of large language models (LLMs) across longitudinal documentation may overcome these limitations. We developed and externally validated an LLM-based pipeline (VTE-LLM) for longitudinal VTE extraction from retrospective electronic health records in patients with cancer. We further created a cost-efficient hybrid approach and tested the model in a prospective cohort study.
Methods: Clinical notes from cancer patients at a county safety-net health system were preprocessed using a validated rules-based algorithm to retain VTE-relevant text, then chronologically concatenated per patient. A prompt-engineered LLM (Claude Sonnet 4.5, extended thinking, zero-shot) was iteratively refined using a training cohort of 200 previously annotated patients to identify any VTE event, first incident VTE after cancer diagnosis, and recurrent VTE, each referenced against a gold-standard adjudicated by a senior hematologist following prespecified definitions of incident, recurrent, and historical VTE. Performance was assessed in three independent cohorts: internal validation (400 patients, 200 VTE-positive), external validation (400 patients from a Veteran’s Affairs hospital with advanced solid tumors, 14.3% VTE incidence), and prospective validation (215 county hospital patients with newly diagnosed solid tumors enrolled at chemotherapy initiation, 2021-2025, with outcomes prospectively collected at 3/6/12 months, 7.4% VTE incidence). We compared VTE-LLM against initial medical-trainee annotation, research coordinator annotation, a previously validated note-level classifier (VTE-BERT), and ICD-10-based algorithms. We also tested a hybrid workflow using VTE-BERT (PMID: 40754035) prescreening to triage patients before VTE-LLM adjudication, evaluating both diagnostic performance and computational cost.
Results: VTE-LLM accurately reconstructed longitudinal thrombotic histories across all validation cohorts. For incident VTE detection, precision/recall were 96%/95% internally and 98%/98% externally. For recurrent VTE, VTE-LLM achieved 90%/90% internally and 87%/93% externally, substantially outperforming VTE-BERT (28%/100%) and ICD-10 algorithms (52%/22%), both of which frequently misclassified repeated documentation of the same thrombus as new events. VTE-LLM also outperformed initial medical-trainee annotation on overall event detection (95%/94% vs. 86%/89% precision/recall). In prospective validation, VTE-LLM achieved 100% precision and recall (16/16) against adjudicated one-year outcomes, similar to the performance of trained research coordinators. Additionally, VTE-LLM identified 15 confirmed historical VTE events versus only 8 self-reported by patients at enrollment. A hybrid VTE-BERT prescreening followed by VTE-LLM adjudication workflow preserved high accuracy (95% precision, 92% recall) while triaging 45% of patients away from VTE-LLM entirely, reducing per-cohort computational cost by roughly 50% (from $48.00 to $26.40 per 400 patients).
Conclusions: VTE-LLM accurately identifies incident and recurrent VTE across longitudinal clinical documentation, addressing a persistent gap in automated VTE phenotyping that note-level NLP and billing-code algorithms cannot resolve. Consistent performance across internal, external, and prospective cohorts supports its readiness for real-world deployment. A hybrid workflow combining inexpensive VTE-BERT prescreening with targeted VTE-LLM adjudication offers a practical, cost-efficient framework for scalable endpoint ascertainment in thrombosis registries, real-world evidence studies, and prospective hematology clinical trials.
DISCLOSURES
Ang Li, MD, MS: No relevant conflicts of interest to declare.