Isaac Kohane
Professor
Make your next move.
Prepare your application
Review drafts before you send them.
Explore related searches
Articles (6)
Background Clinical narratives in electronic health records (EHRs) contain essential diagnostic, therapeutic, and temporal information that is often missing from structured fields, leaving manual chart review as the de facto standard for high-quality labels, but slow, costly, and variable, thereby constraining accurate cohort construction for clinical trials, large-scale epidemiologic studies, and the development of robust machine-learning models. Methods We developed CLINES, a modular agentic pipeline that extracts and structures clinical concepts: semantic chunking of long notes; extraction by reasoningcapable large language models; assignment of attributes (assertion/experiencer, numerical values with SI units); normalization to the Unified Medical Language System (UMLS); resolution of explicit and relative dates; and aggregation into an i2b2-style schema. Zero-shot evaluation was conducted on de-identified EHR: MIMIC-III notes, 4CE notes, and CORAL oncology reports (breast, pancreas). Comparators included rule/lexicon systems, transformer encoders, and single-prompt LLM baselines. Outcomes were F1 scores for entity extraction, assertion status, value&unit extraction, and date processing. Findings Across all datasets, CLINES led every baseline. F1 scores (entity / assertion / value&unit / date) were: MIMIC-III 0.69 / 0.93 / 0.90 (date not evaluated); 4CE 0.87 / 0.88 / 0.79 / 0.79; CORAL–Breast 0.81 / 0.84 / 0.77 / 0.73; CORAL–Pancreas 0.85 / 0.87 / 0.90 / 0.78. Gains over the strongest single-prompt LLM were +0.21–0.38 across tasks, and transformer encoders trailed by +0.28–0.68 F1 on entity extraction. Performance remained stable across note-length quantiles, while transformer baselines lost recall as notes lengthened. Interpretation CLINES translates narrative text from electronic health records into ontology-grounded, auditable, and schema-ready data, offering a practical route to scale chart-review-like extraction for cohort discovery and real-world evidence. CLINES is model agnostic–different open and close models can be substituted to achieve specific cost, performance, and privacy goals. Future work aims to quantify inter-annotator agreements and explore adaptive feedback and domain-specific fine-tuning.
Year:
2025
Background Objective assessment of left ventricular function remains a key prognosticator that is used to guide therapeutic decisions for patients with heart failure (HF). However, the left ventricular ejection fraction (LVEF) is dynamic, with worsening LVEF linked to increased morbidity and mortality. Identifying patients at risk of LVEF decline would improve prognostication and enable timely therapeutic intervention. Methods We developed a deep learning model to P redict changes in left ventric UL ar S ystolic function from E lectrocardiograms (ECG) of patients who have H eart F ailure (PULSE-HF). The model integrates 12-lead ECG waveforms with a patient’s history of prior LVEF measurements to calculate the likelihood that the LVEF will be less than 40% during the year after the ECG is obtained. The model is retrospectively developed and tested using data from one hospital and externally validated on retrospective cohorts from two different hospitals. Findings PULSE–HF demonstrates strong discriminatory ability with respect to forecasting whether the LVEF would be below 40% within the next year, achieving areas under the receiver operating characteristic curve (AUROC) of 87.5-91.4% across all three HF cohorts. Among patients with HF who have a baseline LVEF above 40%, PULSE–HF effectively identified those at risk of worsening LVEF with AUROCs of 81.6-86.3% across all three datasets. PULSE–HF’s discriminatory ability remained consistently high across a range of subgroups with different comorbidities and regardless of medical therapy. Assuming an underlying prevalence of LVEF worsening of 10% per year, PULSE-HF’s negative predictive values are over 97%, assuming an underlying sensitivity of 80%. Lastly, we demonstrate that a lead I version of PULSE–HF has a performance similar to the performance of the model that uses all 12 ECG leads. Interpretation PULSE–HF robustly predicts worsening LVEF in patients who have a prior diagnosis of HF. The method provides a platform for identifying patients who are at an increased risk of worsening systolic dysfunction. Funding This work was supported, in part, by a grant from Quanta Computers. Research in context Evidence before this study Heart failure (HF) remains a leading cause of morbidity and mortality across the globe. Left ventricular ejection fraction (LVEF) plays an important role in prognostication and treatment of HF. Moreover, LVEF is dynamic, with worsening LVEF being associated with increased mortality and poor outcomes compared to patients with HF with stable LVEF. Identifying patients with HF at risk of future decline in LVEF would improve prognostication and facilitate the timely delivery of appropriate medical therapy. However, models that use the electronic health record (EHR) to forecast declines in LVEF have demonstrated modest discriminatory ability. For instance, a model incorporating echocardiographic features, demographics, and comorbidities from the EHR had areas under the receiver operating characteristic curve (AUROC) between 54%–68% for predicting a 30% decline in LVEF within one year after HF diagnosis. Electrocardiograms (ECG) may contain information about LVEF, as deep learning models of ECGs have been used to estimate the contemporaneous LVEF. A prior method uses ECG to forecast LVEF below 40%, but this method was developed for a much shorter future time window of six months and specifically for patients with premature ventricular complexes. To date, the ECG has not been used to forecast future changes in LVEF among HF patients. Overall, no robust method exists for identifying patients with HF at high risk of future LVEF decline. Added value of this study We developed a deep learning model, called PULSE–HF, to P redict changes in left ventric UL ar S ystolic function from E CGs of patients who have H eart F ailure. The model leverages both the 12-lead ECG and history of prior LVEF measurements to identify patients at risk of having a reduced LVEF (≤ 40%) wit
Year:
2025
As large language models (LLMs) are increasingly deployed in high-stakes domains like healthcare, understanding how well their decision-making aligns with human preferences and values becomes crucial, especially when we recognize that there is no single gold standard for these preferences. This paper applies a systematic methodology for evaluating preference alignment in LLMs on categorical decision-making with medical triage as a domain-specific use case. It also measures how effectively an alignment procedure will change the alignment of a specific model. Key to this methodology is a novel simple measure, the Alignment Compliance Index (ACI), that quantifies how effectively a LLM can be aligned to a given preference function or gold standard. Since the ACI measures the effect rather than the process of alignment, it is applicable to alignment methods beyond the in-context learning used in this study. Using a dataset of simulated patient pairs, three frontier LLMs (GPT4o, Claude 3.5 Sonnet, and Gemini Advanced) were assessed on their ability to make triage decisions consistent with an expert clinician’s preferences. The models’ performance before and after alignment attempts was evaluated using various prompting strategies. The results reveal significant variability in alignment effectiveness across models and alignment approaches. Notably, models that performed well, as measured by ACI, pre-alignment sometimes degraded post-alignment, and small changes in the target preference function led to large shifts in model rankings. The implicit ethical principles, as understood by humans, underlying the LLMs’ decisions were also explored through targeted questioning. These findings highlight the complex, multifaceted nature of decision-making and the challenges of robustly aligning AI systems with human values. They also motivate the use of a practical set of methods and the ACI, in the near term, to understand the correspondence between the variety of human and LLM decision-making values in specific scenarios.
Year:
2024
A small number of cancer patients respond exceptionally well to therapies and survive significantly longer than patients with similar diagnoses. Profiling the germline genetic backgrounds of exceptional responder (ER) patients, with extreme survival times, can yield insights into the germline polymorphisms that influence response to therapy. As ERs showed a high incidence in autoimmune diseases, we hypothesized the differences in autoimmune disease risk could reflect the immune background of ERs and contribute to better cancer treatment responses. We analyzed the germline variants of 51 ERs using polygenic risk score (PRS) analysis. Compared to typical cancer patients, the ERs had significantly elevated PRSs for several autoimmune-related diseases: type 1 diabetes, hypothyroidism, and psoriasis. This indicates that an increased genetic predisposition towards these autoimmune diseases is more prevalent among the ERs. In contrast, ERs had significantly lower PRSs for developing inflammatory bowel disease. The left-skew of type 1 diabetes score was significant for exceptional responders. Variants on genes involved in the T1D PRS model associated with cancer drug response are more likely to co-occur with other variants among ERs. In conclusion, ERs exhibited different risks for autoimmune diseases compared to typical cancer patients, which suggests that changes in a patient’s immune set point or immune surveillance specificity could be a potential mechanistic link to their exceptional response. These findings expand upon previous research on immune checkpoint inhibitor-treated patients to include those who received chemotherapy or radiotherapy.
Year:
2024
Continuous glucose monitors (CGM) provide patients and clinicians with valuable insights about glycemic control that aid in diabetes management. The advent of large language models (LLMs), such as GPT-4, has enabled real-time text generation and summarization of medical data. Further, recent advancements have enabled the integration of data analysis features in chatbots, such that raw data can be uploaded and analyzed when prompted. Studying both the accuracy and suitability of LLM-derived data analysis performed on medical time series data, such as CGM data, is an important area of research. The objective of this study was to assess the strengths and limitations of using an LLM to analyze raw CGM data and produce summaries of 14 days of data for patients with type 1 diabetes. This study used simulated CGM data from 10 different cases. We first evaluated the ability of GPT-4 to compute quantitative metrics specific to diabetes found in an Ambulatory Glucose Profile (AGP). Then, using two independent clinician graders, we evaluated the accuracy, completeness, safety, and suitability of qualitative descriptions produced by GPT-4 across five different CGM analysis tasks. We demonstrated that GPT-4 performs well across measures of accuracy, completeness, and safety when producing summaries of CGM data across all tasks. These results highlight the capabilities of using an LLM to produce accurate and safe narrative summaries of medical time series data. We highlight several limitations of the work, including concerns related to how GPT-4 may misprioritize highlighting instances of hypoglycemia and hyperglycemia. Our work serves as a preliminary study on how generative language models can be integrated into diabetes care through CGM analysis, and more broadly, the potential to leverage LLMs for streamlined medical time series analysis.
Year:
2024
Genomics for rare disease diagnosis has advanced at a rapid pace due to our ability to perform "N-of-1" analyses on individual patients with ultra-rare diseases. The increasing sizes of ultra-rare disease cohorts internationally newly enables cohort-wide analyses for new discoveries, but well-calibrated statistical genetics approaches for jointly analyzing these patients are still under development. The Undiagnosed Diseases Network (UDN) brings multiple clinical, research and experimental centers under the same umbrella across the United States to facilitate and scale N-of-1 analyses. Here, we present the first joint analysis of whole genome sequencing data of UDN patients across the network. We introduce new, well-calibrated statistical methods for prioritizing disease genes with de novo recurrence and compound heterozygosity. We also detect pathways enriched with candidate and known diagnostic genes. Our computational analysis, coupled with a systematic clinical review, recapitulated known diagnoses and revealed new disease associations. We further release a software package, RaMeDiES, enabling automated cross-analysis of deidentified sequenced cohorts for new diagnostic and research discoveries. Gene-level findings and variant-level information across the cohort are available in a public-facing browser (https://dbmi-bgm.github.io/udn-browser/). These results show that N-of-1 efforts should be supplemented by a joint genomic analysis across cohorts.
Year:
2024
Collaborators (6)
Griffin Weber
Associate Professor of Biomedical Informatics
Harvard Medical School
Julian Martinez-Agosto
Associate Professor (with tenure)
David Geffen School of Medicine at UCLA
Collin Stultz
Massachusetts General Hospital
Sheng Yu
Assistant Professor of Statistics
Tsinghua University
Stanley Nelson
Professor
David Geffen School of Medicine at UCLA
Shamil Sunyaev
Professor
Harvard University
Make your next move.
Prepare your application
Review drafts before you send them.