Michael To
Clinical Associate Professor
Make your next move.
Prepare your application
Review drafts before you send them.
Explore related searches
Articles (2)
Background State-of-the-art (SOTA) large language models (LLMs) are poised to revolutionize clinical medicine by transforming diagnostic, therapeutic, and interdisciplinary reasoning. Despite their promising capabilities, rigorous benchmarking of these models is essential to address concerns about their clinical proficiency and safety, particularly in high-risk environments. Methods This study implemented a multi-disciplinary, randomized, single-blind evaluation framework involving 27 experienced specialty clinicians with an average of 25.9 years of practice. The assessment covered 685 simulated and real clinical cases across 13 subspecialties, including both common and rare conditions. Evaluators rated LLM responses on medical strength (0–10 scale, where > 9.5 signified leading expert proficiency) and hallucination severity (0–5 scale for fabricated or misleading medical elements). Seven SOTA LLMs were tested, including top-ranked models from the ARENA leaderboard, with statistical analyses applied to adjust for confounders such as response length. Findings The evaluation revealed clinical plausibility in general-purpose LLMs, with Gemini 2.0 Flash leading raw scores and DeepSeek R1 excelling in adjusted analyses. Top models demonstrated proficiency comparable to a physician of 6 years post qualification experience (score ∼6.0), yet significant risks were noted. Instances of incompetence (scores ≤4) were detected across specialties, and 40 hallucination instances involving fabricated conditions, medications, and classification errors. These findings underscore the importance of implementing stringent safeguards to mitigate potential adverse outcomes in clinical applications. Interpretation While SOTA LLMs show substantial promise in enhancing clinical reasoning and decision-making, their unguarded application in medicine could present serious risks, such as misinformation and diagnostic errors. Human expert oversight remains crucial, particularly given reported incompetence and hallucination risks. Larger, multi-center studies are warranted to evaluate their real-world performance and track their evolution before broader clinical adoption.
Year:
2025
Osteogenesis imperfecta (OI) type V is typically characterized by radial head dislocation, calcification of interosseous membrane, and hyperplastic callus. It is caused by the c.-14C>T mutation in the 5′ UTR of IFITM5 gene, adding 5 amino acids (MALEP) to the N-terminal of IFITM5 protein. Previous studies have suggested a neomorphic function of the MALEP-IFITM5 protein. However, the underlying mechanisms remain unclear due to embryonic lethality in previous mouse models. Therefore, we developed an inducible mouse model (Ifitm5flox c.-14C>T) that could be induced by Cre expressed at different developmental stages to explore the pathogenic effects of the neomorphic MALEP-IFITM5. The mutant Ifitm5 allele could be regulated by the endogenous regulatory elements after Cre recombination, maintaining its spatiotemporal expression pattern and physiological level. Specifically, Prx1-Cre; Ifitm5flox c.-14C>T mutant mice were born with fractures in all limbs, showing impaired ossification and enhanced chondrogenesis associated with increased SOX9 abundance. Analyses of single-cell RNA sequencing data revealed arrested osteogenesis in Prx1-Cre; Ifitm5flox c.-14C>T mouse. A major population of cells expressing both osteogenic and chondrogenic signature genes was identified in the mutant mouse. Reduced expression of SP7 and SOST in the cortical regions of mutant mice confirmed delayed osteocyte maturation and compromised osteogenesis. Elevated bone marrow adipocytes were found in the adult mutant mice. Ectopic chondrogenesis and SOX9 expression were also observed in the perichondrium regions of Col1a1-Cre; Ifitm5flox c.-14C>T and Ocn-Cre; Ifitm5flox c.-14C>T mutant mice. The inducible Ifitm5flox c.-14C>T mouse model and integrated single-cell transcriptomic analyses elucidated that ectopic expression of SOX9 and disrupted homeostatic balance among osteogenesis, chondrogenesis, and adipogenesis may contribute to the pathogenesis caused by MALEP-IFITM5, helping to gain deeper insights into the molecular mechanisms of type V OI.
Year:
2025
Collaborators (1)
Nan Wu
Assistant Professor
Baylor College of Medicine
Make your next move.
Prepare your application
Review drafts before you send them.