Andrew Farah

McGill University
Country flagCanada

Research Interests

ApplyKite TwinLearns as you apply

Make your next move.

Prepare your application

Review drafts before you send them.

Explore related searches

Articles (9)

Year:

2025

Background/aims To compare the performance and cost-effectiveness of DeepSeek-R1 with OpenAI o1 in diagnosing and managing ophthalmology clinical cases. Methods In this cross-sectional study, a total of 300 clinical cases spanning 10 ophthalmology subspecialties were collected from StatPearls, each with a multiple-choice question on diagnosis or management. DeepSeek-R1 was accessed through its public chat interface, while OpenAI o1 was queried via its Application Programming Interface with a standardised temperature of 0.3. Both models were prompted using plan-and-solve+. Performance was calculated as the proportion of correct answers. McNemar’s test was employed to compare the two models’ performance on paired data. Intermodel agreement for correct diagnoses was evaluated via Cohen’s kappa. Token-based cost analyses were performed to estimate the comparative expenditures of running each model at scale, including input prompts and model-generated output. Results DeepSeek-R1 and OpenAI o1 achieved an identical overall performance of 82.0% (n=246/300; 95% CI: 77.3 to 85.9). Subspecialty-specific analysis revealed numerical variation in performance, though none of these comparisons reached statistical significance (p>0.05). Agreement in performance between the models was moderate overall (κ=0.503, p<0.001), with substantial agreement in refractive management/intervention (κ=0.698, p<0.001) and moderate agreement in retina/vitreous (κ=0.561, p<0.001) and ocular pathology/oncology (κ=0.495, p<0.01) cases. Cost analysis indicated an approximately 15-fold reduction in per-query, token-related expenses when using DeepSeek-R1 vs OpenAI o1 for the same workload. Conclusions DeepSeek-R1 shows strong diagnostic and management performance comparable to OpenAI o1 across ophthalmic subspecialties, while significantly reducing costs. These results support its use as a cost-effective, open-weight alternative to proprietary models.

Year:

2025

Year:

2025

Objectives: The integration of multimodal capabilities into GPT-4 represents a transformative leap for artificial intelligence in ophthalmology, yet its utility in scenarios requiring advanced reasoning remains underexplored. This study evaluates GPT-4’s multimodal performance on open-ended diagnostic and next-step reasoning tasks in complex ophthalmology cases, comparing it against human expertise. Methods: GPT-4 was assessed across three study arms: (1) text-based case details with figure descriptions, (2) cases with text and accompanying ophthalmic figures, and (3) cases with figures only (no figure descriptions). We compared GPT-4’s diagnostic and next-step accuracy across arms and benchmarked its performance against three board-certified ophthalmologists. Results: GPT-4 achieved 38.4% (95% CI [33.9%, 43.1%]) diagnostic accuracy and 57.8% (95% CI [52.8%, 62.2%]) next-step accuracy when prompted with figures without descriptions. Diagnostic accuracy declined significantly compared to text-only prompts (p = 0.007), though the next-step performance was similar (p = 0.140). Adding figure descriptions restored diagnostic accuracy (49.3%) to near parity with text-only prompts (p = 0.684). Using figures without descriptions, GPT-4’s diagnostic accuracy was comparable to two ophthalmologists (p = 0.30, p = 0.41) but fell short of the highest-performing ophthalmologist (p = 0.0004). For next-step accuracy, GPT-4 was similar to one ophthalmologist (p = 0.22) but underperformed relative to the other two (p = 0.0015, p = 0.0017). Conclusions: GPT-4’s diagnostic performance diminishes when relying solely on ophthalmic images without textual context, highlighting limitations in its current multimodal capabilities. Despite this, GPT-4 demonstrated comparable performance to at least one ophthalmologist on both diagnostic and next-step reasoning tasks, emphasizing its potential as an assistive tool. Future research should refine multimodal prompts and explore iterative or sequential prompting strategies to optimize AI-driven interpretation of complex ophthalmic datasets.

Year:

2025

Objectives: To develop a proof-of-concept machine learning (ML) neural network model to predict post-operative visual outcomes in children with congenital cataracts undergoing intraocular lens (IOL) implantation, thereby guiding the optimal timing for IOL insertion. Determining the ideal timing and predicting outcomes for IOL implantation in children remains clinically complex due to variability in eye development and measurement accuracy. Methods: Retrospective analysis using a publicly available dataset from 110 children diagnosed with congenital cataracts, who underwent IOL implantation at the Eye and ENT Hospital of Fudan University. A neural network model with a hidden layer of 10 nodes was developed in MATLAB 2024a using the scaled conjugate gradient algorithm. Input variables included demographic and clinical features; the target was achieving visual acuity greater than 20/40. Performance metrics were evaluated using cross-entropy loss, sensitivity, specificity, and accuracy. Results: Training completed after 14 epochs with the test set reaching the highest performance metrics: 88.2% accuracy, 88.9% sensitivity, and 87.5% specificity. ROC curve analysis showed AUC values of 0.942 (training), 0.920 (validation), 0.885 (test), and 0.917 (overall). Conclusions: The neural network effectively predicted post-operative visual outcomes, offering potential clinical utility in guiding IOL implantation decisions. Despite limitations in dataset diversity, this study lays the foundation for future development of personalized strategies in pediatric cataract care.

Year:

2025

Purpose To compare the performance and cost-effectiveness of DeepSeek-R1 with OpenAI o1 in diagnosing and managing ophthalmology clinical cases. Study Design Cross-sectional evaluation. Methods A total of 300 clinical cases spanning 10 different ophthalmology subspecialties were collected from StatPearls. Each case presented a multiple-choice question regarding the diagnosis or management of the clinical case. DeepSeek-R1 was accessed through its public chat-based interface, while OpenAI o1 was queried via an Application Program Interface (API) with a standardized temperature setting of 0.3. Both models were prompted using the Plan-and-Solve+ (PS+) prompt engineering method, instructing them to answer multiple choice questions for each case. Performance was calculated as the proportion of correctly answered multiple choice questions. McNemar’s test was employed to compare the two models’ performance on paired data. Inter-model agreement for correct diagnoses was evaluated via Cohen’s kappa. A token-based cost analysis was performed to estimate the comparative expenditures of running each model at scale, accounting for both input prompts and model-generated output. Results DeepSeek-R1 and OpenAI o1 both achieved an identical overall performance of 82.0% (n=246/300; 95% CI: 77.3-85.9). Subspeciality-specific analysis revealed numerical variation in performance, though none of these comparisons reached statistical significance (p>0.05). Agreement in performance between the models was moderate overall (κ=0.503, p<0.001), with substantial agreement in Refractive Management/Intervention (κ=0.698, p<0.001) and moderate agreement in Retina/Vitreous (κ=0.561, p<0.001) and Ocular Pathology/Oncology (κ=0.495, p<0.01) cases. Cost analysis indicated an approximately 15-fold reduction in per-query, token-related expenses when using DeepSeek-R1 compared with OpenAI o1 for the same workload. Conclusions DeepSeek-R1 demonstrates robust diagnostic reasoning and management decision-making capabilities, performing comparably to OpenAI o1 across a range of ophthalmic subspecialty cases, while also offering a substantial reduction in usage costs. These findings highlight the feasibility of utilizing open-weight, reinforcement learning-augmented LLMs as an accessible, cost-effective alternative to proprietary models.

Year:

2025

Background/aims This study assesses the proficiency of Generative Pre-trained Transformer (GPT)-4 in answering questions about complex clinical ophthalmology cases. Methods We tested GPT-4 on 422 Journal of the American Medical Association Ophthalmology Clinical Challenges, and prompted the model to determine the diagnosis (open-ended question) and identify the next-step (multiple-choice question). We generated responses using two zero-shot prompting strategies, including zero-shot plan-and-solve+ (PS+), to improve the reasoning of the model. We compared the best-performing model to human graders in a benchmarking effort. Results Using PS+ prompting, GPT-4 achieved mean accuracies of 48.0% (95% CI (43.1% to 52.9%)) and 63.0% (95% CI (58.2% to 67.6%)) in diagnosis and next step, respectively. Next-step accuracy did not significantly differ by subspecialty (p=0.44). However, diagnostic accuracy in pathology and tumours was significantly higher than in uveitis (p=0.027). When the diagnosis was accurate, 75.2% (95% CI (68.6% to 80.9%)) of the next steps were correct. Conversely, when the diagnosis was incorrect, 50.2% (95% CI (43.8% to 56.6%)) of the next steps were accurate. The next step was three times more likely to be accurate when the initial diagnosis was correct (p<0.001). No significant differences were observed in diagnostic accuracy and decision-making between board-certified ophthalmologists and GPT-4. Among trainees, senior residents outperformed GPT-4 in diagnostic accuracy (p≤0.001 and 0.049) and in accuracy of next step (p=0.002 and 0.020). Conclusion Improved prompting enhances GPT-4’s performance in complex clinical situations, although it does not surpass ophthalmology trainees in our context. Specialised large language models hold promise for future assistance in medical decision-making and diagnosis.

Year:

2024

Purpose: The aim of this study was to demonstrate the role of optical coherence tomography angiography in the management of dome-shaped maculopathy. Methods: This was a retrospective case review. Results: A 52-year-old woman was referred to the authors’ retinal service for potential bilateral choroidal neovascular membrane (CNVM) and blurry vision bilaterally. Initial spectacle-corrected visual acuity was 20/30-2 in the right eye and 20/30 + 2 in the left eye. Dome-shaped maculopathy was diagnosed on optical coherence tomography. In both eyes, optical coherence tomography B-scan passing through the fovea showed shallow irregular retinal pigment epithelium elevation suspicious of occult (type 1) CNVM. The outer retinal and choriocapillaris angiograms showed a zone of nonexudative CNVM in the right eye and exudative CNVM in the left eye. Given the persistent subretinal fluid with CNVM in the left eye, the authors elected to perform intravitreal injections of ranibizumab 0.5 mg on a treat and extend regimen. On the most recent follow-up, the best-corrected visual acuity improved to 20/20 in the left eye with no persisting subretinal fluid. Conclusion: The authors present a case where assessing disease progression and the development of CNVM and evaluating the efficiency of therapies were realized through the application of novel optical coherence tomography angiography technology. This diagnostic tool may be used to guide clinicians in their management of dome-shaped maculopathy, as demonstrated through the authors’ experience. Optical coherence tomography angiography can also make it possible to visualize nonexudative CNVM lesions that may be missed on traditional imaging assessments.

Year:

2023

Year:

2023

Collaborators (7)

Elif Bilgic

Assistant Professor and Education Scientist

McMaster University

CANADA
View details

Quang N. Ngo

Associate Professor

McMaster University

CANADA
View details

carlos cordoba

Université de Montréal

CANADA
View details

Pearse Keane

University College London

UNITED KINGDOM
View details

Renaud Duval

Université de Montréal

CANADA
View details

Rajeev H. Muni

-

CANADA
View details

Iqbal Jaffer

McMaster University

CANADA
View details