Daniel I. Kwon

Objective To understand the quality of informational Graves' disease (GD) videos on YouTube for treatment decision‐making quality and inclusion of American Thyroid Association (ATA) treatment guidelines. Study Design Cross‐sectional cohort. Setting Informational YouTube videos with subject matter “Graves' Disease treatment.” Method The top 50 videos based on our query were assessed using the DISCERN instrument. This validated algorithm discretely rates treatment‐related information from excellent (≥4.5) to very poor (<1.9). Videos were also screened for ATA guideline inclusion. Descriptive statistics were used for cohort characterization. Univariate and multivariate linear regressions characterized factors associated with DISCERN scores. Significance was set at P < .05. Results The videos featured 57,513.43 views (SD = 162,579.25), 1054.70 likes (SD = 2329.77), and 168.80 comments (SD = 292.97). Most were patient education (52%) or patient experience (24%). A minority (40%) were made by thyroid specialists (endocrinologists, endocrine surgeons, or otolaryngologists). Under half did not mention all 3 treatment modalities (44%), and 54% did not mention any ATA recommendations. Overall, videos displayed poor reliability (mean = 2.26, SD = 0.67), treatment information quality (mean = 2.29, SD = 0.75), and overall video quality (mean = 2.47, SD = 1.07). Physician videos were associated with lower likes, views, and comments ( P < .001) but higher DISCERN reliability ( P = .015) and overall score ( P = .019). Longer videos ( P = .015), patient accounts ( P = .013), and patient experience ( P = .002) were associated with lower scores. Conclusion The most available GD treatment content on YouTube varies significantly in the quality of medical information. This may contribute to suboptimal disease understanding, especially for patients highly engaged with online health information sources.

Objective With burgeoning popularity of artificial intelligence‐based chatbots, oropharyngeal cancer patients now have access to a novel source of medical information. Because chatbot information is not reviewed by experts, we sought to evaluate an artificial intelligence‐based chatbot's oropharyngeal cancer‐related information for accuracy. Methods Fifteen oropharyngeal cancer‐related questions were developed and input into ChatGPT version 3.5. Four physician‐graders independently assessed accuracy, comprehensiveness, and similarity to a physician response using 5‐point Likert scales. Responses graded lower than three were then critiqued by physician‐graders. Critiques were analyzed using inductive thematic analysis. Readability of responses was assessed using Flesch Reading Ease (FRE) and Flesch–Kincaid Reading Grade Level (FKRGL) scales. Results Average accuracy, comprehensiveness, and similarity to a physician response scores were 3.88 (SD = 0.99), 3.80 (SD = 1.14), and 3.67 (SD = 1.08), respectively. Posttreatment‐related questions were most accurate, comprehensive, and similar to a physician response, followed by treatment‐related, then diagnosis‐related questions. Posttreatment‐related questions scored significantly higher than diagnosis‐related questions in all three domains ( p < 0.01). Two themes of the physician critiques were identified: suboptimal education value and potential to misinform patients. The mean FRE and FKRGL scores both indicated greater than an 11th grade readability level—higher than the 6th grade level recommended for patients. Conclusion ChatGPT responses may not educate patients to an appropriate degree, could outright misinform them, and read at a more difficult grade level than is recommended for patient material. As oropharyngeal cancer patients represent a vulnerable population facing complex, life‐altering diagnoses, and treatments, they should be cautious when consuming chatbot‐generated medical information. Level of Evidence NA Laryngoscope , 134:2252–2257, 2024