Ophthalmology Questions Can be Answered by AI Model

According to a study published in JAMA Network Open, large language models (LLMs) like ChatGPT can reply to patient-written ophthalmology inquiries and usually generate suitable responses.

Stanford University’s Isaac A. Bernstein and colleagues compared the quality of ophthalmology advice generated by an LLM chatbot to ophthalmologist-written guidance. The researchers examined deidentified data from an online medical forum where patients’ inquiries were answered by ophthalmologists. A masked panel of eight board-certified ophthalmologists were asked to distinguish between ChatGPT chatbot responses and ophthalmologist responses.

Two hundred user questions and replies were evaluated. The researchers discovered that the average accuracy for distinguishing between artificial intelligence (AI) and human responses was 61.3 percent. Of 800 chatbot-written answers evaluated, 21.0 and 64.6 percent were classified as human-written and AI-written, respectively. When compared to human responses, chatbot responses were more frequently rated as probably or certainly written by AI. The chance of chatbot responses including incorrect or improper material, as well as the likelihood of injury, was equivalent to human responses.

“We intend for this study to catalyze more extensive and nuanced dialogue and joint efforts surrounding the use of LLMs in ophthalmology among various health care stakeholders, including patients, clinicians, researchers, and policy makers,” the authors write. “The primary goal is to prudently leverage these early research findings to shape the responsible implementation of LLMs in the field of ophthalmology.”

Source Link