Studies presented at Digestive Disease Week included some focused on the quality and empathy of AI responses versus those from physicians related to patient questions about gastrointestinal cancers, Medpage Today reported May 6.
Here are five things to know about the studies:
1. The studies measured responses to questions about the stigmatizing terms used in media coverage of cirrhosis and a meal that may be appropriate to eat the evening before colonoscopy bowel prep.
2. The researchers used the social media platform Reddit to look for patient queries on the forum r/AskDocs, which includes approximately 675,000 patients and family members asking questions to verified healthcare providers. They selected 15 questions about pancreatic, liver, esophageal, stomach and colon cancers, with three questions per disease. These questions were then entered into ChatGPT 3.5 and were compared with the responses posted in the forum by actual physicians.
3. The responses were assessed for readability with the Flesch reading ease score and grade level with the Flesch Kincaid grade level, then presented to 12 board-certified medical and surgical oncologists to assess the quality and empathy of the replies on the five-point Likert scale.
3. ChatGPT outperformed physicians in the quality and empathy of these responses. On a five-point Likert scale, AI responses scored a 4.2 on both quality and empathy compared with scores of 3.0 on quality and 2.0 on empathy for doctor responses.
4. Muhamad Anees, MD, a research fellow at the Pittsburgh-based Allegheny Health Network, said that the physicians’ responses were judged to be more readable and written at a lower grade level.
5. “Effective education in communication regarding these cancer patients is really important because it helps improve outcomes,” Dr. Anees said. The study implied that “there is value in large language models in providing or enhancing GI cancer patient education.”
The post AI outscores GIs on empathy, quality responses: 5 things to know appeared first on Becker’s ASC.
