After passing another medical exam, ChatGPT is moving one step closer to ChatMD

ChatGPT is becoming a great medical test taker.

Its latest and most advanced version, ChatGPT-4, can already pass the US Medical Licensing Exam with flying colors. And now, it's moved one step closer to becoming a specialized physician. Well, sort of. 

On Tuesday, scientists announced that the newest chatbot software from OpenAI can handily pass a Canadian or US-style radiology board exam – scoring more than ten points above the 70% passing threshold. 

But there's a catch: Because ChatGPT is only designed to process language (so far), the AI's radiology exam didn't include any images. Feels like kind of a major oversight for a branch of medicine concerned with diagnosing based on X-rays, MRIs, and other body pictures.

ChatGPT has shown itself to be a formidable test taker — it's passed exams including the SAT, the bar exam, and even the challenging master sommelier tests. 

On the radiology exam, ChatGPT-4 delivered a passing score of 81%. The bot scored particularly well on higher-order thinking questions that require skills beyond just memory recall, like analysis, synthesis, and evaluation; ChatGPT-4 did well at describing imaging findings (85%), and applying concepts (90%). But it didn't do so great on some of the more straightforward questions on the test, and actually got 12 of those questions wrong that its predecessor, GPT 3.5, got right.

"We were initially surprised by ChatGPT's accurate and confident answers to some challenging radiology questions," study author Dr. Rajesh Bhayana, an abdominal radiologist at Toronto General Hospital, said in a press release. "But then equally surprised by some very illogical and inaccurate assertions."

At its core, ChatGPT is simply designed to predict the best next word to spit out – in other words, it's built to chat you up. That chattiness can easily be harnessed to pontificate on exams, producing a very confident (but sometimes wrong) test taker. 

Brown computer science professor Ellie Pavlick, a natural language processing expert, says this issue isn't unique to chatbots. She's always struggled to tell whether students really grasp the concepts she's taught based on their written exam answers. 

"If you give some well-constructed language, it seems like you know, but maybe you do, maybe you don't," Pavlick said during a recent ChatGPT roundtable at Brown University. "This is a nice indication of why we kind of want to attribute much more knowledge and awareness to ChatGPT than it actually really has, because there's just something about well-constructed language that can really mask [poor] understanding."

Doctors say that ChatGPT shouldn't be used to diagnose or treat patients, and should always have its medical accuracy checked by a person. But medical experts are also discovering how ChatGPT can be a nice tool for improving a doctor's communication with their patients. It is precisely because ChatGPT excels at banter that it is often regarded as more compassionate than hurried doctors.

Afghanistan: Doctors criticise delay in deciding date of medical expert exam

Doctors in Afghanistan have criticised the delay in setting the date of the medical expert examination, which is scheduled annually by the Taliban's Ministry of Public Health (MoPH), TOLO News reported. TOLO News is an Afghan news channel.

The doctors said they enrolled about one year ago, but the due date of the examination has yet to be determined. A doctor, Ahmad Zia, said: "The date of the exam should be clear to us because we need to determine our destiny. We want to serve the Afghan society."

"We call on the Taliban's Ministry of Public Health to take the examination as soon as possible, so the fate of the students is clear," said Nasrallah, another doctor. Some doctors also expressed concerns that delays in the examination will have a negative impact on the health sector.

"We face a shortage of academic figures, experts, and we will be forced to invite experts from abroad. Or our patients will be forced to go abroad for treatment," said Ahmad Shikib, a doctor, according to TOLO News. "If the students are not allowed to take expert examinations, everyone knows what will happen to our health system," said Najmul Sama Shafajo, head of the Afghanistan Society of Obstetricians and Gynecologists.

Despite consecutive contacts, TOLO News was unable to obtain comments from the Taliban's MoPH on this issue. Earlier, members of the Afghanistan Society of Obstetricians and Gynecologists (ASOG) criticized the lack of female doctors in medical expert exams held by the Ministry of Public Health. (ANI)

(This story has not been edited by Devdiscourse staff and is auto-generated from a syndicated feed.)

After passing another medical exam, ChatGPT is moving one step closer to ChatMD

Getty Images

ChatGPT is becoming a great medical test taker.

Its latest and most advanced version, ChatGPT-4, can already pass the US Medical Licensing Exam with flying colors. And now, it's moved one step closer to becoming a specialized physician. Well, sort of.

On Tuesday, scientists announced that the newest chatbot software from OpenAI can handily pass a Canadian or US-style radiology board exam – scoring more than ten points above the 70% passing threshold.

But there's a catch: Because ChatGPT is only designed to process language (so far), the AI's radiology exam didn't include any images. Feels like kind of a major oversight for a branch of medicine concerned with diagnosing based on X-rays, MRIs, and other body pictures.

SCIENCE PHOTO LIBRARY/Getty Images

ChatGPT has shown itself to be a formidable test taker — it's passed exams including the SAT, the bar exam, and even the challenging master sommelier tests.

On the radiology exam, ChatGPT-4 delivered a passing score of 81%. The bot scored particularly well on higher-order thinking questions that require skills beyond just memory recall, like analysis, synthesis, and evaluation; ChatGPT-4 did well at describing imaging findings (85%), and applying concepts (90%). But it didn't do so great on some of the more straightforward questions on the test, and actually got 12 of those questions wrong that its predecessor, GPT 3.5, got right.

"We were initially surprised by ChatGPT's accurate and confident answers to some challenging radiology questions," study author Dr. Rajesh Bhayana, an abdominal radiologist at Toronto General Hospital, said in a press release. "But then equally surprised by some very illogical and inaccurate assertions."

Story continues

At its core, ChatGPT is simply designed to predict the best next word to spit out – in other words, it's built to chat you up. That chattiness can easily be harnessed to pontificate on exams, producing a very confident (but sometimes wrong) test taker.

Brown computer science professor Ellie Pavlick, a natural language processing expert, says this issue isn't unique to chatbots. She's always struggled to tell whether students really grasp the concepts she's taught based on their written exam answers.

"If you give some well-constructed language, it seems like you know, but maybe you do, maybe you don't," Pavlick said during a recent ChatGPT roundtable at Brown University. "This is a nice indication of why we kind of want to attribute much more knowledge and awareness to ChatGPT than it actually really has, because there's just something about well-constructed language that can really mask [poor] understanding."

Doctors say that ChatGPT shouldn't be used to diagnose or treat patients, and should always have its medical accuracy checked by a person. But medical experts are also discovering how ChatGPT can be a nice tool for improving a doctor's communication with their patients. It is precisely because ChatGPT excels at banter that it is often regarded as more compassionate than hurried doctors.

Read the original article on Insider