board exam results 2017

Muncie fire captain receives harshest penalty possible for role in 'extended' exam cheating scandal

Fire department cheating scheme punishment in Muncie

SHARE

SHARE

TWEET

SHARE

EMAIL

What to watch next

Sheriff's deputy arrests police officer that fled from traffic stop

USA TODAY

Video shows the moment after multiple British soldiers collapsed due to extreme heat

CNN

Hear rooster interrupt CNN reporter outside courthouse

CNN

Rep. Brendan Boyle on reforming debt ceiling process: We don't need to take the economy hostage

CNBC

How these fitness instructors budgeted out of debt

The Wall Street Journal

Archer aims to make the sport accessible to LGBTQ+ community

TODAY

Conservative Koch network releases ads targeting Trump

MSNBC

California 19-year-old completes Ph.D.

NBC News

Andrew Tate Claims Huge Legal Win in Battle for His Freedom

Newsweek

Tory Lanez to be sentenced for Megan Thee Stallion shooting

CBS News

Russian missile strikes hit President Volodymr Zelensky’s hometown

CNN

Donald Trump's Arraignment: What's the legal process going forward?

CNN

Trump federal arraignment: What to expect inside and outside the courtroom

CBS News

Eye Opener: Former President Donald Trump to be arraigned in Miami

CBS News

Plane Crash Site Eerily Preserved In North Carolina Mountain

Newsweek

Miami equipped to handle up to 50K protestors as Trump heads to court

Scripps News

Click to expand

UP NEXT

A board that oversees discipline against Indiana firefighters issued the harshest penalty available under state law against a longtime Muncie firefighter and EMT accused of coordinating an elaborate cheating scheme.

Tuesday afternoon, the 12-member Indiana Board of Firefighting Personnel Standards and Education voted to permanently revoke all state firefighter certifications held by Troy Dulaney, a 19-year member of the Muncie Fire Department. Dulaney has 15 days to appeal the board’s decision once it publishes its order.

The unusual board action comes after a 13News investigation helped expose a systemic cheating scandal inside MFD that state leaders and regulators could not ignore. And at Tuesday’s public hearing, state investigators revealed the scheme had taken place for much longer than originally thought.

Pedro Caceres said he and other board members voted for Dulaney’s lifetime revocations to protect the public’s trust in firefighters and EMTs.

“He violated that trust and that is the foundation of public service,” said Caceres, who serves as a captain with the Wayne Township Fire Department. “It clearly sends a message that we are not here for ourselves but for those who we serve.”

This spring, the Indiana Department of Homeland Security and 13 Investigates received anonymous tips alleging that Dulaney provided test questions and answers to his colleagues before they took their EMS and firefighter tests. Dulaney, who was acting as a part-time instructor to help Muncie recruits prepare for the certification exams, was also accused of pressuring students to provide him with additional questions and answers after they took their tests, according to a state attorney who spent 45 minutes presenting evidence to board members at Tuesday’s public meeting.

“The investigation revealed a pattern of Dulaney instigating and encouraging “quid-pro-quo” conduct with students, in which Dulaney provided his students with test questions prior to their examinations, and in return pressured the students to report back to him with questions they remembered from the exam after they finished taking the exam, often telling these students they ‘owed him,’” IDHS assistant general counsel Hillary Egan told board members. “Interviews with past students and colleagues reveal that students felt pressured to engage in this quid-pro-quo exchange of test questions because Dulaney was in a position of authority at Muncie Fire Department, and they feared repercussions if they did not give Dulaney what he wanted.”

State Fire Marshal Steve Jones told 13News his department interviewed nearly 30 witnesses -- including Dulaney – during the state’s two-month investigation, which he said took “hundreds of hours” of staff time. Those interviews established a consistent pattern of misconduct, according to Egan. She testified that Dulaney accessed MFD recruits’ cell phone information to send group text messages to students with actual test questions and answers.

While state investigators say students did not request the test information from Dulaney, eight Muncie EMT recruits were nevertheless punished by IDHS for responding to the captain’s request for testing information and/or failing to report the cheating. The recruits have been placed on probationary status for two years, and they must re-take their EMT exam and pay a $100 fine.

For his role in the cheating scheme, Dulaney was notified that the state EMS director has revoked his EMTs certifications for a period of seven years. 

Tuesday’s hearing before the state firefighting board was to address Dulaney’s many firefighting certifications, which are subject to different oversight and disciplinary regulations than EMT certifications in Indiana. Dulaney is also named in a federal lawsuit filed by the National Registry of Emergency Medical Technicians (NREMT), the organization that develops and administers EMT certification exams, for violating a non-disclosure agreement that jeopardized the organization’s copyrighted material. 

Muncie’s mayor and fire chief insist the cheating scheme was isolated to Dulaney and that other city leaders had no knowledge of the misconduct. IDHS told 13News its investigation has not implicated any other individuals in coordinating the scandal.

But Tuesday afternoon, the state acknowledged for the first time that its investigation revealed the Muncie cheating has been going on for years.

“It became evident that this was not a 2023 isolated incident. This went back to 2018,” Egan told board members.

She told board members that investigators received six pages of test questions that had been photographed directly off a computer screen during a 2018 Firefighter Officer Strategy and Tactics exam, and that the state believes Dulaney “used his position and access as proctor” to log into a student’s testing portal to view the questions on his exam and to take photos with his cell phone camera. Those same questions were then provided to a student who was taking the FOST exam in 2022 when Dulaney was an adjunct instructor for that class.

Egan told board members the cheating appeared to get more blatant in recent years, with Dulaney providing questions and answers to entire classes of students, rather than to individuals. She referenced many pages of questions that Dulaney texted to students taking their EMT exams, and the state attorney also said an entire class of students taking a Rope Rescue Operations certification exam was provided with test questions prior to their April 2023 exam.

The information she shared next made several board members visibly shake their heads in disbelief.

“By the time this course ended, and testing took place in spring of 2023, Troy Dulaney had already removed himself because he had received a suspension letter from IDHS and knew that IDHS was aware of his actions. While this course was ongoing, however, the test questions were found on site at the training facility in students’ test packets. Before this course’ examination took place… IDHS personnel changed up the test and the questions, and this was really concerning: every single student flunked.,” Egan said. “This emphasizes the seriousness of Dulaney’s conduct and suggest that students were relying on memorizing the answers to the test questions they received prior to the exam and may not have been competent enough to pass the exam without knowing the questions and answers in advance.” 

She called Dulaney’s actions “really serious stuff” that could jeopardize the lives of both Muncie residents and other firefighters who rely on adequate training to do their jobs safely.

The information shared today with board members echoes the accounts of whistleblowers who spoke with 13 Investigates this spring, sharing accounts of cheating within the Muncie Fire Department that they saw first-hand.

Board members voted 10-1 to revoke Dulaney’s firefighting certifications. (The state fire marshal abstained from casting a vote due to his role in overseeing the state investigation.) While some members suggested revoking only some of Dulaney’s certifications, potentially allowing him to still legally serve as a firefighter in Indiana, the majority of the board ultimately decided on a full revocation due to the seriousness and scope of the misconduct. Rather than the seven-year revocation proposed by Egan, the board opted instead for an even harsher indefinite revocation.

The vote was supported by both the current and former presidents of the state firefighter’s union, who hold two seats on the 12-member board.

“This is very serious,” said longtime former union president Tom Hanify, who said the limited number of prior revocations issued by the board lacked the amount of evidence presented during Tuesday’s public meeting.

“I think all certifications for Mr. Dulaney should be revoked,” Caceres explained to his fellow board members, later telling 13News he has never seen such a blatant case of cheating. “Looking at what happened, I’m just in disbelief. This is really disheartening,” he said.

The State Fire Marshal said the severe punishment issued by the board is warranted, and he hopes it will send a strong message. 

“It just helps ensure the integrity of the fire service and EMS service. It’s important to do our training the right way,” Jones told 13 Investigates, adding that “we’ve got to hit reset” when it comes to emphasizing an expectation of ethical behavior among Indiana firefighters and EMTs.

Dulaney has not responded to multiple requests for comment from 13News. In a letter sent to NREMT, the fire captain denied any wrongdoing.

At Tuesday’s meeting, state investigators said Dulaney told them he did not recall sending text messages with test questions and answers to Muncie Fire Department recruits and that any possible text messages from the time period in question had already been deleted from his cell phone.

You can read the Indiana Department of Homeland Security’s full investigative summary and board recommendation here.

ChatGPT flunks self-assessment test for urologists

At a time of growing interest in the potential role of artificial intelligence (AI) technology in medicine and healthcare, a new study reported in Urology Practice finds that the groundbreaking ChatGPT chatbot performs poorly on a major specialty self-assessment tool.

ChatGPT achieved less than a 30% rate of correct answers on the AUA's widely used Self-Assessment Study Program for Urology (SASP). "ChatGPT not only has a low rate of correct answers regarding clinical questions in urologic practice, but also makes certain types of errors that pose a risk of spreading medical misinformation," comment Christopher M. Deibert, MD, MPH, and colleagues of University of Nebraska Medical Center.

Recent advances in large language models (LLMs) provide opportunities for adapting AI technology as a tool for mediating human interaction. "With adequate training and application, these AI systems can process complex information, analyze relationships between ideas, and generate coherent responses to an inquiry," note the authors.

ChatGPT (Chat Generative Pre-Trained Transformer) is an innovative LLM chatbot that has spurred interest in use in a wide range of settings—including health and medicine. In one recent study, ChatGPT scored at or near passing levels on all three steps of the United States Medical Licensing Exam (USMLE), without any special training or feedback on medical topics. Could this innovative AI-trained tool perform similarly well on a more advanced test of clinical knowledge in a surgical specialty?

To find out, Dr. Deibert and colleagues evaluated ChatGPT's performance on the AUA's Self-Assessment Study Program (SASP)—a 150-question practice examination addressing the core curriculum of medical knowledge in urology. The SASP is a valuable test of clinical knowledge for urologists in training and practicing specialists preparing for Board certification. The study excluded 15 questions containing visual information such as pictures or graphs.

Overall, ChatGPT gave correct answers to less than 30% of SASP questions: 28.2% of multiple-choice questions and 26.7% of open-ended questions. The chatbot provided "indeterminate" responses to several questions. On these questions, accuracy was decreased when the LLM model was asked to regenerate its answers.

For most open-ended questions, ChatGPT provided an explanation for the selected answer. The explanations provided by ChatGPT were longer than those provided by SASP, but "frequently redundant and cyclical in nature," according to the authors.

"Overall, ChatGPT often gave vague justifications with broad statements and rarely commented on specifics," Dr. Deibert and colleagues write. Even when given feedback, "ChatGPT continuously reiterated the original explanation despite it being inaccurate."

ChatGPT's poor accuracy on the SASP contrasts with its performance on the USMLE and other graduate-level exams. The authors suggest that while ChatGPT may do well on tests requiring recall of facts, it seems to fall short on questions pertaining to clinical medicine, which require "simultaneous weighing of multiple overlapping facts, situations and outcomes."

"Given that LLMs are limited by their human training, further research is needed to understand their limitations and capabilities across multiple disciplines before it is made available for general use," Dr. Deibert and colleagues conclude. "As is, utilization of ChatGPT in urology has a high likelihood of facilitating medical misinformation for the untrained user."

More information: Linda My Huynh et al, New Artificial Intelligence ChatGPT Performs Poorly on the 2022 Self-assessment Study Program for Urology, Urology Practice (2023). DOI: 10.1097/UPJ.0000000000000406

Citation: ChatGPT flunks self-assessment test for urologists (2023, June 6) retrieved 13 June 2023 from https://medicalxpress.com/news/2023-06-chatgpt-flunks-self-assessment-urologists.html

This document is subject to copyright. Apart from any fair dealing for the purpose of private study or research, no part may be reproduced without the written permission. The content is provided for information purposes only.