McMaster researcher leads new venture to guide responsible AI use in patient care

As artificial intelligence transforms the way physicians practise medicine globally, a McMaster researcher has launched a collaborative to advance the safe and effective use of AI in health care.

McMaster, the birthplace of evidence-based medicine, is well positioned to lead the development of standards for the use of AI, said Bright Huo, who founded the interdisciplinary group of clinicians, researchers and AI experts, called EBAiM.

Physicians are using AI for all kinds of tasks, from creating patient chart summaries to assisting with diagnoses. Despite its rapid uptake, the integration of AI in health care is fractured, inconsistent, and unregulated, said Huo, a general surgery resident and PhD candidate in Health Research Methodology at McMaster.

“We are truly at a pivotal moment in medicine. There are all these innovations happening in AI, but the work is siloed,” he said. “This is a critical moment. The pieces are all there. We just need to put them all together.”

EBAiM is working to develop rigorous, evidence-informed methodology at a time when more than 80 per cent of physicians report using AI professionally — a number that has more than doubled since a 2023 poll asking the same question of the same population.

“The whole point of evidence-based medicine was to improve clinical care, standardize how we approach research, and develop a common language and accepted level of rigour in how we conduct medical research,” said Huo. “There’s no reason why it should be any different for AI.”

Given the relative infancy of AI as a technology, there are still plenty of questions to be answered about how to use it to give patients the best possible care, said McMaster professor Gordon Guyatt, who coined the term “evidence-based medicine” in 1990.

“This is a new challenge, but every challenge is an opportunity,” said Guyatt, who is Huo’s PhD supervisor and methodological lead at EBAiM. “Our expertise in health research methods puts us in a position to take a leadership role in this area.”

Earlier this year, stakeholders from around the world gathered in Kraków, Poland at the McMaster International Review Conference for Internal Medicine (MIRCIM). This year’s event featured a parallel three-day conference called AiMed which focused on the use of AI in medicine. It was co-chaired by Dan Perri, an associate professor of medicine at McMaster who is part of the EBAiM team.

The future of AI in medicine needs to be driven by clinicians and health care researchers rather than for-profit corporations, and collaborations like EBAiM are an important part of that, Perri said.

“What McMaster has always done through its methodology leadership, journal publications, and guides that we’ve produced over the decades about how to interpret clinical studies needs to be recreated for AI,” said Perri.

“It doesn’t matter whether it’s a new ultrasound probe or whether it’s an AI diagnostic tool. The way we do the studies and the way we report meaningful outcomes shouldn’t really change.”

“We should expect evidence before we implement health AI tools.”

Huo and his team at EBAiM have already been hard at work providing evidence that can help clinicians use AI more effectively with the publication of the Chatbot Assessment Reporting Tool (CHART), which helps evaluate how different AI chatbots perform when summarizing clinical evidence and providing health advice.

Engineers, mathematicians and patients work on AI framework

They’re also working on a consensus framework that brings together stakeholders from all areas — computer scientists, computer engineers, mathematicians, statisticians, patient partners, ethicists, legal experts, and many more — to establish a standard for validating the use of generative AI systems for health purposes.

Huo said the hope is that the framework serves as a springboard for EBAiM to launch further projects that maintain a high level of rigour and quality when evaluating AI models for different health purposes – and not just in Canada.

“It’s a global initiative, this kind of endeavour,” said Huo.

“An important factor to consider in much of the research we do is that the integration of AI in health care is going to differ widely based on the setting in which you practise. How AI enters our health system in Canada will differ from how it’s implemented in the health system of another country in Europe and Asia and other continents, and because of that, we need people from all over the world with a diverse range of experiences and knowledge.

“We can all benefit from learning from other perspectives as we navigate this new era of medicine.”