Woman’s Death Blamed on Hospital’s AI System

As AI finds its way into hospital systems around the world, the case of one Brazilian woman makes it clear that the tech doesn’t have to diagnose patients to determine whether they live or die.

According to Brazilian news publication MG1, 32-year old Rebeca Cardoso Tenente Molina died after a state-run AI system for assigning hospital beds forced her to wait five days to transfer to an intensive care unit.

In interviews with MG1, Molina’s family members explained that the AI system delayed her access to the care she needed. Prior to her death, Molina was hospitalized in the small municipality of São João Nepomuceno after seeking medical treatment for gallstones. As her health rapidly deteriorated, Molina waited for an ICU bed in the state of Oliveira to open up, at a hospital about 186 miles away.

Though the family went as far as pursuing emergency legal action against the hospital system to get her transferred, the move was significantly delayed. The family now believes that five-day wait was fatal, MG1 reported.

According to Molina’s sister and family lawyer Sâmela Cardoso Tenente Furtado, the AI hospital-management system assigned the patient a much lower “score” than the one her condition actually reflected. This automatic scoring, completed by Brazil’s State Regulation Operations Center (Core-MG) using AI tools, is alleged to have played the deciding role on delaying transfer to an ICU bed.

“What we saw was that doctors lost the autonomy to decide if a patient is very seriously ill,” Furtado told MG1. “The one who has to accept whether a patient is seriously ill is no longer the doctor who is there experiencing that reality with the patient, it’s the Core.”

In her interview, Furtado describes a flawed and inflexible system which refused to budge, even as test data showed her sister’s condition taking a turn for the worst.

“She would have been a 10, and the system only accepted her as a 6.8,” Furtado continued. “So she couldn’t progress properly in the system because a patient at 8, a patient at 6.9 would jump ahead of her. And the system wouldn’t accept increasing her severity level within the system because of the tests that were constantly feeding it data.”

“My sister, other people, are not just numbers, they are not just protocols, they are not just a CPF [Brazilian tax ID number] thrown into the system,” Molina’s sister told MG1. “They have families, they had dreams, they had a whole life ahead of them.”

In an official statement shared after the AI-system’s launch on May 19th, Minas Gerais’ Deputy Secretary of Health, Poliana Cardoso Lopes said that “Core provides a bed map that is updated three times a day. With this, it will be possible to have much more control over the process and generate better data on the clinical condition and needs of each person waiting for a bed.”

Responding to Molina’s death, the state health department told MG1 that transfers are determined based on the availability of beds that fit a patient’s clinical needs, and added that Core-MG has not fundamentally altered the protocol for transferring patients to other facilities.

More on AI in healthcare: America’s Largest City Hospital System Ready to Start Replacing Radiologists With AI, Its CEO Says