Researchers explain how computer scientists and clinicians are trying to reduce fatal medical errors by building “ambient intelligence” into the spaces where patients reside.

As many as 400,000 Americans die each year because of medical errors, but many of these deaths could be prevented by using electronic sensors and artificial intelligence to help medical professionals monitor and treat vulnerable patients in ways that improve outcomes while respecting privacy.

“We have the ability to build technologies into the physical spaces where health care is delivered to help cut the rate of fatal errors that occur today due to the sheer volume of patients and the complexity of their care,” said Arnold Milstein, a professor of medicine and director of Stanford’s Clinical Excellence Research Center (CERC).

Milstein, along with computer science professor Fei-Fei Li and graduate student Albert Haque, are co-authors of a Nature paper that reviews the field of “ambient intelligence” in health care — an interdisciplinary effort to create such smart hospital rooms equipped with AI systems that can do a range of things to improve outcomes. For example, sensors and AI can immediately alert clinicians and patient visitors when they fail to sanitize their hands before entering a hospital room. AI tools can be built into smart homes where technology could unobtrusively monitor the frail elderly for behavioral clues of impending health crises. And they prompt in-home caregivers, remotely located clinicians and patients themselves to make timely, life-saving interventions.

Li, who is co-director of the Stanford Institute for Human-Centered Artificial Intelligence (HAI), said ambient technologies have many potential benefits, but they also raise legal and regulatory issues, as well as privacy concerns that must be identified and addressed in a public way to win the trust of patients and providers, as well as the various agencies and institutions that pay health care costs. “Technology to protect the health of medically fragile populations is inherently human-centered,” Li said. “Researchers must listen to all the stakeholders in order to create systems that supplement and complement the efforts of nurses, doctors and other caregivers, as well as patients themselves.”

Li and Milstein co-direct the 8-year-old Stanford Partnership in AI-Assisted Care (PAC), one of a growing number of centers, including those at Johns Hopkins University and the University of Toronto, where technologists and clinicians have teamed up to develop ambient intelligence technologies to help health care providers manage patient volumes so huge — roughly 24 million Americans required an overnight hospital stay in 2018 — that even the tiniest margin of error can cost many lives.

“We are in a foot race with the complexity of bedside care,” Milstein said. “By one recent count, clinicians in a hospital’s neonatal intensive care unit took 600 bedside actions, per patient, per day. Without technology assistance, perfect execution of this volume of complex actions is well beyond what is reasonable to expect of even the most conscientious clinical teams.”

The Fix: Invisible light guided by AI?

Haque, who compiled the 170 scientific papers cited in the Nature article, said the field is based largely on the convergence of two technological trends: the availability of infrared sensors that are inexpensive enough to build into high-risk care-giving environments, and the rise of machine learning systems as a way to use sensor input to train specialized AI applications in health care.

The infrared technologies are of two types. The first is active infrared, such as the invisible light beams used by TV remote controls. But instead of simply beaming invisible light in one direction, like a TV remote, new active infrared systems use AI to compute how long it takes the invisible rays to bounce back to the source, like a light-based form of radar that maps the 3D outlines of a person or object.

Such infrared depth sensors are already being used outside hospital rooms, for instance, to discern whether a person washed their hands before entering and, if not, issue an alert. In one Stanford experiment, a tablet computer hung near the door shows a solid green screen that transitions to red, or some other alert color that might be tested, should a hygiene failure occur. Researchers had considered using audible warnings until medical professionals advised otherwise. “Hospitals are already full of buzzes and beeps,” Milstein said. “Our human-centered design interviews with clinicians taught us that a visual cue would likely be more effective and less annoying.”

These alert systems are being tested to see if they can reduce the number of ICU patients who get nosocomial infections — potentially deadly illnesses contracted by patients due to failure of other people in the hospital to fully adhere to infection prevention protocols.

The second type of infrared technology are passive detectors, of the sort that allow night vision goggles to create thermal images from the infrared rays generated by body heat. In a hospital setting, a thermal sensor above an ICU bed would enable the governing AI to detect twitching or writhing beneath the sheets, and alert clinical team members to impending health crises without constantly going from room to room.

So far, the researchers have avoided using high-definition video sensors, such as those in smartphones, as capturing video imagery could unnecessarily intrude on the privacy of clinicians and patients. “The silhouette images provided by infrared sensors may provide data that is sufficiently accurate to train AI algorithms for many clinically important applications,” Haque said.

Constant monitoring by ambient intelligence systems in a home environment could also be used to detect clues of serious illness or potential accidents, and alert caregivers to make timely interventions. For instance, when frail seniors start moving more slowly or stop eating regularly, such behaviors can presage depression, a greater likelihood of a fall or the rapid onset of a dangerous health crisis. Researchers are developing activity recognition algorithms that can sift through infrared sensing data to detect changes in habitual behaviors, and help caregivers get a more holistic view of patient well-being.

Privacy is of particular concern in homes, assisted living settings and nursing homes, but “the preliminary results we’re getting from hospitals and daily living spaces confirm that ambient sensing technologies can provide the data we need to curb medical errors,” Milstein said. “Our Nature review tells the field that we’re on the right track.”