‘Alexa, pull those lab results’: A hospital tries out virtual assistants

Picture this: At every step of a patient’s trip through the hospital, a virtual assistant awaits, ready to answer questions, pull up medical records — or snap photos during surgery.

Hospital staff and patients walked through that futuristic hospital scenario one recent afternoon at a new simulation center at Boston Children’s Hospital. Fifty people came to dream up ways that Amazon’s Alexa voice-recognition software program could help in the clinic and at home.

The hospital dove into voice-assisted software last month, when it released an Alexa-based app called KidsMD that gives parents advice when their children catch a fever. Children’s “is leading the way when it comes to hands-free work in hospitals” using Alexa, an Amazon spokeswoman said.

Besides Children’s, other medical facilities are “working to bring Alexa into patient rooms, help doctors take notes, and read back charts, among other things,” the spokeswoman said, though she declined to share details.

To give visitors a sense of what’s possible, Children’s staff whipped up a few basic apps and set them up on electronic equipment in three mocked-up rooms in the simulation center — an operating room, an intensive care unit, and a child’s bedroom.

“We’re thinking about how these technologies come into play in every stage of the patient’s journey,” John Brownstein, Children’s chief innovation officer, told a crowd of clinicians, patients, and software engineers. “We’re counting on you to think about where this can go, if anywhere.”

The two-hour session generated enthusiasm for how the new tool could help — and gave a glimpse at how another layer of technology might frustrate staff.

In the operating room, visitors met Dr. Michael Docktor, a gastroenterologist and the clinical director of the hospital’s innovation group. As a “patient” lay on the operating table, Docktor walked through a scenario: During a colonoscopy, he said, he typically takes photos of the patient’s intestinal tract using a small camera called an endoscope. After the surgery ends, one to three hours later, he tries to sort through the photos to document the procedure. But that’s difficult, he said, because “every part basically looks the same.”

Instead, Docktor could just holler, “Alexa, record gastric body,” which could activate the camera and log the photo and a description into the patient’s electronic medical file.

Docktor walked over to an Amazon speaker to try it out. “Alexa, record duodenal bulb,” he said. “Alexa, stop.”

Alexa didn’t stop: “What else can I help you with?” she asked.

“Be quiet,” pleaded Docktor.

“What else can I help you with?” she asked.

“Stop, will ya?” Docktor said with a smile. He turned apologetically to the crowd. “This is obviously version 0.0.1.”

Iffy Wi-Fi hotspots messed up the system, which usually runs much more smoothly, he said.

In the ICU, visitors saw how Alexa could help nurses. Each time they draw blood, they have to figure out how much to draw, and choose which of many color-coded vials to put it in.

Nurse Paula Lamagna estimated she could save 15 to 30 minutes per patient if she used a voice-activated system to get that information.

In a third room, Jared Hawkins, director of informatics in the hospital’s innovation group, showed how parents can use KidsMD for simple medical advice. After some miscommunication — Alexa mistook 101.2 degrees for 1,001.2 degrees — the virtual assistant told him how much acetaminophen to use for a newborn child.

Alexa also helped him troubleshoot a problem with flushing a catheter.

“I can definitely see that being useful,” said 16-year-old Lena Williamson, a patient at Children’s who recently went home with a catheter.

Lena and others broke into small groups to brainstorm what else Alexa could do: When patients are lying in a hospital bed, for instance, they could summon Alexa to dim the lights, or turn up the heat, without having to bother a nurse.

Doctors teemed with ideas about how Alexa could help while they are wearing sterile gloves.

Dr. Anjolie Laubach, an anesthesiologist, said when she’s working with a patient in critical condition, she could ask Alexa to read aloud the patient’s heart rate every 30 seconds. Or Alexa could help find music during surgery, relieving nurses of that task.

“Nurses are our deejays,” Laubach said: In the operating room, doctors will often ask nurses, “Who has a Pandora account?”

Dr. Claire McCarthy, a pediatrician, said when patients show up in the emergency room, they have to repeat the story behind their illness as many as six times to a string of nurses and doctors.

“Literally, you would tell the exact same story,” she said. If a patient had a virtual assistant, he or she could simply record it once and say, “Alexa, repeat.”

Alexa could also help patients remember discharge instructions, and answer questions that patients might be embarrassed to ask, suggested research software developer Peter Warrington.

“The more we get them asking questions, the more engaged they are in their care,” noted Sarah Brand, a psychologist who works with kids with cancer. But if patients rely too much on a robotic interaction, they risk losing the human relationship with medical staff, she said.

Warrington raised another challenge: You have to make sure that people don’t hack into the system and get patients’ private details.

Jim Gregoric, who’s also a software engineer at Children’s, said the top concern with technology like this is patient safety: “Alexa better not give any information” — to medical staff, patients, or parents — “that causes any harm.”

Another question is whether patients really want to interact with a virtual assistant. Gregoric is an early adopter: He already has an Amazon Echo, an Alexa-enabled smart speaker, at home. He uses it to play music, get the weather report, or set a timer to remind him to pick up his wife.

His children, however, have shown little interest in the new technology, he said.

“You’d think the kids would take to it like ducks to water,” he said. But so far, he’s the only one welcoming the robotic voice into his life.

Pulse of Longwood takes you inside one of the nation’s largest hubs of hospitals and biomedical research.