In the summer of 2012, in Queens, New York, a 12-year-old boy dove for a basketball at school and cut his arm. He awoke the next morning with a high fever, vomiting, mottled skin and low blood pressure. His pediatrician diagnosed him with stomach flu and sent the boy to hospital to get re-hydrated. The emergency room doctors gave him fluids, ran blood tests and sent him home with instructions to take Tylenol.
But no one followed up on test results that showed an extremely high white blood cell count. His condition worsened and by the time the boy was admitted to the intensive care unit, it was too late. He died of severe septic shock, a complication from a bacterial skin infection that went undetected.
Missed diagnoses like this are a major cause of severe patient harm and preventable deaths. In fact, diagnostic errors result in more disability or death than surgical mistakes or medication overdoses, resulting in $2.4 billion in malpractice payouts over 10 years, according to a 2019 Johns Hopkins School of Medicine study.
Training medical students to solve real-world clinical problems like an expert is a focus for Susanne Lajoie (BA’78, MA’80) a McGill professor in education and counselling psychology. She is developing technology-rich learning environments (TREs) to avoid the cognitive errors that lead to dangerous diagnostic mistakes.
Solving virtual cases in BioWorld
Just off McTavish Avenue in McGill’s downtown campus, in a spacious, cutting-edge lab in the Education Building, 15 third-year medical students sit in small groups at interconnected computer terminals. They are all students in an internal medicine course, but who are also taking part in Lajoie’s research. The project involves solving challenging diagnostic situations in a TRE known as BioWorld.
As they solve virtual patient cases in BioWorld, every movement they make is recorded and analyzed in real time. They wear EDA (electrodermal activity) bracelets on their wrists to measure sweat secretions and emotional arousal, while cameras on each monitor use face-recognition to capture their joy, sadness, surprise, anger, fear or disgust.
“Social and emotional skills are just as important as cognitive skills in training students to perform well in high-stakes, real-life situations in internal or emergency medicine,” says Lajoie, Canada Research Chair in Advanced Technologies for Learning in Authentic Settings (ATLAS) and director of the Canada Foundation for Innovation-funded ATLAS lab. “Technology allows you to build immersive environments that engage students, so they feel they’re doing something like what they would be doing in the real world.”
These physicians-in-training also wear headsets to conduct think-alouds. By verbalizing the thought process, think-alouds make explicit the reasoning used in diagnosing cases, mapping how a novice or expert physician arrives at a correct answer. Students see similarities and differences between their own and an expert’s solution, learning how flaws in clinical reasoning contribute to diagnostic gaffes.
Give everyone a chance to learn
Lajoie’s personal interest and fascination with individual differences in learning began as a 10-year-old, growing up in Oshawa, Ontario, when she was abruptly separated from her elder cousin Diane. Due to an intellectual debility, Diane was sent away to live at a hospital school for developmental disabilities in Smith Falls. She was 16.
“I was shocked she was no longer part of my life,” recalls Lajoie. “The unfairness made me want to help and the irony was Diane had so much to offer. She taught me a lot as a kid. She was highly verbal and could explain things I didn’t know, like family dynamics. She knew how to cook and played piano.”
Lajoie’s passion for designing adaptive learning systems stems from her belief that everyone is capable of learning with the right guidance and support: “I think there’s never enough emphasis on learning,” she says.
Adapting instruction for different aptitudes
It was in the 1980s that Lajoie first used technology to help neophyte experts. She developed two novel intelligent tutoring systems, Orthographic Projection Tutor (OPT) and Sherlock, as a graduate and post-graduate student.
While a PhD student in educational psychology at Stanford University, Lajoie studied differences between psychology and engineering students. Like her mentor, educational psychologist Richard Snow, she believed that instruction could and ought to be adapted to individual aptitudes to enhance learning.
This spurred her to design a computer-based tutoring system, OPT, to teach the more verbally-oriented psychology students the spatial strategies and skills they lacked to solve geometric drawing problems. Her interactive computer tutorial taught them various strategies used by experts, gave feedback on their drawing errors, and tailored problem difficulty to an individual’s skill level
“OPT was one of the first computer-based learning environments developed and it helped psychology students improve their spatial ability significantly,” says Lajoie, who won the 1987 American Psychology Association award for best dissertation in education psychology.
Sherlock elevated avionics expertise
Lajoie’s next project involved teaching United States Air Force technicians troubleshooting strategies to repair electronic faults in an avionics test station for F-15 fighter aircraft in 1988. As a University of Pittsburgh post-doctoral fellow, together with mentor Alan Lesgold, she helped design and test an intelligent avionics tutor called Sherlock.
The program provided a realistic test station environment and dynamic assessment of the learner’s competence in solving real-world problems. As each trainees worked through an electronic fault isolation problem on Sherlock, they could seek coaching or feedback by clicking on the help button in the control menu. After two weeks’ training, novices were as adept as technicians with four years’ job experience.
Creative thinking to solve messy problems
Math or engineering problems are well structured with a clear procedure and path to the right solution. With OPT and Sherlock, Lajoie showed how TREs could help novices develop expertise in solving these types of problems. But this is not the case in medicine. Students need to develop expertise in solving unstructured, real-world problems, such as diagnosing uncertain conditions where specific symptoms could represent multiple diseases.
“Medical diagnostic reasoning is complex and ill-defined. Diagnostic problems are messy and harder to solve, since there is no single problem-solving sequence or set of rules that will lead to the right answer,” explains Lajoie, who has been affiliated with McGill’s Institute of Health Sciences Education since 2002.
At McGill, Lajoie has collaborated with Dr. Jeffrey Wiseman – an experienced internal medicine physician and winner of several McGill teaching awards – for over a decade in developing, refining and testing BioWorld as a cognitive apprenticeship tool to help medical students learn and become proficient in diagnostic reasoning.
“Internal medicine is a thinking-based specialty,” explains Wiseman. “Susanne created a participatory learning tool that makes expert thinking and decision-making more visible, and it captures students’ thinking. Traditionally, in a class of 30, one student presents and 29 are observing. In the ATLAS lab, all 30 participate in solving six virtual cases.”
Students are also more willing to make and learn from their errors: “Students feel more comfortable telling experts what they’re thinking with software than when speaking in person to a specialist. There’s no blame and no shame for making an error. I’d rather students make mistakes while trying to solve a case in BioWorld than in a critical situation.”
Meta-cognitive skills matter
A key dimension of expertise in diagnostic reasoning is metacognition, or the ability to think about one’s thinking. BioWorld includes several features to promote metacognitive skills. Sharpening these skills helps avoid premature closure (failing to consider other possible diagnoses after identifying an initial diagnosis) and confirmation bias (favouring evidence that supports their hypothesis while failing to consider evidence that would rule it out).
“Cultivating metacognitive skills helps reduce cognitive errors and improve diagnostic performance by encouraging students to consider more hypotheses and evaluate evidence in a flexible manner,” says Lajoie.
Her use of multimodal technologies like EDA bracelets and face recognition software is aimed at helping them understand the role emotions can play to enhance learning and improve performance. “Positive emotions lead to better learning and negative emotions to poorer learning. If we know what emotions a learner is experiencing, we can adapt the instruction, change the feedback to keep them in a positive emotional state and help move their learning forward,” she says.
Breaking bad news with empathy
When it comes to medicine, however, technique is one dimension but the human factor also plays a major part in healing. Lajoie’s learning tools are also helping medical students to develop emotional and cross-cultural communications skills. “Communication plays a significant role in physicians’ lives. Yet non-technical skills, such as how to break bad news to patients and show empathy, are minimally covered in medical education, despite research showing empathic communication plays a critical role in patients’ decisions,” she says.
Lajoie, Wiseman and Cindy Hmelo-Silver, a learning sciences professor at Indiana University, developed a digital and video-based tool called HOWARD, to help medical students in Montreal and Hong Kong learn how to monitor and manage their own and their patients’ emotions when communicating bad news to patients from different cultural backgrounds. Students delivered diagnoses of Hodgkin’s lymphoma and HIV-positive test results to patients via video, with critiques and feedback from physician experts and peers.
The students became more skilled at showing empathy, and in adapting their responses to cultural and individual differences in patients. “At first, some students would look unemotional, while others had huge smiles when giving bad news, because they were stressed or nervous. With practice, they were able to modulate their own emotional responses, listen more to patients and give results in a way that matched the individual patient,” says Lajoie, who plans to scale up the project with three medical schools on different continents.
“Sue has been a leading player in AI and educational psychology, building adaptive learning systems that use technology to engage learners’ emotions and see what ill-defined problems look like in the real world,” says Hmelo-Silver. “She’s super-organized and has a knack for putting together strong teams. She’s always positive, optimistic and appreciates her collaborators, which makes the research fun and productive.”
Saving a rapidly deteriorating patient
Lajoie’s knack for putting together multidisciplinary, international research teams is evident through her leadership of the eight-year, $2.5-million Learning Environments Across Disciplines (LEADS) partnership, funded by the Canadian Social Sciences and Humanities Research Council (SSHRC). LEADS brought together educators, psychologists, computer scientists, engineers, physicians and students across six countries, 18 universities and 13 partner organizations to design and implement TREs to teach 21st century skills from middle school to university in multiple domains.
HOWARD was one of 19 LEADS projects to enhance learning through digital technology. “In the LEADS partnership, we’re interested in seeing how technology can support the cognitive skills of problem-solving, and how it can engage the emotions behind learning,” says Lajoie.
Next fall, some McGill fourth-year medical students will have the opportunity to learn diagnostic techniques with a smartphone-based gaming app developed by Jeffrey Wiseman, The Deteriorating Patient (Lajoie assisted in its creation). This tool simulates crises where a physician must act fast, often with incomplete information. As learners decide what steps to take to stabilize a patient with deteriorating vital signs, they see the patient’s condition change in response to their actions or inaction. “Students experience very powerful emotional arousal because the patient is dying. In piloting this, we want to see if their emotional reactions interfere with cognitive ability, and how these emotions can help or hinder learning,” says Lajoie.
Authentic learning for all
Authenticity in feeling, thinking and doing reflects who Lajoie is as an education researcher, collaborator, innovator and person. “Susanne is very good at creating research and learning environments where experiment and play are welcomed and valued. She makes it comfortable and safe for her collaborators and students to try new ideas, and not feel bad if something they try crashes and burns,” says Wiseman. “Susanne also brings a wonderful dose of honesty, humility, humanity and caring to what she does.”
What Lajoie does is design digital practice environments to help learners develop expertise in solving real-world problems by engaging their emotions and advancing thinking. “I want to give everyone the best chance to learn by using technology to create authentic settings, where they can fail and succeed on tasks that are real and meaningful to them,” she says.