Toward intelligence augmentation: design approaches for effective deployment of healthcare artificial intelligence
Dr Sharon Tan Swee Lin, Associate Professor, School of Computing
Abstract:
While artificial intelligence (AI) technologies are poised to automate a variety of tasks that are currently performed by human workers, many complex and high-stake decisions are often solved through humans and AI working cooperatively. It is therefore important to understand how to effectively leverage advanced AI applications to amplify human capabilities. However, little attention has been paid to the design, deployment, and use of AI in daily work routines. This thesis attempts to close this knowledge gap by examining how to effectively deploy healthcare AI applications in clinical training and practice.
This thesis encompasses two essays. The first essay, titled "Designing AI-based Work Routines: The Timing of AI Advice Provision Shapes Medical Decision Making," focuses on the potential of healthcare AI to provide decision support in clinical practice. Drawing upon the egocentric advice discounting, the cueing effect, and other relevant theories from psychology literature, I experimentally examine the differential impacts of two different AI advice provision timing designs (i.e., independent timing design versus dependent timing design) in a medical diagnosis context using the think-aloud approach. Specifically, physicians in the independent timing condition have the opportunity to independently evaluate clinical cases and develop an initial diagnosis decision before obtaining AI advice, whereas those in the dependent timing condition do not. The experimental results show that physicians in the independent timing condition make more accurate, relevant, comprehensive, and calibrated diagnosis decisions than those in the dependent timing condition. Overall, our analysis demonstrates that providing AI assistance at an appropriate timing is critical to successful decision augmentation.
The second essay, titled "Artificial Intelligence-Based Training Promotes Diagnostic Skill Acquisition in Medical Novices," focuses on the potential of healthcare AI as a novel and supplementary training tool to existing undergraduate medical training programs. Building upon the ACT theory of cognitive structure and other relevant learning theories, this essay proposes a theory-driven AI-based training intervention for diagnostic skill acquisition. I conducted a randomized field experiment to evaluate the effectiveness of AI-based training in a real-world medical training environment. The training outcomes of interest include self-efficacy, diagnostic accuracy, and diagnostic efficiency, which capture three important dimensions of diagnostic expertise. The results show that AI-based training has a positive effect on self-efficacy, diagnostic accuracy, and diagnostic efficiency, and the skills gained from the training endure over time. Moreover, the training effectiveness on diagnostic accuracy is partly driven by improved diagnostic reasoning capabilities. These results demonstrate that AI-based training can serve as a practical and easily scalable training approach for diagnostic skill development. Overall, this essay indicates that AI has the potential to promote learning and skill acquisition among novices.
Overall, this thesis enriches our understanding of the impacts of healthcare AI applications on diagnosis decision-making, working processes, and skill acquisition in clinical training and practice. It concludes by summarizing the findings and implications of the two included essays and providing recommendations for future research in this area.