摘要
An obstacle to effective teaming between humans and AI is the agent's "black box" design. AI explanations have proven benefits, but few studies have explored the effects that explanations can have in a teaming environment with AI agents operating at heightened levels of autonomy. We conducted two complementary studies, an experiment and participatory design sessions, investigating the effect that varying levels of AI explainability and AI autonomy have on the participants' perceived trust and competence of an AI teammate to address this research gap. The results of the experiment were counter-intuitive, where the participants actually perceived the lower explainability agent as both more trustworthy and more competent. The participatory design sessions further revealed how a team's need to know influences when and what teammates need explained from AI teammates. Based on these findings, several design recommendations were developed for the HCI community to guide how AI teammates should share decision information with their human counterparts considering the careful balance between trust and competence in human-AI teams.
学科与关键词
- I Know This Looks Bad, But I Can Explain: Understanding When AI Should Explain Actions In Human-AI TeamsZhang, Rui*; Flathmann, Christopher; Musick, Geoff; Schelble, Beau; McNeese, Nathan J.; Knijnenburg, Bart; Duan, WenACM Transactions on Interactive Intelligent Systems, 2024, 14(1): 6.被引 4
- Designing Transparency for Effective Human-AI CollaborationVoessing, Michael; Kuehl, Niklas; Lind, Matteo; Satzger, GerhardINFORMATION SYSTEMS FRONTIERS, 2022, 24(3): 877-895.被引 36
- How Being Outvoted by AI Teammates Impacts Human-AI CollaborationHu, Mo; Zhang, Guanglu; Chong, Leah; Cagan, Jonathan; Goucher-Lambert, Kosa*International Journal of Human-Computer Interaction, 2024.
