Automation as an intelligent teammate: social psychological implications

C2 processes will be increasingly affected by automation. Think, for example, of artificial intelligence that supports situation awareness or gives advice on a course of action. Insight into these effects are particularly important as automation evolves from intelligent task support towards an intelligent teammate. To enable effective human-autonomy teamwork, in addition to task-related factors, more knowledge is required on relational aspects. In general, human functioning in teams is largely affected by factors that emanate from an unconscious level. Think, for example, of assumptions regarding level of knowledge of other team members (taskwork), assessment of trust regarding commitment to the task, or assessment of social intent in communication (teamwork). This raises questions whether psychological mechanisms still operate in the same way, when human teams would be complemented by intelligent autonomous systems. Do humans apply the same rules to autonomous systems as they would to fellow humans and can these systems ever fully understand and act on the endless number of implicit social rules and underlying dynamics? We recently started a research programme to experimentally investigate these issues. We use a virtual environment resembling a first-person shooter task. In this task the participant makes decisions together with his buddy (an autonomous system) and by systematically varying various characteristics of the buddy we can investigate their impact on aspects such as acceptance of advice and trust.
To date, one experiment has been conducted mainly demonstrating that the quality of the buddy’s advice has a significant impact on acceptance and trust. A second experiment is currently being conducted focusing on trust repair. The main research question is to what extent trust repair will be affected by giving an apology and/or an explanation for the incorrect advice. The paper will provide an overview of relevant social-psychological factors on human automation interaction, the first results of our own research and some implications for future C2 processes.