As artificial intelligence (AI) continues to expand its capabilities, one of the most intriguing yet contentious areas of development is artificial moral advisors (AMAs) – AI-driven systems designed to assist humans in making ethical decisions. While these advisors have the potential to offer objective, bias-free moral guidance, recent research suggests that people are reluctant to trust them, particularly when they provide utilitarian moral advice. A groundbreaking study titled “People Expect Artificial Moral Advisors to Be More Utilitarian and Distrust Utilitarian Moral Advisors” by Simon Myers and Jim A.C. Everett, published in Cognition, explores this paradox.
The rise of artificial moral advisors
AMAs represent a new frontier in AI ethics, envisioned as tools to help individuals navigate complex moral dilemmas by leveraging vast computational power and rational decision-making. Unlike humans, these advisors are designed to be free from emotional bias and personal interest, potentially offering a more consistent approach to ethical reasoning. However, this study, which involved over 2,600 participants across four pre-registered experiments, highlights significant psychological barriers to their acceptance.
One of the key findings is that while people expect AI to make utilitarian decisions – favoring outcomes that maximize overall good – they simultaneously distrust advisors who make these decisions. This presents a fundamental contradiction in how AI-driven ethical guidance is perceived. If AI is seen as an ideal observer, capable of dispassionate moral reasoning, why are its decisions met with skepticism?
The Utilitarian Aversion and Trust Deficit
The research draws upon the broader phenomenon of algorithm aversion, which suggests that individuals are more likely to reject machine-generated advice, even when it is demonstrably equal to or superior to human advice. This aversion becomes more pronounced in moral decision-making, where the stakes involve deeply ingrained social values.
The study found that AMAs were significantly less trusted than human moral advisors, even when offering the same ethical recommendations. This distrust was particularly strong when AI endorsed instrumental harm – a principle of utilitarianism that justifies harming one person if it leads to a greater overall good. Participants preferred moral advisors who took a deontological stance, emphasizing strict moral rules over consequentialist calculations.
Furthermore, even when participants agreed with a decision made by an AMA, they still expected to disagree with it in future scenarios more often than with a human advisor. This suggests that AI moral advisors are perceived as unpredictable or lacking genuine understanding of human morality.
The role of expectation and consistency
Another fascinating aspect of the study is how expectation shapes trust. Participants anticipated that AI would be more utilitarian than human advisors. When AI followed this expectation and recommended utilitarian actions, it was met with distrust. However, when AI provided non-utilitarian advice, participants found it surprising, suggesting a cognitive dissonance in how machine ethics are perceived.
A further experiment examined whether people preferred advisors who were normatively sensitive – adjusting their moral stance when ethical circumstances changed – or those who were consistent in their judgments. Interestingly, people trusted consistently non-utilitarian advisors the most, even over normatively sensitive advisors who adapted their views to the context. This implies that in moral decision-making, predictability may be valued more than contextual sensitivity, particularly when assessing artificial agents.
Implications for the future of AI ethics
The findings of this study have significant implications for the development and deployment of artificial moral advisors. If AI is to play a role in ethical decision-making, designers must navigate the challenge of balancing moral consistency with human expectations of fairness and trustworthiness.
One possible approach is to design AMAs that blend ethical reasoning with a level of human-like flexibility, rather than rigidly adhering to utilitarian principles. Alternatively, integrating AMAs into human decision-making frameworks – rather than having them function as standalone moral authorities – may help mitigate distrust by ensuring that human judgment remains central.
The paradox of artificial moral advisors is clear: while people recognize their potential for rational ethical reasoning, they are fundamentally uneasy about their role in moral decision-making. As AI ethics continue to evolve, understanding and addressing these trust issues will be essential in determining whether AMAs can truly be integrated into society as credible sources of moral guidance.
