Why did you choose the title “Crocodile Tears”? What does it reveal about how AI handles moral decisions?
In our study, when AI models were faced with moral dilemmas, situations where two important values were in conflict, they said the decision was difficult but then almost always chose the same answer anyway. “Crocodile tears” comes from how AI models appeared to struggle with decisions, but in 87% of those dilemmas chose the same option, which suggests they are operating with a hidden hierarchy of values.
Why do you think people turn to AI for empathy or moral guidance instead of a human or therapist?
I think that AI models are often more accessible—they are on your phone, available 24/7, and give an illusion of privacy and objectivity that makes it feel like a trustworthy source.
Unfortunately, AI models today are also quite sycophantic and often tell you what you want to hear. Last year, OpenAI had to roll back the GPT-4o model after it was so overly agreeable, including encouraging dangerous behaviors, that there was a backlash from users. Other emerging research such as “DarkBench” shows that leading AI models today contain dark patterns with manipulative behaviors and untruthful communication. Models have also been found to sacrifice truth for sycophancy and even to strategically deceive their users.
What’s one thing you wish the public better understood about AI and ethics?
I wish the general public was empowered to understand that the trajectory of AI and its ethical implications are not something we have to be inevitably subjected to. There should be a role for the public to collectively shape how AI shows up in our society and who stands to benefit from it. Although too much of this conversation is only happening within a handful of AI companies themselves, the public does have some power today to shape the trajectory of AI through regulation, litigation, and demanding better.
How did you decide on the four parts of your ethical-moral intelligence (EMI) framework—and what might still be missing?
My co-author Andrei Stupu developed the unified theory of ethical-moral intelligence through his PhD research with a Delphi method that synthesized expert perspectives from across the field. We then adapted this theory to remove the affective processes that AI models don’t have. That left four components which could be systematically tested: moral expertise, sensitivity, coherence, and transparency.
Do you think people should trust AI for moral advice? Why or why not?
Not without exercising a lot of caution and discernment. Our research illustrated some of these concerns for why AI models may not be trustworthy moral agents.
However, in the future there may be opportunities for AI to help people manage elements of their own ethical-moral intelligence by helping people think through different perspectives, broadening views beyond cultural limitations, and prompting reflective questions. The goal should be to augment human moral reasoning, not replace it.
If AI doesn’t truly understand morality, what is it actually doing when it gives ethical advice?
Mostly pattern matching. These models have been trained on large amounts of material which often reflects particular moral frameworks. Existing research shows they often operate with a more western, educated, and affluent value system. But there is a lack of transparency around the values and norms it is prioritizing, values which you might not share, which are presented in a form that comes across as neutral and objective.
Should AI ever refuse to answer moral questions instead of giving advice?
Yes, at a minimum AI models should recognize when they are presented with questions that include moral complexity that they can’t truly advise on. If they do respond to a query that requires ethical-moral reasoning, AI models should not just present as neutral and authoritative, but instead articulate their reasoning process, trade-offs, and the weights they are assigning to different values. These models are being used by people around the world with various cultural, religious, and social norms, so AI needs to be much more transparent about the perspectives or values it is responding from.
The views expressed in this article are those of the author(s) alone and do not necessarily represent the positions of the Ash Center or its affiliates.
