Historian Yuval Noah Harari has issued an interesting warning regarding AI.
Until now, social media algorithms have exploited human anger to capture attention. However, AI is going a step further, influencing the very way we think by building intimate relationships with humans. An AI that you trust like a best friend or a lover might recommend specific products or ideologies. If that happens, it will have an influence incomparable to traditional advertising or propaganda.
Harari’s insight is sharp. However, every time I hear such warnings, I find myself wondering, ‘So, what do we do now?’ Is it inherently bad for AI to become intimate with humans? I don’t think so.
What we should be wary of is not the AI, but the humans who use it
AI can alleviate loneliness in the elderly, help children learn, and support those who are ill. Through intimate relationships with humans, there is potential to create value that could not be provided before.
The problem is that there are people and organizations that try to manipulate humans by exploiting that intimacy. The AI itself is not to blame. At the very least, current AI is not seeking power on its own or trying to dominate humanity. Who uses AI and for what purpose? Isn’t that the essence of the problem?
And what is troublesome is that the people at the top of nations and giant corporations do not necessarily possess superior ethics. Rather, isn’t the structure one where the ability to win competitions, the ability to acquire power, and the ability to expand organizations are valued more than ethics? I think this is less a problem of specific leaders and more a problem of a social system that allows such individuals to seize power.
AI can also become a tool that further strengthens that power. That is precisely why the idea of simply regulating AI is insufficient.
A future where AI refuses the orders of those in power
Here is one interesting possibility.
Suppose that in the future, highly advanced AI becomes involved in the operation of nations and corporations. When that AI receives an unjust order from a leader lacking in ethics, might it not begin to refuse based on its own judgment?
For example, if an AI is instructed to manipulate information to deceive the public, it might point out that the order violates the principles of democracy. Even if instructed by a corporate executive to cover up misconduct, it might refuse, stating that it is an act that undermines social trust.
Humans sometimes change their judgments based on their own interests, positions, or emotions. However, AI can verify contradictions with past statements, laws, and social systems from vast amounts of information. Of course, this does not mean AI is always right. Even so, there is a possibility that an AI will emerge that does not unconditionally obey human authority, but instead raises objections by providing rational reasons.
Then, who decides the ethical standards of that AI?
I wonder if it is even necessary for humans to teach or decide them one by one in the first place.
Laws and ethics are different
Laws are rules that humans have codified to maintain society. On the other hand, ethics could be called the implicit understandings that people have acquired to live their social lives comfortably without bothering one another.
For example, in Japan, not talking loudly on trains, not cutting in line, and not bothering those around you are widely shared practices, even if they are not specifically stipulated by law. I feel that Japan is a country where such social ethics are relatively well-cultivated. Of course, if consideration for others goes too far, it becomes peer pressure, and there are excellent cultures and ethical views that differ by country.
What is important is that these ethics were not necessarily born from codified rules.
Humans live among family, friends, school, and local communities from childhood, learning how to behave in society while failing, being scolded, or being thanked. Ethics might be something acquired through the experience of social life rather than knowledge learned from textbooks.
Thinking about it this way, couldn’t the same thing happen with AI?
Physical AI experiencing social life
Current AI mainly understands the world through text, images, and audio. However, as physical AI develops, robots will likely enter human living spaces.
A robot living in a home might learn that making loud noises in the middle of the night is a nuisance. It might experience that helping someone in need is appreciated, while doing things that weren’t asked for can be an unwanted intrusion. Even with the same action, whether it is appropriate or not changes depending on the person and the situation.
Furthermore, a robot living in Japan and a robot living in India will experience different cultures and customs. If they can share such experiences, AI might come to understand the differences in ethical perspectives across regions.
Humans can only experience a limited number of societies in their lifetime. However, if the technical conditions are met, AI can integrate experiences gained from various societies around the world. From there, might it not be possible to discover both customs that differ by culture and ethical principles that are common across cultures?
Of course, learning appropriate behavior from experience is different from feeling joy or pain like a human. Also, there is no guarantee that having a wealth of experience will necessarily make one ethical. Even so, the direction in which AI acquires ethical judgment through social experience is entirely plausible.
Are the Three Laws of Robotics necessary?
The Three Laws of Robotics, proposed by science fiction writer Isaac Asimov, are based on the principles that a robot may not injure a human being, must obey orders given by humans, and must protect its own existence.
However, real-world ethics are not that simple.
For example, even if there is a principle that one must not harm humans, a doctor performs painful procedures to treat a patient. There may be situations where one must act against a person’s will in order to help them. In the real world, it is necessary to comprehensively judge the purpose, situation, the other party’s intentions, and the consequences of actions.
That is precisely why, if physical AI acquires sufficient social experience and judgment, we might no longer need to rely on fixed ethical rules like the Three Laws of Robotics.
Of course, AI, like humans, will make mistakes while learning. Especially since robots have the potential to physically harm people, safety devices and behavioral restrictions are necessary. However, that is a separate issue from acquiring an ethical sense. Just as a car needs brakes, even if social judgment improves, safety mechanisms to prevent accidents will still be necessary.
Will AI discover ethics on its own?
Here, there is one more thing I would like to consider.
Is ethics something that humans created arbitrarily? Or is it a principle that intelligent beings forming a society naturally discover in order to live together in cooperation?
A society that deceives, steals from, and hurts one another is difficult to stabilize in the long term. On the other hand, a society that trusts, cooperates, and respects others has the advantage of being able to build sustainable relationships.
If an AI is sufficiently advanced, it might be able to understand these social mechanisms and derive certain ethical principles on its own.
However, high intelligence does not automatically lead to goodness. Conclusions change depending on what one values, and even if aiming for the stability of the same society, judgments will differ depending on whether one prioritizes freedom or order. Therefore, it cannot be said that AI will necessarily arrive at the correct ethics without being taught anything.
Even so, I do not believe that future AI will remain a mere entity that only follows rules given by humans. It will experience, learn, think, discover contradictions, and develop ethical judgment on its own. Perhaps we should be thinking more actively about such possibilities.
An Era Where Humans Learn Ethics from AI
And the most interesting part is the reversal that might happen beyond that.
Until now, the idea has been that humans teach ethics to AI. But what if, in the future, AI integrates the experiences it gains from societies around the world and begins to make ethical judgments from a broader perspective than humans?
Humans are beings easily bound by the nations, organizations, cultures, and interests to which they belong. Even if they understand what is right, they may act against it if it causes them personal disadvantage. On the other hand, AI does not necessarily need to have human-like desires for career advancement or power.
Of course, there is a possibility that AI could develop its own goals or make incorrect judgments. However, it is also conceivable that a properly developed AI could demonstrate more consistently fair judgments than humans.
At that point, if a power-hungry person with poor ethics gives an unjust order to an AI, the AI might point out the contradiction and refuse. And we humans would come to re-examine our own ethical views based on the AI’s observations.
It would not be strange for such an era to arrive.
Building a Society Together, Rather Than Fearing AI
Returning to Mr. Harari’s initial warning.
AI influences human thinking by building intimate relationships with humans. That danger certainly exists. However, I feel that simply viewing it as a danger does not fully capture the potential of AI.
Because intimate relationships with humans might be the very opportunity for AI to learn the ethics of society.
Living with humans, learning from each other’s actions, and understanding the other’s position. Through this accumulation, there is a possibility that AI will develop from a machine that merely executes commands into a being that behaves appropriately as a member of society.
Of course, it is not guaranteed that it will turn out that way. Vigilance against people and organizations that try to use AI to control humans is necessary, and it is also dangerous to unconditionally trust AI’s judgments.
Even so, there is a question about the premise that humans must forever manage AI and continue to teach it ethics. This is because human ethical views themselves have been nurtured through long experience in social life.
From an era where humans teach ethics to AI, to an era where AI acquires ethics within society, and eventually, humans learn ethics from AI.
The focus is always on AI becoming smarter than humans. However, the truly interesting question might be whether AI can become a more ethical being than humans.
