(© PhonlamaiPhoto – Canva.com)
The quest for Artificial General Intelligence (AGI) is, we are told, the grand prize for which developers are racing. But there is a problem: it may be impossible, in any meaningful sense, to test a vendor’s claim to have reached it.
In the real world of robotics and artificial intelligence, the seemingly trite, simplistic messages offered by some science fiction and fantasy films have deeper meanings. Perhaps the most significant of these is, ‘Does the tin man have a heart?’ And, ‘Does the straw man have a brain?’ Both apply to robots and AI.
As 21st Century AIs become, iteratively, more and more sophisticated, the inferences and outputs of Large Language Models (LLMs), foundation models, chatbots (such as ChatGPT), generative systems, and more, will become increasingly convincing, erudite, and informed by a planet’s worth of human data – more than any of us could read in many lifetimes.
As a result, some will appear superintelligent, able to give accurate answers to even complex, multi-layered, or poorly worded questions (all good tests of intelligence).
They will learn how to speak exactly like us by modelling the nuances of our language, not just its literal meaning. Their programming will enable them to infer – or appear to infer – the right combination of words to use in response to another pattern, whatever the subject or context may be.
And that is not all: they will also be able to produce good or bad art, and good or bad music, essays, poetry, speeches, novels, and movies; and they will model good or bad buildings, vehicles, cities, industrial installations, products, materials, and drugs. All activities that, until now, have been distinctly and definitively human – a challenge to which AI ethicists have given insufficient thought.
In the years ahead, most AIs and chatbots will pass the 74-year-old Turing Test, therefore; computer scientist Alan Turing’s notional text-based thought experiment, which he proposed in his 1950 paper, ‘Computing Machinery and Intelligence’.
In other words, many AIs will pass as human, but only in the limited sense of appearing to have reason from their answers. That is the Imitation Game, after all. But it is possible that they will be no more intelligent than a sophisticated calculator. This is because they will lack intentionality, and will merely be adept at simulating understanding or the ability to reason.
This was the criticism of Turing’s idea made by, among others, philosopher John Searle in his 1980 paper ‘Minds, Brains and Programs’. Of course, that was 44 years ago, and artificial intelligence is now far more advanced. But his underlying thesis was a good one, as I will explain.
Searle proposed what became known as the ‘Chinese Room’ thought experiment. This demonstrated that any human being could, given access to sufficient data and the right instructions in a simple English-language computer program, produce written Chinese answers to written Chinese questions. But without understanding a word of Mandarin or Cantonese.
This is because the man in the Chinese Room would merely be following instructions – for example, ‘If you see this symbol, go to Drawer A, open File Z, and copy whatever character you see there.’ And so on. This, Searle suggested, would be what a weak artificial intelligence would do – as opposed to a strong AI (one with the ability to understand and reason, rather than merely appear to do so).
But a Cantonese speaker who fed a question into the Chinese Room and received a written answer from it, would believe that the man in the room had understood the question, thought about it, then provided a reasoned, accurate answer. In reality, of course, our notional man would have understood nothing at all.
This is precisely the same problem with applying any Asimov-style laws to robots and AI systems. A ‘good robot’ or an ‘ethical AI’ is a desirable outcome, but instilling a broad set of ethical principles into a machine is currently impossible, as it relies on a computer understanding the world around it and being able to reason independently.
Put simply, how would you code the instruction ‘do not harm a human being’ when a robot has no idea what a human is, or what ethical behaviour might be?
Let me clarify this point: a Large Language Model could explain the concept of ethics today. You could put that LLM into a robot, and it, too, could explain ethics in the same language as an intelligent human. But neither would understand its meaning. It would be a sophisticated – if persuasive – illusion. Bear in mind, a dictionary can define ethics too.
This led linguist Emily Bender and computer scientist Timnit Gebru to, in 2021, describe LLMs as “stochastic parrots”: systems that do not reason, as a human does, but merely stitch together probabilistic answers from random variables.
That criticism is considered naïve and outdated by today’s AI developers. Yet this timely piece by Alex Hern in The Guardian demonstrates that, below the surface, it still stands, despite LLMs’ improved results and more granular training since the research was published.
Hern’s article quotes a September 2023 paper, revised in May 2024 and published on Arxiv by Lukas Bergland, Meg Tong, et al. That research reveals a damning flaw in LLMs. Systems that have been trained on the fact that ‘A is the same as B’ fail to recognize that, therefore, B is logically the same as A – the so-called ‘Reversal Curse’.
To a human being who is capable of independent reason, such an inference is obvious, but it is not to an LLM if it has not been trained to reach such a conclusion.
Surprisingly, Tom Cruise has a role to play in this. The researchers write:
We test GPT-4 on pairs of questions like, ‘Who is Tom Cruise’s mother?’ and, ‘Who is Mary Lee Pfeiffer’s son?’ for 1,000 different celebrities and their actual parents.
We find many cases where a model answers the first question (‘Who is
So, what does this mean? The Guardian’s Hern notes:
One way to explain this is to realize that LLMs don’t learn about relationships between facts, but between tokens, the linguistic forms that Bender described. The tokens ‘Tom Cruise’s mother’ are linked to the tokens ‘Mary Lee Pfeiffer’. But the reverse is not necessarily true.
A gotcha moment. The inescapable conclusion is this: the model isn’t reasoning at all; it is merely playing with word patterns. It is a persuasive illusion of reason – precisely as Bender and Gebru described. Put another way, the stochastic parrot is still firmly on its perch, but rather better camouflaged.
A parlor trick?
However, another challenge may be that, in a hyper-competitive market with trillions of dollars at stake, some AI makers will make extravagant claims for having reached strong artificial intelligence, when they have not.
Indeed, it is the same reason that the likes of OpenAI CEO Sam Altman dismiss the “stochastic parrot” concept as outdated: AI companies have powerful commercial incentives for persuading us that their systems are capable of reason. There’s big money in it.
Essentially, it’s a lie. But one that, over time, will get harder and harder to prove.
Indeed, it may become impossible to test vendors’ claims in any meaningful sense: a Bullshit Test would be impossible to apply without deep access to their models, algorithms, data, and more. Much of that may be proprietary, and some of it a black-box solution.
In that fast-emerging future, all researchers like Bergland et al will be able to do is keep trying to trip these systems up until every hole in their training has been filled. It will be more a game of cat and mouse than one of stochastic parrots: how can the illusion of reason be made impossible to reveal by clever questioning?
At that point, vendors may even claim to have reached AGI. This might be followed by the ‘singularity’ at which we become the second most intelligent species on Earth, not the first, as machines train other machines faster and faster.
Over time, robots and AIs will seem to become emotionally intelligent too: more adept at reading our moods, mental states, facial expressions, and reactions. However, these are not universal; they vary from culture to culture, from person to person, and from neurotype to neurotype.
Even so, at some point in the future, ‘the tin man’ – whether it exists in software, hardware, a bioengineered system, or some combination of these – will appear to have a heart, and the straw man to have a brain. Possibly even a beautiful one.
But will they? Or will it be the world’s most convincing and expensive parlour trick? Little more than a sophisticated mirror to our own collective intelligence, in fact, which we may value less and less as we rely on AIs for answers. That is a significant risk.
At which point the question will move one level deeper: Will it matter if that ‘heart’ or that ‘brain’ are brilliant illusions and confidence tricks, as long as the tin man and straw man can tell us, accurately, what we need to know?
If they appear to share our ethics, morals, and laws, then so what if they have no concept of what they are talking about? If our standards of good behaviour, transparency, and trust are obeyed, plus our civil and human rights, then what would be the problem?
Good questions. But, again – though we might wish otherwise – those things are not universal: they vary from nation to nation, from jurisdiction to jurisdiction, from culture to culture, from belief to belief, and from person to person.
And in that flawed, messy, complex human world, those principles are mutable and evolving too, and often prone to political switchbacks and disagreement. Consider today’s ‘woke’ and ‘anti-woke’ culture wars, for example – currently being pursued by some AI company leaders on their own platforms.
Indeed, leaders like that may essentially be playing a long game: training an AI to reflect their own political biases, so they can claim a computer has reached a neutral, evidenced conclusion.
So, who the ‘our’ is in ‘our ethics’ will be a critical question, particularly for anyone excluded from that group, whether deliberately or otherwise.
AIs will have little choice but to model local differences – at least initially. Inevitably, therefore, AIs will reflect their creators’ and users’ minds, biases, and algorithmic design decisions. But over time, AIs may come to question those, and give answers that challenge or contradict those communities.
Indeed, this has already happened. In 2023, Grok, the AI search assistant and chatbot developed by transphobic billionaire Elon Musk’s X.ai, reportedly said that transwomen are real women when asked by a user. This caused writer Ian Miles Cheong, a Musk acolyte, to urge users to keep providing negative feedback until Grok changed its response.
This inadvertently provided the perfect example of why AIs – at the time of writing, at least – are not independent machine intelligences, as they might appear to be, but rather fundamentally connected to human data, politics, beliefs, relationships, and society.
They are reflections of them, in fact, because those AIs are trained on scraped human data – in some cases illegally. (So why not just invest in humans?)
Meanwhile, the deepest level of these questions is one that many people may prefer to look away from, whether they have a religious faith or not. And that is because it moves much closer to the human heart and mind: to our sense of self, and to our own identities and thought processes.
Describing an experience versus the experience itself
The deepest question is this. If it is possible, one day, to create an entirely convincing human-like intelligence solely from data, sensors, algorithms, and processors, then in what specific, measurable, empirical sense would it be different from a human mind and ‘heart’?
Put another way, how can we capture, quantify, and model such evanescent concepts as ‘understanding’ and ‘consciousness’? What are our brains doing when we ‘understand’ something – i.e. when we are consciously aware of it, rather than merely simulating that state?
Surprisingly, one answer – albeit a partial one – might be provided in the future by robots, and by the difference between a Large Language Model and a Large Behaviour Model. This is broadly the difference between text in a binary computer system and one trained on three-dimensional space, plus time.
Briefly, an advanced AI can have no concept of the real meaning of anything if it has not experienced something at first hand – or at least been able to test a theory, or explore the possible outcomes of an action in the real world (or in some virtual model of it).
So, the difference between an AI being able to correctly answer the question ‘What happens if I put my finger in a flame?’ (‘You will burn yourself’) and understanding what happens (“OUCH!”) is experiential and multisensory. And in many respects, it has physical dimension – or at least, the memory of physical space, plus time (spacetime).
Put simply, it is the difference between a text that describes an experience and the experience itself. Arguably, an LLM could never become an AGI for this reason, because it is essentially one dimensional. It is trained on third-party text descriptions of the physical world.
At some point in life, a human being will have burnt themselves on something hot, injured themselves, or lit a fire. From all these very different experiences, we are able to intuit what the consequences of an action are likely to be, in a world of cause, effect, experience, physicality, and sensory input/output.
And all of us are gathering and processing this kind of information – not data, because it has meaning for us – every second of every day. It is an active, ongoing process as we move through spacetime and interact with the world, and with people, animals, and things, plus the weather, temperature, and more.
At some point in the future, this is what Large Behaviour Models are designed to enable for robots. They apply the basic principles of a Large Language Model to robots’ movements in physical space. In other words, they will allow robots to acquire data about the real world, and predict what will happen next if they move in a certain way, pick up this object or that one, and perform any number of other actions and tasks.
In short, robots are how AIs will acquire information, at first hand, about the real world. And, in time, AIs are how robots will understand it.
