Is the Turing Test Dead?

When in 1950 Alan Turing first proposed an approach to distinguish the “minds” of machines from those of human beings, the idea that a machine could ever achieve human-level intelligence was almost laughable.

In the Turing test—which Turing himself originally called the “imitation game“—human participants conduct a conversation with unknown users to determine if they’re talking to a human or a computer. In 2014, a chatbot masquerading as a Ukrainian teenager named Eugene Goostman seemed to put one of the first nails in the Turing test’s coffin by fooling more than one-third of human interrogators into thinking they were talking to another human, although some researchers dispute the claim that the chatbot passed the test.

Today, we run into seemingly intelligent machines all day long. Our smart speakers tell us to bring umbrellas on our way out the door and large language models (LLMs) like ChatGPT can write promotion-worthy emails. Stacked up against a human, these machines might be easy to confuse with the real thing.

Does this mean the Turing test is a thing of the past?

In a new paper published 10 November in the journal Intelligent Computing, a pair of researchers have proposed a new kind of intelligence test that treats machines as participants of a psychological study to determine how closely their reasoning skills match those of human beings. The researchers are Philip Johnson-Laird, a Princeton psychology professor and pioneer of the mental model of human reasoning, and Marco Ragni, a professor of predictive analytics at Chemnitz University of Technology, in Germany.

“As chatbots have approached and succeeded at the Turing test, it has quietly slipped away from importance.” —Anders Sandberg, University of Oxford

In their paper, Johnson-Laird and Ragni argue that the Turing test was never a good measure of machine intelligence in the first place, as it fails to address the process of human thinking.

“Given that such algorithms do not reason in the way that humans do, the Turing test and any others it has inspired are obsolete,” they write.

This assertion is one that Anders Sandberg, a senior research fellow at the University of Oxford’s Future of Humanity Institute, says he agrees with. That said, he’s not convinced that a human-reasoning assessment will be the ultimate test of intelligence either.

“As chatbots have approached and succeeded at the Turing test, it has quietly slipped away from importance,” Sandberg says. “This paper tries to see if a program reasons the way humans reason. That is both interesting and useful, but will of course only tell us if there is human-style intelligence, not some other form of potentially valuable intelligence.”

Likewise, even though Turing tests may be going out of fashion, Huma Shah, an assistant professor of computing at the University of Coventry, in England, whose research has focused on the Turing test and machine intelligence, says that doesn’t necessarily…