Measuring AI’s Societal Impact, Not Just Performance

As AI systems become more capable, a lot of resources and effort are being put toward measuring their abilities. Researchers look at technical evaluation metrics, subject AIs to reasoning tests, track their throughput, and much more. But there’s one key metric that often gets overlooked, and it’s arguably the most important of all: What is AI doing to humans?

Imran Khan leads psychosocial evaluation of AI at the nonprofit Center for Humane Technology. In a recent essay published on the organization’s Substack, Khan points out that we’re deploying AI tools capable of reshaping our cognition, relationships, and behavior, but with little systematic effort to measure the downstream impacts they’re having on us.

The push to look more closely at AI’s psychosocial effects is similar to debates that emerged around social media and its harms, but Khan believes AI could have even broader and more intimate effects. The focus on measuring AI performance and progress misses the question of whether the technology is ultimately helping humans flourish—or eroding some of our most fundamental capacities.

IEEE Spectrum spoke with Khan about why AI evaluation is so narrowly focused, what meaningful measurement of human outcomes might look like, and whether the AI industry has incentives to ask these questions at all.

The missing question about AI model performance

In your essay, you argue that we’ve become very good at measuring what AI systems can do, but bad at measuring what they do to humans. What made you realize this was the missing question?

Khan: If you spend any time in and around the AI development space, you see this amazing progress in terms of what models are capable of, with graphs of how well different models perform on tests like SWE-bench or humanity’s last exam or LLM arena. There’s a competitive dynamic to how AI companies want to progress and be known for their models being the best. You see that impressive data, but then you also see these scary and dangerous things that happen in the real world, like teenagers dying by suicide and people succumbing to AI psychosis.

So on the one hand, we’re devoting an incredible amount of energy to measuring how AI does on these sometimes quite abstruse things that have limited relevance to most people’s day-to-day lives. And on the other hand, AI is impacting human well-being, and we’re measuring that much less. It seemed like a strange paradox that the things we should care about most, we’re measuring least.

Your essay points out that with social media, harms were already entrenched by the time the evidence was strong enough to act on them. Do you think AI is already producing measurable harms at scale, or are we still in an early-warning phase? What differences might there be in how quickly harm from AI evolves?

Khan: There are some really high-profile cases that I think are the tip of the iceberg—the teen suicides, AI psychosis, people spending immense amounts of time or money…