The Inefficiency Paradox

Why are some studies showing AI could make people less productive? Simple: how much an individual produces does not equal productivity - we are measuring the wrong things.

There have been a few things recently that challenge our view of the core benefits of AI. The best known is probably this study from METR which showed that AI actually reduced productivity for software developers.

Cal Newport summarised several studies for and against time saving in software. Capturing the confusion with his title No-one Know Anything About AI.

And on a different but related topic, Arvind Narayanan and Sayash Kapoor published a long and thoughtful analysis of why AI might slow down scientific progress rather than accelerate it.

What is going on here?

How people use their time is not a simple assembly line type process. Knowledge workers - most workers in fact - have complex jobs.

So in the software example, writing a specific bit of code faster is not the same as doing a job faster. There is some indication in the METR study that the time saved was spent looking deeper for better ways to write the same code. Quality improvement in other words. Not so easy to measure.

Think of the old Chinese saying:

The solution to every problem is a gateway to another problem.

Passing through the gateway quicker just means more time to work on the next problem.

We are measuring the wrong things

In any case, how much an individual produces does not equal productivity

Productivity is the total of what a group of people produces divided by the number of people who produced it. Whether it is a company or a whole economy, improved productivity is a team effort. Time saving is an individual sport with only a tangential connection.

Of course, what makes for productivity is not just people but the tools and resources they use to do their work. Go back to the software engineer who spent his time making the code better rather than just write more code. On the face of it his productivity has reduced. Yet the better code might have led to more efficient work for other developers or for users of the software. Thus increasing overall productivity.

For a bigger picture view of this take this graph showing how the number of packages handled per amazon employee has grown over the years:

The purpose of this chart was to show how effectively amazon deploys robots. It also illustrates my point. Many of those employees probably never touch a single package. With robots, some packages may never pass through the hands of a single employee. Yet the overall productivity of the organisation has grown exponentially.

We are not replacing people

There are infinite micro and macro examples of how this works. The real world of producing goods and services is not about substituting machines (or algorithms) for people. That is the wrong paradigm.

People working together, making the best use of tools and resources, doing their best work on the right problems. The whole is not just greater than the sum of the parts, it is an organism with its own life. That's what drives productivity.

All these studies that show how much faster AI makes writing code/ producing advertising copy/ creating tik tok videos or whatever are measuring the wrong thing.

Thanks for reading

Bonus: Link to my follow up post Productivity and Personal Taylorism