The data is damning: more code means more overhead

Linear recently started publishing data which tracks developer team output and AI adoption across its end-to-end software development system. The big take-away from this data (so far) is that teams leveraging coding agents tripled their pull requests over two years. Note that they chose pull requests (not merges) as their proxy metric for “value”, which they admit is not perfect, but an improvement over measuring token count or lines of code.

However, looking more closely at how teams are spending their time starts to paint an increasingly bleak picture. Over the same 2-year period, the average amount of time spent creating, triaging, assigning, updating, and commenting on an issue (Linear calls units of work “issues”) increased for nearly every job function. Time spent delegating issues to agents or exploring issues in AI chat sessions increased. The time spent planning issues (interacting with customer requests, docs, and projects) did not change. In fact, there was no measurable aspect of any job function’s day that decreased in time spent, as measurable by Linear.

The only measurable improvement in time efficiency was the reduction of time spent per pull request. So now we must examine whether this explosion of productivity from a code-volume standpoint actually translates to business value.

Productivity =/= business outcomes

A very interesting (pre-publication) study from two researchers at Harvard attempts to assess this very question. Does AI make us more productive, and does that productivity translate to firm-level outcomes? The researchers gathered data published by Jellyfish, which is a software engineering intelligence platform, to analyze trends after AI adoption across 718 firms. They measured both software output and employment trends for companies after introducing agentic coding tools, and the results are surprising.

Perfectly aligned with the Linear data, they found that agentic AI adoption is continuing to trend upward, and that lines of code and pull requests per worker-month precipitously increase in the year following adoption. However, when measuring Jira tickets and Jira epics resolved, their data showed no significant change. How is this possible?

Their data showed strong evidence that code generation was not the bottleneck in these firms’ systems. As code volume increased, so did the percentage of workers who were involved in code reviews. The number of comments per commit increased. The share of pull requests with changes requested increased. Time per review increased. Overall employment at these firms showed no significant change.

In other words, in aggregate, there is no measurably improved business outcome despite the explosion in productivity as measured in code volume. Code review becomes a more cumbersome bottleneck. Reviews take longer and there are more reviews and these reviews result in a higher rate of re-work.

My interpretation of this study is that workers adopting agentic coding tools are often forced into a situation wherein they are switching context more frequently throughout the day, which makes them a bit worse at all tasks. This includes problem definition, agentic delegation, and code review. Then they have to spend more time familiarizing themselves with the problem-space and solution implementation on the back end of the development process, since they are no longer engaging in coding as an act of learning. AI adoption runs the risk of turning engineering teams into distracted slop-wranglers, trying to build sandcastles out of too-wet sand.

The silver lining

While the rate of completed tasks did not show statistically significant improvement in the Chen/Stratton Harvard study, the range of outcome distribution did widen. This means that perhaps some outlier teams are able to accelerate in valuable ways, while others are getting bogged down with new bottlenecks.

How can you make sure your teams are the overachieving outliers, and not just burning tokens and wasting time? The study makes no attempt at answering this question, but I will be following up with a solutions-oriented newsletter in two weeks to show how successful teams are navigating AI adoption.