What’s The ROI Of Software Engineers’ Spending On AI?

TL;DR

Before you orderOffer from Amazon

Get the latest gadgets delivered free with Prime

• Fast, free delivery on millions of items
• Prime Video, Amazon Music and more included
• Member-only deals all year

Start your free Prime trial →Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.Hide

The Pragmatic Engineer gathered AI spending and pull-request data from 27 workplaces, alongside aggregate figures from CodeRabbit and Gitar. CodeRabbit reported roughly twice as many weekly pull requests among platform users as a year earlier, but the report found that companies generally could not say whether the extra output justified their AI bills.

The Pragmatic Engineer reported that AI coding-tool spending varies widely across 27 workplaces, while companies often cannot yet quantify its return on investment. Data shared by CodeRabbit showed platform users creating 113% more weekly pull requests than a year earlier, but that measure alone does not establish that AI caused the increase or that the additional work produced greater business value.

The report’s author said they spoke with engineering leaders and senior engineers at 27 workplaces and collected aggregate data from CodeRabbit and Gitar. The sample included companies with sharply different spending policies: a two-person .NET business reportedly spent $3,000 per developer per month, while a publicly traded company with 1,500 engineers capped spending at $150 per developer monthly. The examples are not a representative survey of the software industry.

CodeRabbit, an AI code-review platform, shared anonymized data from engineers using its service. It reported a 113% increase in weekly pull requests and a 127% increase in monthly pull requests. For a tracked group of 20,000 developers, the company reported a 128% rise over a year. Those figures describe activity among platform users; they do not by themselves measure code quality, completed product work, or productivity attributable to AI.

Gitar, described in the report as an AI code-verification agent, supplied figures on pull requests per developer per week across hundreds of customers. Its data indicated that some teams classified as “AI-native” ship more than 60 pull requests per developer weekly. The report also described varied spending patterns, common spending caps among medium and higher spenders, and costs that some companies said had fallen from summer peaks as they changed models or moved to open models.

More Output, Unproven Returns

The findings illustrate the measurement problem facing companies that are expanding access to AI tools: software activity is easier to count than economic value. Pull-request volume can offer a readily available comparison over time, but it does not show whether code is useful, whether review and maintenance burdens have grown, or whether work reached customers faster.

The difference matters for budget decisions. The report says some firms use spending caps and that subsidized subscriptions at several companies may end, potentially raising bills. If firms face higher direct costs, leaders will need to compare them with measurable outcomes such as delivery time, reliability, or engineering capacity. The report’s evidence shows that spending and output are changing, not that one has caused the other or that the investment is profitable.

How the Report Measured Activity

The author chose pull-request throughput because it is available at many companies and can be compared before and after adoption. The report also cautioned, in effect, that workplace culture shapes output: teams differ in how often engineers submit changes, so raw PR counts may not mean the same thing across organizations.

The reported spending examples span individual subscriptions, API-based usage and company subsidies, making direct comparisons difficult. The author said company size did not track with spending in this small set. Some interviewed firms reportedly saw costs peak in June and July and later fall, while others faced rising bills. These observations reflect the workplaces contacted, not a market-wide cost index.

““Lots of traders are using Claude. Engineering is learning the importance of rate limiting for traders who accidentally run expensive queries without realizing it.””

— An engineer at the trading firm, quoted by The Pragmatic Engineer

What the Metrics Cannot Show

The report does not establish that AI tools caused the higher pull-request counts. It provides no controlled comparison separating AI use from changes in staffing, work mix, company practices, or other factors. Nor do the vendor figures establish whether the added PRs were accepted, maintained, or tied to better business outcomes.

The 27 workplace accounts are a limited, partly anonymized sample, and the source material does not give a complete methodology for selecting companies or calculating every spending figure. It also does not provide a common accounting method for subscriptions, API charges, and internal infrastructure. A comparable, independently verified ROI estimate remains unavailable in the report.

Companies Face Cost Tests

The report describes companies weighing spending controls, model choices, and the end of some subsidized subscriptions. Firms facing higher bills may need to decide whether to keep broad access, set tighter limits, or pay for infrastructure to run models internally. The examples do not establish which approach delivers the best results.

The next meaningful evidence will require companies to pair spending with outcomes beyond PR counts, including code quality, maintenance demands, delivery speed, and business impact. The report does not announce a follow-up study or a shared industry standard for those measures. Until comparable data emerge, the financial return on engineers’ AI spending remains unsettled.

Key Questions

What did the report find about AI spending?

In accounts from 27 workplaces, spending varied sharply, from a reported $150 monthly cap per engineer at one large company to $3,000 per developer monthly at a two-person startup. The examples are not a representative industry average.

Did AI make developers produce more pull requests?

CodeRabbit reported substantially higher PR counts among platform users than a year earlier. The figures show an increase in activity, but do not prove AI caused it.

Does more pull-request activity mean better productivity?

Not on its own. PR counts do not show whether changes were accepted, high-quality, useful to customers, or costly to maintain.

Do companies know whether AI tools pay for themselves?

The report says few companies it contacted could answer that question. It describes spending and output measures but does not provide a verified, comparable ROI calculation.

Source: rss