I don't think that one has held up very well. A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
UPDATE: I'm going to take this one back a a bit: it turns out Apollo publish articles like this on a daily basis, so the occasional headline that appeals to the "hungry for negative stories about AI" crowd is less of a pattern than if they published less frequently than that.
You’re implying that Apollo is biased, but the thing you’re quoting is a very brief report on publicly available research. This is the entirety of the article:
“The US Census Bureau conducts a biweekly survey of 1.2 million firms, and one question is whether a business has used AI tools such as machine learning, natural language processing, virtual agents or voice recognition to help produce goods or services in the past two weeks. Recent data by firm size shows that AI adoption has been declining among companies with more than 250 employees, see chart below.”
Two questions:
1) What does that chart show today? (Genuine question. I’m curious and don’t know where to find it.)
2) Regardless of what it shows now, why do you think Apollo reporting on it is noteworthy?
I'm implying they may be biased in what they chose to report on and how they frame it, yes. Deciding which figures to turn into a chart is an editorial decision.
> I'm implying they may be biased in what they chose to report on and how they frame it, yes. Deciding which figures to turn into a chart is an editorial decision.
I think the Apollo thing is a fairly dumb hot take based on a near-term wiggle in a graph, but it's hilarious that you are calling them biased.
Pot, kettle, black, etc.
Substantively, I don't think it's surprising at all to see those lines go up -- it would be shocking if they didn't go up, given the intensity of the hype. The slope is what matters. In particular, I wouldn't argue that your graph shows any kind of acceleration in adoption over time.
I’m surprised it’s so low, actually. AI is defined so broadly that it includes voice recognition and machine learning of any sort, and in any business function. It’s hard to draw conclusions from that. I wish they had kept it narrowed to LLMs specifically.
It’s too bad they changed the definition midstream, too. It would have been nice to see the two lines side by side.
> Of course Apollo is biased. They are a hedge fund.
They are not a hedge fund, they are a Private Equity/Private Credit fund, a financing firm and a venture fund all wrapped up in an alternative asset management company that does almost everything related to private credit.
For being one of the biggest AI boosters on HN, you'd really think you could make a stronger case than "Apollo is bad because they observed and reported on a fact". It wasn't a prediction, so there's nothing to "hold up very well".
Misunderstanding what actually qualifies as a "prediction" seems to be a recurring problem among AI commentators.
I've seen dozens of posts/top comments in the last few years dunking on "AI skeptics" for statements that didn't even vaguely resemble an actual prediction.
Like the front page post from Karpathy which got 600+ upvotes by throwing an AI model at old Hacker News comments without any human vetting to score "predictions". And upon further inspection included a bunch of unambiguously factual statements of the present or conditional if-then logic about the future which when glossed over for convenience become "failed" predictions.
we're not in the mysterious AI ocean, we're in the artificial swimming pool where the surfers and the guys selling wave-making machines are mostly the same people.
The person whose comment we're discussing literally has a sponsorship from an AI company on top of his blog that just raised 50 million bucks lol
I don’t think your judgment matters especially regarding bias because you’re selling content, you feed off of AI hype, you create nothing substantial (content-wise) but just trying to funnel traffic to your website. Sorry but you’re not at a place to talk about objectivity especially when it comes to AI related stuff.
I’m very familiar with your content. Commenting on the qualities of my judgement only comes off more like how dare you!? but chill not everyone has to find value in your work or text. If you really feel like you want to prove a point, share a post you find really original, that adds something we couldn’t find anywhere else. Relaying publicly available info with no authentic take or point is more like parroting to my taste so not something like that.
Your blog has been in my RSS feed for a long time, even before you became AI-pilled.
For at least a year or so, I’ve found very little substance in your writing. It’s mostly surface-level material, resembling the fluff we see on startup company blogs.
Even though I don’t use your relatively recent open-source projects, I respect the effort and contribution. But that’s not what I’m talking about. So, it’s not about your work in general.
Your blog and website content lack depth, originality, and sincerity. You’re trying to sell something, and I’m not a buyer. Still, I wouldn’t go so far as to say that people who buy your stuff have terrible judgment.
You cover AI extensively, but your style and apparent depth of knowledge on the subject (at least judging by your content) remind me of those “experts” invited onto news shows who have no solid credentials but have somehow figured out how to get airtime through social skills, connections, or networking. They present surface-level information riddled with logical and intellectual holes, yet manage to sound convincing to people with limited or no background in the subject.
> A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
This is kind of gross. Make the claim or not, don't do this pseudo-sealioning thing. Moreover, engage with the actual content (or you could just not engage at all!), not this shallowest possible dismissal.
eh sometimes you don't need any thorough dismissal. for all the "AI does make you productive" headlines, there's bound to be a few that goes the opposite way. Its just statistics, and these headlines get upvoted because of emotional reasons.
There's no way AI is going away, nor any way companies are going to use AI even less than they are today. Hell, even Linus Torvalds is using AI. At this point, there's no need for a serious rebuttal to these arguments.
A cynic might opine that an AI shill is not a great source of unbiased review of an article. The article is about a biweekly survey of companies and the trend was clearly dropping. This isn't about token use by tokenmaxxers increasing. And it isn't about the opinions of the folks you hang out with when you're flown to conferences to gush about AI. It's a survey of businesses. And aside from programming and math, AI sucks. Even in programming and math it's a tool with a lot of downsides.
I don’t think I’d judge someone offering a prediction the same way I’d judge an observation of what’s already happened. It’s implied that guesses about the future are uncertain in a way that statistics about the present are not.
Does for us (data science, specialty chemistry, finding/optimizing formulas). We can get from idea to prototype in a day compared to 1-2 weeks. All that boilerplate to process data took a long time to code by hand, especially for the data science/math guys who are not programmers. Now we can actually try all ideas instead of guessing the most promising and only try that.
And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
I don’t think anyone is disputing that it makes many kinds of work easier and/or faster, just the degree of those improvements and how much of those improvements actually translate to dollars earned or saved.
Seeing hard data on that would help settle the issue.
Unfortunately not everything will cleanly show up in PnL, which is a function of the entire company’s processes.
If developers are shipping more code (and most of the enablement is in software right now) does that show up in top line? Maybe… if sales and marketing are also selling more, and product people are building the right thing, and…
It’s not just code quality, it’s all the other business fundamentals like the product itself having value.
I see this in a very non-AI way working at a company that’s a hodgepodge of acquisitions. Some of our products just don’t make a lot of money, don’t resonate, or are behind the competition in various ways.
The amount of code or general engineering complexity of the application seems completely detached from profitability a lot of times.
Now the bar is going to be higher for software to actually be worth buying when it’s so easy to make your own for yourself, so that also means that our ability to pump out code loses value.
> The amount of code or general engineering complexity of the application seems completely detached from profitability a lot of times.
Of course, they're almost entirely unrelated. It's just programmers believe solving difficult problems with specifically code is valuable in of itself. The goal is to get as much value as possible from as little code & complexity as possible.
> btw, shipping more code isn't necessary a good thing.
This is FUD. It isn't necessarily a good or bad thing.
At the very least, using AI to audit your code is responsible use. The testing is the same as it ever was. If there was a chronic problem with shipped AI assisted code, it would be a popular talking point.
> If quality isn't controlled properly, it eventually shows up as negative in the top line.
This an idealistic pillar of capitalism that has been falsified across time. Depending on what someone means by "controlled properly", I don't think this wasn't been true before AI and I don't think it's demonstrated post-AI. Momentum (notably capture) keeps the software markets resistant to quality defects.
Usually, things that aren’t clearly correlated with profit tend to get ignored and canned by businesses.
Think security, optimization work, refactoring, etc. Today everybody is drunk with AI, but what happens when the hype is over and you can’t justify to the finance dept. why the company is spending X on tokens?
I know a very specific use case in management consulting where hours spent by a consultant for a repetitive task (mainly report writing and source data gathering) went from on average of 40 hours to 12 hours per project. It costs the firm roughly ~$150 in token spend to save those 28 hours. They used to be roughly $200/hr (so $5k~$6k).
You haven’t shared enough data to tell whether there was a financial payoff for the firm or not. The consultant is a fixed cost to the firm, so saving consultant time doesn’t necessarily translate directly to higher margins nor revenue per worker.
Are the consultants delivering more billable hours per person now?
All sorts of reasons why the answer to that may be: no.
The firm charges a fixed fee to the client. The client doesn't see anything different from what they got before, the firm increased their gross margin overnight.
Yes, assuming the consultants are salaried like most, you can increase gross margin while operating margin stays flat. The question still stands — did revenue per consultant increase for this firm?
Uhm, no. The OP makes specific claims wrt to AI and financial payoff; you offered up a case study, with incomplete data, as possible evidence that AI is generating these financial payoffs. I'm asking questions to uncover the data necessary to reconcile your case study against the actual claims from OP.
> Even if they were FTEs, it would mean the consultants could do ~3x the work.
Could. But again, are they? That's the whole point of Apollo's claims: people keep saying AI can or could increase capital efficiency, but it's not showing up in any actual financial results.
You're extrapolating a 3x increase in throughput from the optimization of a single process that exists in a whole business system. Given your background, I don't think I have to explain to you why that doesn't make sense.
Also, do we think the S&P 493 is so incompetent that they'd miss these optimizations? Or is it more likely that this firm you're referencing is a small inefficient player, and the S&P 493 already realized these efficiencies decades prior without AI?
> you offered up a case study, with incomplete data, as possible evidence that AI is generating these financial payoffs.
You're confusing the original article and the parent comment I replied to. I think the Apollo article is way off, BUT I have very intimate knowledge with very concrete numbers of a use case where AI has a very clear and definable ROI.
> Could. But again, are they?
YES. That's what I'm telling you. They just implemented the system. The contractors spend less time on each project which means they have more time to do another project. How is that so hard to understand?
> You're extrapolating a 3x increase in throughput from the optimization of a single process that exists in a whole business system.
Are you deliberately being obtuse? The company very clearly drives ~$5k additional profit per project. They do hundreds of similar projects.
> Also, do we think the S&P 493 is so incompetent that they'd miss these optimizations? Or is it more likely that this firm you're referencing is a small inefficient player, and the S&P 493 already realized these efficiencies decades prior without AI?
I never made any of these claims, why are you saying I did?
This assumes the client is oblivious to the fact that they are paying the same for 12 hours of work as they used to pay for 40 hours of work.
For non-technical clients that may well be the case for a while, but I suspect even the most non-technical clients will eventually realize they can claw back a good portion of the efficiency as discounts.
I'm negotiating some contracts where I could do the work in-house for about 1/20th of the previously-reasonable vendor cost. I'd rather not do it, so I'm happy to pay 1/20th plus a reasonable percentage above that, but that's the limit. The vendor can accept that, or goodbye.
Interesting times on vendor contract negotiations these days.
> I suspect even the most non-technical clients will eventually realize they can claw back a good portion of the efficiency as discounts.
If the outcome is the same then the fees shouldn't matter to the client in the long run. Said differently, if the client who pays the management consulting fee gets the same return out of the service, why do they care?
Now, what may end up happening is that some other management consultant vendor in the market comes in an says "ya know what we're willing to do the same work for less" (which means their margin goes down). This could indeed drag their pricing down.
> Interesting times on vendor contract negotiations these days.
> Said differently, if the client who pays the management consulting fee gets the same return out of the service, why do they care?
At least at every company where I've been in charge of my vendor contracts, I am always grilled by management on how we can lower the cost of the contract and whether we are getting maximum efficiency from each vendor. There is huge pressure every year to find ways to lower the cost.
I actually feel like it’s really premature for a relatively new technology to be showing profit improvements. We are talking on the order of a few years of investment and “figuring out” how to use it productively in business. Most new technologies take a decade or longer to really yield tangible value.
This is the first cycle in my (relatively) long career where the demand for return felt instant. The tools and techniques are almost comically immature and the average person interfacing with the tools is largely bewildered.
For word processing for instance people spent years upon years coming up to speed on wordstar, word perfect, word. Decades later they’re a common part of life and the tangible benefit over secretarial pools is obvious - there is no such thing as a secretarial pool.
AI it feels like almost a blink of an eye and the demand is “well, it’s been a few years, where’a the post scarcity society I was promised?”
The difference is that other new technologies didn't have a literal trillion dollars invested into them within the first few years, nor were the industry leaders warning that the tech could destroy humanity. Given those conditions I think "Has it made any money?" is a reasonable question to ask.
I think part of the issue is-if AI makes you twice as productive at X, that doesn’t necessarily result in any increased revenue, because now your customer expects twice as much for the same price, and if you don’t deliver that one of your competitors will.
A new technology can deliver significant increases in real economic value in terms of utility per $ expended without significantly increasing revenue for the average firm.
I agree. Being twice as fast as X might not help depending on how X impacts Y and Z.
As an example, my X is that boilerplate and general speed/effort of modifying my code is lower, but I’m noticing that this doesn’t necessarily make my team any more effective overall.
It seems like it’s been a code review burden and it’s led to some team members being too ambitious with picking up more projects that don’t make sense.
I have seen some managers in particular toss prototype work that they shouldn’t be doing at underlings, and one of those projects in particular really ended up taking me longer than if I had done it from scratch without AI. The fastest result would probably been if I did it from scratch with AI, but because I took it over from someone who didn’t understand the fundamentals and farmed out decisions to AI I had to spend a lot of time correcting mistakes.
Obviously, good management beats bad management all day long, but I bet you most companies don’t have stellar management.
Can’t share more than that publicly. Just that the time saved on this one thing alone is saving us way more money than the subscription per employee that is needed (~100$). I don’t doubt there are companies where this is not the case. Would like to share more details, but the rough math of doing something in a day vs several days + engineering salary have to do here.
Speaking annecdotally because im an engineer, not in sales:
Our company has secured projects and executed them to a standard which we would not have otherwise were it not for AI or hiring more people. Now, these are at least 3 year projects, so we are only seeing ~3 being AI accelerated.
For us, yes, it's a multiplier, directly translated to more $$.
With the advent of the internet and email, many people surely made similar observations. I wonder what the fate is of the companies who refused to accept these productivity enhancements.
Unfortunately we're at the mercy of Goodhart's law and in an interesting twist, AI is really good with things that have an immediate feedback mechanism, eg writing code that doesn't compile or pass tests, but you're trying to grade AI o something that doesn't have an immediate feedback mechanism. AI can slop out a PowerPoint deck in a fraction of the time it used to take a human by hand. The competent human then goes in and unslopifies it, but overall, still time is saved. Unfortunately, that employee is salaried and not hourly, so they can't click out early and even if they were, what's their incentive to clock out early if it means less money in their pocket? Or they can use the hours now free on something else. My gut, as an Internet random who's used AI is that it helps. I don't know how to measure it, just like how do you measure how much better it is to live in a clean house vs a cluttered one? How would you put that in a metric? If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.
> And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
Nobody even tries to measure this. Their CEOs and board members have AI shares, and potential investors need to see your AI story to justify your valuation. It's a prison of lies.
It's probably just another Red Queen's Race. The benefits of LLMs won't show up in higher profits for most companies because their competitors are also using the same tools to move faster. You can't opt out of the race or else you'll be left behind.
Internally the hard data is there. On a budget of $2000 of API-priced tokens (Claude models + GPT models) we were able to migrate ~6 teams of infrastructure in 1 month, done by 1-2 people. Without AI it would've taken 10x longer wall clock time to do, and involved far more folks, and probably would have been much bumpier of a rollout. The big lift was mostly using AI for massive information gathering, digging through huge volumes of discussions, documents, commits, PRs, alerts, dashboards, etc pretty exhaustively for the past 5 years of history, to discover all the "chestertons fences" far more thoroughly than even the folks on the team who built it were able to. Then the 1-2 folks could go to each team with a super comprehensive "here is every need, every interaction, every discussion, everything you've ever done w/r/t this infeastructure, and here's how we're going to handle each of those with the migration we're going to do. We will do all the work for you, is that acceptable, yes or no?"
The teams throughly reviewed and brought up 2 overlooked cases, but mostly we all reminisced about things forgotten that AI discovery had turned up which would have been totally overlooked otherwise. Forgotten discussions, designs, outages, odd decisions, weird edge cases prepared for but never encountered, etc. Then they were migrated, it went smoothly, all to save a low seven-figures per year dollar amount.
The fundamental approach of exhaustively researching to that degree would never have been possible without AI. It's just not feasible to analyze 5 years of even one of those discussions sources (e.g. slack) to figure that out, let alone ALL of them. Now a team of ~1.5 engineers has tackled all that by leveraging AI.
The hard data is piling up in companies, but it'll take a while to get out there.
(Numbers and details lightly greeked in the above account)
It can take a long time for the economic gain of new technology to be visible in GDP because people have to learn how to use it effectively and processes have to change to integrate it. It took about 40 years for factory floors to be fully reconfigured around distributed electric motors rather than central steam shafts.
Tons of unstructured data that needs to be cleaned up in so many different ways that are hard to predict. Knowledge distributed in the heads of greybeards, handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software. Which you have to repeatedly ask for and discover more piece by piece. Then clean up, discover issues, make the customer gather more data/correct it, etc.
No, but if we go from data to experiment faster we see earlier if something is wrong. Once we get data we quickly can fix the data pipeline and get results. If there are issues we can quickly show what/why and discuss it with them in detail and make changes based on their response faster.
Yes, but they tell things and maybe forget to mention certain things or contradict other data they have given you. Now we may not notice it initially until we make an experiment and get the results. Something is wrong and we have to find out what. Talk to the customer again. Fix experiment. Repeat. Faster experiments means the latency from new data to results is less and we finish the project faster. Errors become less expensive and cause less delays. People make errors and data quality is often horrible so the shorter iterations are nice.
It's not all boilerplate of course. But iteration cycle is shorter and the latency matters. We are trying find a formulation, so a lot of experimenting, fine-tuning, checking errors (first via simulations, then real-world with measurements). Then we see what needs to be changed, repeat until error is in desired bounds. But the real-world is messy so we see a problem (lets say adding some certain color does not change the final properties of paint the way predicted) and we go back to the customer, some expert remembers temperature/moisture of the air was different on this day and that affects printing results. Now we need to measure this too and add it to the simulation and add it to the data pipeline. But we may have to wait until the real world has implemented that too (create a process, buy hardware, whatever) and then a while later we finally get real data and can start doing our part. Reducing our latency makes things faster, as you may do something else in between and need to load everything in you head again. The LLM gives results in sometimes minutes: frontend, backend and database. We now have the feature branch where you can upload the data like the customer would do, start the calculation, see the predicted formula. From idea to working prototype. We can focus on making assumptions and checking them quickly. Previously they can make the changes in their Python scripts but getting it in a presentable form took much more work. Now we can go to the customer and present a pretty custom made UI like an end-user would see same day they give us new data or report issues.
The stack certainly could need improvement, but much of the overhead comes from the process being pretty serial because each step depends on the previous. If someone we needs to talk to goes on vacation, sick or has a full calendar this delays it much further. Iteration speed is not linear in such cases. We do a lot of custom solutions and have to implement custom parsers or interface with undocumented ancient software or simply no existing software. Everything is custom, because previously they have no processes, no IT infrastructure or simply only data intended for humans (roughly structured, informally written, ever changing naming schema, typos).
Oh yes. Verification needs to be done. Some statistics and validation is absolutely required. Also most customers don’t want their private data uploaded anywhere, so ai gets only test data and writes the script to transform it. As long as it is just XML, CSV or similar it’s pretty reliable. Handwritten notes and bad scans however…
A friend of mine runs a business similar to this for systems migration, which is a fairly tough and tedious "short-put" job (you have to know both old and new system fairly well, and if all goes well, you get the same job done on the new system, while if something goes wrong, you're in a world of pain...):
The AI analyses sample data from old and new system, and then builds a prototype deterministic mapping pipeline. The experts can then inspect said pipeline and with their domain expertise catch edge cases/exceptions/uncovered cases, fix them and improve things, and then you use the vetted deterministic pipeline for migration. Yet, the initial AI draft saves enormous time.
> handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software
This area is one which AI has shown to be pretty good (as long as some error rate is acceptable). Character recognition, voice transcribing, converting documents from one standard to an other. I can see an argument that if people spent most time doing those things, AI will speed it up (with a bit of added errors).
I am reminded however that in translations and AI generated subtitles, there is still a lot to be desired compared to the human touch. Handwritten papers in a different language, using local short hand and assumptions, may result in some odd and error prone AI transcription.
Boilerplate becomes burdensome as complexity increases. Perhaps sometimes we don’t give ourselves enough credit for how much applications and their infrastructure are actually taking care of as part of boilerplate.
I look at stuff like terraform code or AWS infrastructure and it’s annoying and repetitive as hell sometimes but at the same time it’s incredibly impressive that you can just describe something as complex as your own private company’s network existing on global data centers and on a relative level you can just bake in pretty complex stuff like a spiderweb of IAM rules, disaster recovery/backup/databases you barely have to think about, horizontal scaling, security policy and all kinds of other stuff I’m forgetting off the top of my head.
It’s heavy boilerplate because like you said you can basically design an infinitely complex corporation as code and once you’re done with that deploy it instantly which used to be pretty much unheard of.
And yet, many companies today are spinning up full stack teams, asking them to be experts from user interviewing to database design, architecture, hosting, devops, API management, data product, and also keep an eye on GDPR compliance.
I see dozens of teams that are doing cloud to the wazoo when really they need to just be told “we’ll load your routes from main.py, build your assets in /static, you need a DB click the button to provision it”.
It boggles the mind that so many teams are reinventing the wheel a million times over to publish their APIs, register AWS accounts, setup CI, decide on ECS vs Lamda, etc.
This sort of activity has been the source of income for a very large number of people. You can't really fault them for trying to extend that gravy train's run.
Probably a controversial take - but in the pursuit of elegance. 'Uhh, so I need to hook this connector up to that receiver using this adapter and then uh, I pass it to the connector manager which sends it to the scheduler that... @#%! FML.' And then it's all so polymorphicized that going through the debugger you spend an eternity just trying to figure out what the types even mean and why/where they came from.
When you look at old-school code that just hacked things together, it was ugly, but also very easy to in turn further hack. There was no elegance or grand base types with 80 levels of inheritance to enable clean extension at every possible juncture of types. And no spending forever to find out the exact function you want to be able to override, only to discover that it's outside your modifiable codebase and wasn't marked virtual. Everything just worked.
This reminds me of a nice quote about engineering: 'Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands.' Another is that 'If people knew how many assumptions went into designing a bridge, they’d never dare cross one.'
Engineering isn't about creating some perfect thing that is prepared for any conceivable scenario, but about solving a problem as efficiently as possible.
Well, that's a funny thing. The "well engineered" code can actually be harder to work on. And maintainability is one aspect of "well engineered".
What I've seen (at least at one place I worked) is that the code may be very well engineered for further modification - but only by people who have worked quite a bit on that code. For someone coming from outside, it's almost impossible. It takes forever to find your way around, and figure out where the magic place is that you need to modify in order to work within the existing design.
Well, you say, that's what the design documents are for! But they didn't have enough detail. They described the major objects, but not all the ancillary helper objects. They didn't describe the flow of data through the system in enough detail to know what to modify. The knowledge was there, in peoples' heads, but it was hard to get those peoples' time.
So, yeah. "Well engineered", but not engineered for maintenance by people who are not those already in the group.
The question is: how does any of that translate into more revenue? If you try more ideas and they mostly go nowhere that's not necessarily helpful to the bottom line.
If that were universally true in all domains then all the best and most reliable software in the world would be written in interpreted languages, because they have the fastest iteration speed. Sometimes rapid low-quality iteration is just spinning your wheels.
I'm not sure that is strictly true. Speed of iteration is increased by interpreted languages in the small, but when working on larger systems or on teams where collaboration is required they can have the opposite effect.
I also find that thinking through a design first helps me achieve a result faster. So I prefer languages that encourage you to think first, type later. Interpreted languages encourage you to type first, think later.
No goalpost is being moved here. Don't assume that an aphorism coined in one domain generalizes to other domains, especially when we have evidence to the contrary.
Not all experiments work out. If we find the right approach faster we can finish the project faster. Some projects are mostly fixed price or offer bonuses for reaching milestones in a certain time. Meaning we more often get a bonus and can take more projects. Also we can take projects with tighter deadlines that we would need to decline otherwise.
A lot in this world is not optimized by software yet. For our customers they have employees with decades of experience guessing things, maybe some software that kind of helps but still not good enough (slow, not finding really good formulas). They often have difficult problems and many failed projects. We try to find such customers and offer our service. If we are cheaper (+ show portfolio of successful projects) more are willing to give it another try with us.
Right, and while that might be working out for you right now, there is a finite budget that companies have for projects like that. The "zero financial payoff" framing sounds about right to me.
You can have a better solution, but customers don't normally pay for that and CEOs don't care or understand. AI improves what businesses don't care about.
AI is going to effectively let random people compete and displace big tech companies with free software they simply release. This is why google wants to lock android down, they see it coming. CEO led companies with layers of management and lots of investors are obsolete. We are going to go back to teams of 1-6 people doing all the work on their own like in the 90s.
It makes things viable that previously would just sit in a backlog for months or years because something else is always more important. On the other hand it can create messes that cause a growing backlogs. Can confirm that smaller teams are much more viable now. We are 5 people only.
I understand there's some benefit of this. But is it worth putting the global economy into a death spiral? So ManBeardPc can save 4-9 business days to go and say "yeah, that doesn't work".
There's a difference between the cost of coding and the cost of delivering and maintaining software? AI has demonstrated that these two things are almost entirely unrelated.
Almost everywhere I've worked, the "designing and writing code" part was about 5% to at most 10% of the calendar time that it takes to make and release a software product. The remaining 90-95% of the time is: aligning stakeholders, meeting with all contributing teams to prioritize, gaining approval from various executives to commence the project, obtaining security reviews, privacy reviews, legal reviews, and so on, soaking in dev, staging, and beta environments to find bugs, manual QA testing, aligning with marketing--potentially waiting on key external marketing events, waiting for various people to come back from vacation, waiting for Mike to answer his E-mail... These things take the vast majority of a software project's time and can't be automated by AI.
So, AI is making 5-10% of your project much faster. Whoopdie doo.
It’s interesting to me that you don’t understand how 90% of the things you listed can either benefit from automation or be made irrelevant because of automation.
The firms of 2031 won’t be doing business like the firms of 2021 but just using AI for coding or in a few places to accelerate the same workflows.
Has AI demonstrated that or have people using AI demonstrated their lack of ability to properly use it, as they are fumbling in the infancy of an new age?
This is a complete misrepresentation of what the Apollo piece can actually show, because it's not measuring the "payoff" of AI investments, but only the top level revenue of those companies (https://www.apollo.com/wealth/insights-news/insights/daily-s...).
This is a healthy corrective to the most insane AI maximalist takes (mass disemployment in late 2027, 10% GDP growth in a year), but it shows nothing about whether the actual economic impact of AI spending is.
First, to show that, you'd have to justify the counterfactuals (if those companies hadn't spent on AI, would their position be better or worse?)
Second, while AI spending is increasing rapidly, normal companies aren't spending 1% of their revenue on it. So how can you look at a noisy chart of revenue and expect that "oh, it's not up, the AI spending must be useless?" Those companies could be spending < 1% of their revenue on AI, deriving value from it, and it wouldn't be guaranteed to show in a chart like this.
I saw something like the average anthropic employee thinks there will be 5% GDP growth because of AI, which is totally insane when a 1-2% extra growth will be an incredibly good result.
I suspect there is a lot of demand destruction that AI will end up causing, so many companies are running with inefficiencies that it's really easy to ferret out and fix with agents.
I wouldn't be surprised to see a 5% GDP drop with all the wasteful spend that will end up getting deleted.
I think you're mistaken. The US uses the expenditure approach to GDP, so (I believe) it doesn't matter if it's a LLM or an engineer doing the work to generate revenue.
(Of course this breaks down heavily if the human buying power stops constantly rising and there's de-facto less demand for goods and services)
> (Of course this breaks down heavily if the human buying power stops constantly rising and there's de-facto less demand for goods and services)
This is what I'm talking about. I can easily envision waves of layoffs in areas like the service industry that historically have been highly resistant to productivity increases in the past. Hell, there's a whole industry based on arguing with insurance companies over billing that could potentially evaporate overnight[1]. That alone is hundreds of billions of dollars, and that's just one example.
Yes, in fact I heard somewhere that this was part of why the Obama administration chickened out on pushing for single payer when it was in reach. The broken window industry is a load-bearing pillar of the economy.
> During debate on the Affordable Care Act (ACA), as the crucial 60th vote needed to pass the legislation, his opposition to the public health insurance option was critical to its removal from the resulting bill signed by President Barack Obama.
Agreed and zero financial payoff doesn't mean no benefit. It may be that the quality of work improves, as we saw in the transition from the mimeograph to photocopiers and ultimately laser printers powered by desktop publishing software
Your company uses AI and it gets ahead a lot quicker. However, your competition does so as well.
Many areas of economy are zero sum game - so your $ ends up being the same as previously. However, now with a "tax" paid to AI model providers for being able to stay competetive.
Warren Buffett once gave a great answer to a similar question, but it was about the textile industry. He said technological advances that promise to reduce waste by x% or improve efficiency by Y% or boost production by W% or reduce cost by Z% rarely actually pay off. They would get textile machines manufacturers promising them great improvements in the new line of machines, and for the most part, those improvements would be realized, but their end margins as a business wouldn’t improve. He said it’s simply because the same technological edge is given to all their competitors as well. With no moat (his famously favorite metric) all textile makers were always competing on the same grounds regardless of the technological progress.
Maybe moat is also just product quality? I.e. having the "taste" or knowledge of what the consumer actually wants, and the ability to apply it correctly, can also allow you to win. Because right now some of the AI industry wins on Moat (mostly hardware - both in terms of ownership and production know-how), while some wins on perceived quality being higher regardless of price (e.g. Opus 5.5 dominates from what I can see despite Deepseek Flash V4.1 existing). People will choose the best product, not just the cheapest, unless the quality loss is okay in a given context.
Well, the net growth is taken by the AI/tool providers, who indeed bring productivity growth, so the society as a whole see improvements. I guess this is not bad?
Yeah but if that was true we'd see better or cheaper products at least, or new competitors rising. If it increases the speed for everyone, but there's no visible change for the end user, I wonder if AI makes you really more competitive in the first place.
If you are a company and can't figure out how to improve your bottom line through increased per person productvity, sales, product development, even just responses to customers etc, then that seems kind of like a "you" problem and not problem of the AI models...
I dunno: this is pure anecdata but software engineers I know well - including me - all say they’re delivering multiple times faster, so it’s definitely good for something.
I don’t know about other tasks, but it’s pretty rad for coding, testing, DevOps, and config.
This research is specifically NOT about tech companies. It is about the rest of the fortune 500.
If you are in software or IT at a non-tech company that is in the fortune 500, the impact of significantly better productivity may not be measurable in aggregate in the company's P&L.
In smaller companies I expect it could be more measurable because some of them really are using AI much more effectively across many functions. But we're finding it very hard to measure meaningful outcomes.
But actually 40% of the S&P 500 are tech companies, and they are saying only the mag 7 are showing clear revenue increase. Would would suggest that even in most tech we are not seeming measurable revenue growth, right?
One plausible explanations that has occurred to me is this: While programmers etc. can produce way more code with LLMs, the declines in quality and coherence of the work, and a widening gap with a coherent strategy and aim for value means the increased activity is mostly not valuable. I suspect this may just be an intensification of https://en.wikipedia.org/wiki/Productivity_paradox#Little_pr...
The mag 7 are getting direct or nearly direct revenue from AI inference. New money is pouring into that - it is not zero sum. Everyone else is getting internal productivity lifts that are matched by competition, and when they are deploying inference features they are usually ones their customers didn’t ask for and don’t value, OR that they expect everyone to have and are not a differentiator.
I think you are also correct that a lot of value is being wasted with projects that are developed impressively fast but that provide little value. It could be those are simply surplus personnel trying to add value or make people think they do.
> Everyone else is getting internal productivity lifts that are matched by competition
The main metric analyzed in the note is
> Revenue per worker
This can increase regardless of competition, but I guess would require either expansion into new domains or layoffs. So maybe we will see this raise as a lagging indicator if the LLMs are actually delivering the goods.
> when they are deploying inference features they are usually ones their customers didn’t ask for and don’t value, OR that they expect everyone to have and are not a differentiator
I am definitely seeing lots of this happening. Tons of energy going into side quests that aren't critical for anything. Meanwhile, strangely, the central features we actually need to ship, but that require deep redesigns and careful refactors in legacy code, are not moving quickly at all.
In case you are not being sarcastic, layoffs obviously don't "cause" productivity increase. [Edit: but they would help the main metric (Revenue per worker) analyzed in the note. so perhaps that is what we will see.]
Not sarcastic, it's how productivity is usually measured when you see any kind of stats. Not features, not lines of code. Revenue (or GDP if we talk about the country) per worker
My employer - one of the Fortune 500 - has been on the "everything AI" train. The fun part is that the same management which harps on measurable outcomes when asked about AI adoption have the same "AI boosts productivity".
They even brought in McKinsey who I believed, if nothing else, can spin up a good yarn. But even they couldn't create a metric based assessment other than "productivity gains".
Now I am not saying there is no productivity gains. Surely there is but I am always curious what metric people are using to measure these gains.
So if you are delivering multiple times faster I have to ask - How much time are you saving and how many employees worth of work you are doing now? 2-3-5 headcount worth of work? If yes, how do you calculate those numbers? And lastly would you put your money where you mouth is and if your team is lets say 10 person team and the company asks to reduce it down to say 2-4-5 people - would you take that bet confidently to continue with the same pace and produce quality work?
A friend of mine works in fashion. Very briefly for background: there are runway models (who walk on live shows), and catalogue models (who work in studios presenting clothes against a neutral background in standard poses, more or less as living mannequins). The latter are being increasingly replaced by AI.
For starters, because an average of 493 companies of all types and cultures tells you very little about whether AI is having an impact. For all we know, half these companies have implemented AI and are growing like crazy, and the half that haven't are losing revenue at a rapid clip, but it all averages out to zero.
One weird trick that devs learn to deliver faster: simply just drop all guard rails and push directly to prod. We need to make sure our velocity report looks good after all.
> all say they’re delivering multiple times faster
Are you measuring the right metric?
This feels like an on going lack-of-understanding in the industry.
In manufacturing, economies of scale and automation work because you're physically making more objects and objects inherently have value.
For software, making more software doesn't mean you can sell more units of software. The unit price of selling existing software is already 0. Economies of scale do not exist.
The OP says: net business output is not up as a result of AI.
So:
…if your team is doing more work and being more productive…
That just means the productivity of your team (or software teams in general) maybe doesn't have a meaningful business impact.
You should be worried if so. That means productivity = work force reduction (reduce fixed costs for the business).
I'm not sure I'm any faster, but I think the end result is better and I've attempted some big refactors and new architectures that I would never have tried without AI.
N=1 but I'm not sure it's made my bosses any richer. What we're shipping has better tests and documentation than it ever would have before AI.
Most, not all, are just delivering more bad stuff faster. I personally also think I deliver more good stuff with AI, but the products I use are just getting worse daily it feels like. Just the other day I noticed Slack had installed some disgusting menu bar icon for their AI bot, without asking, just to take a fresh example of this.
I've never heard any such claims but meanwhile in my actual job I'm seeing unbelievable productivity gains (in terms of actual cash spent before and after vs the same kind of work being done).
They are not wrong, they're lying. Very important difference. There are a lot of people who will say "I'm getting mega productivity boosts" who aren't getting financial returns from you believing them; those people are wrong.
there is increase in productivity of both types of employees: good actors who want to deliver on quality of work, and bad actors, who now 20x weaponized in generating all kind of slop. Catching second type is now very nontrivial problem.
I'm sure they can just fire half and double up work or outsource. That's the traditional way and it definitely works. The worst that ever happens is a few will reneg on payroll. Everyone who still chooses to work there is happy enough.
If you are an AI company with hundreds of billions at your fingertip, and looking for $2 trillion valuation, and can't figure out how to let your customers execute _on the exact thing you are selling them and basing your valuation on_, sounds like a "you" problem too right?
When it's the right tool, it's a silver bullet. When it's the wrong tool...or not the right tool yet, you can spend more time steering it than you would doing it yourself.
I'm seeing another behavior in my purely personal research...asking AI 'can I do the thing?' and it comes back with 18 pages of 'yes, and here's how you do it'. Often it's the discovery that produces the endorphin bump. It's cool to know I can jailbreak the Peloton...and now that I know I can...I'm not so motivated to actually do the thing.
I suspect the reason is the industry just does not want engineers to be mostly technical. There's a real tradeoff to AI, and I think companies are taking the wrong lesson and thinking "well now that programming is so easy, engineers need to do more of XYZ", when I think it should be "this tool is powerful but also takes more focus so engineers should do less of XYZ".
I’m curious, what are some examples of XYZ? What does “mostly technical” mean to you, and why do you think it is the industry doesn’t want that? Just trying to understand, because I don’t think I’ve experienced this myself.
I'm talking about the day to day work. At least in my experience at several big tech companies, the more senior an engineer is, the more you need to delegate work and are expect to lead projects by managing them. There's technical knowledge involved of course but it's a small part of it, mostly directing other more junior engineers.
AI allows you to move faster, but it also demands that you pay more attention in proportion to how fast you want to move. This requires more focus, i.e. more time developing. Effectively the structure at companies should swing back to how it used to be a few decades ago (I think, maybe I'm wrong and it's always been like this), where engineers mostly focused on delivering code. The difference now is that they can be expected to deliver much more code at higher quality.
To summarize, using a little AI doesn't pay off, you need to be doing that all day, and that is not how engineers spend their time in most of the industry anymore.
Thanks for clarifying. There have always been technical leads and hands-on engineering managers, but two things have changed:
1. The balance of responsibilities attributed to different titles has varied over time and across different companies.
2. Early personal computers were so resource limited that there were only so many programs you could write that actually made effective use of large teams. Today we can write much larger programs, which requires more coordination (across people and with AI—which can reduce the number of people required).
I would also strongly push back on the assertions that being very technical means spending most of your time on the code, and that you have to be using AI all day for it to be useful. Let’s say I’m a technical lead or a manager reviewing a design doc for a large project. I have an idea based on my years of technical involvement for how to significantly improve the quality or delivery timeline for the project. I used to have to ask the doc author to look into this approach, not knowing for sure if it would directly translate. Now I can take my crazy idea, quickly dispatch a coding agent to try it out against the codebase, go to a meeting, and then check on what it came up with to give a much more informed and technically accurate suggestion on the doc. I still have to have the technical expertise to iterate a bit with the agent and not over or under state the possible impact of my idea. But this lets me take say an hour rather than a week to come up with a vetted recommendation that saves months on the schedule.
I'm just speculating why the current structure of companies is not maximizing what AI can offer. Until it becomes completely self-directed and reliable, using a tool that allows you to move fast requires more technical knowledge, not less. What you're missing from your example is releasing to production without breaking it, dealing with increasing complexity as more features are added, maintaining development pace as more features are added, and trying not to create more bugs than are fixed.
The idea is more or less benefiting from doing work in bulk which AI is now allowing. I even think multiple engineers (human redundancy to double check the same code) should team up with one AI system and form a small team that handles millions of lines of code, with the only job to feed incoming tickets and check the AI work. Anyone not reviewing code should be facilitating everything else with their own AIs. I think large companies can collapse entire engineering departments into a few people. The cost would be that everyone else has to think like an engineer to sort out requirements and provide direction, but as you point out, this is now easier with the help of AI.
IMO, a large amount of work, including a large amount of technical work is wasted effort. At startups you see it when they go out of business, at a big companies you see it as bloat. So unless AI is directed at finding the right problems to solve, past some point engineering efficiency via AI at best saves some money and at worst means more ultimately low, zero or negative value work gets done.
Maybe this is why you’re seeing more effort shifted away from technical work. Maybe I’m a nihilist. Maybe both?
That was true when implementation was expensive. Now that implementation is cheap, doing it in bulk is the best way to maximize AI usage, and that means fewer engineers doing more of the technical work.
> unless AI is directed at finding the right problems to solve
This direction can be done by non-engineers with AI to consult (replacing that part of human engineer's role).
My guess is in the end we’ll find the approach that works depends more on the personalities of the people involved as opposed to one approach being strictly better.
Actually, there is one other important thing that has changed: as developing a certain amount of functionality has gotten easier with modern languages, tools, libraries and coding agents, it’s also gotten less interesting for technology-driven people and more accessible to career-driven people.
Implementing the first text edit field in your program is kind of interesting when all you have is asm and 4k of RAM. Nobody wants to bother doing it manually when you have a web browser that can eat gigs of RAM.
This has been happening before AI, but AI has turned it up to 11. The way I usually put it, senior engineers are the new engineering manager, engineering managers are the new product managers, and product managers are now something in between sales, consultants or social media contacts. AI made senior engineers pick up more product management from their engineering managers, and engineering managers are picking up more sales and social engagement, and product managers are in a limbo but the young ones are pushed to junior engineer roles.
In large companies there is also the politics game. The more freed time senior engineering manager and senior product managers get, the more politics intensives and waste more of their time.
I use the term “product management” in its traditional sense of establishing scope, feature prioritization, user experience, shape of integrations, etc.
I’ve worked most of my career without dedicated product management support (very early startups and internal infrastructure teams at large companies), having to do that myself (both as an individual contributor engineer and as a very technical manager). So I have to wonder how much has the industry changed, and how much has your location within the industry drifted?
Yes my view mostly from within large engineering organizations. And yes, what you describe is pretty typical for engineers (and especially founding engineers) in young startups. Doubly so if it’s a startup with a technical product. Large organizations always envied startups’ efficiency and pace because of that, which is why the traditional sense of product management has been handed to engineering teams. This has been happening in the enterprise side for a long over the last 15 years and predates AI as I mentioned. It’s just more collapsing of disciplines now which is the answer I was giving to OP’s question.
First, if your company could double its workforce for free right now with 0 plans for how to convert the extra bodies into revenue, how long would it take them to see a return? Because from what I understand companies aren't leaving money on the table, if hiring someone would make them more money they just would. AI could help those that weren't maximizing revenue despite having high demand and a market because they were resource constrained. That doesn't describe very many large companies.
Second, I'm getting huge value from it, but I'm not making any money and none of it is being redirected. It's simply value that wouldn't have existed otherwise because I would have just gone without the thing because it was prohibitively expensive or too much work. I'm quite sure many companies and people have been unlocked in this way. Plenty of value locked up in an overly expensive economy dedicated to maximum value extraction instead of value creation.
While they are seeing some individual productivity benefit, a lot of that doesn't really impact the bottom line directly and all their competition are using the same tools.
What I'd really want to see is revenue and profit per employee over time tracked against token usage/spend, and not limited to the S&P 493.
Apollo's own argument that most AI spend is coming from non-S&P 493 (as per the last two charts) tells me it's hard to make any conclusions based on these charts.
Conjecture supported by allegories that break down quickly. Arguments conflating the dot-com boom with the current AI boom even though the economics are wildly different. Dismissive arguments conflating anti-AI constituents with people who were against cars, etc...
AI is highly divisive. It's dramatically reduced the value of code, which has derailed a lot of careers who have built their entire adult life around the idea that code would remain valuable, while simultaneously driving others into a psychosis who believe that AI can do anything.
Excellently put. IMO the worst/scariest thing about AI is what it has done to the methods and work habits of some of the people who I've always looked up to as good software engineers. They now earnestly say things like "We don't have to look at code, understand it, or review it anymore. AI does all that." and "We don't have to test our software because if a bug is found in prod, you just tell the AI to fix it and deploy it instantly." Decades of software best practices thrown out the window in a year because of this psychosis!
> It's dramatically reduced the value of code, which has derailed a lot of careers who have built their entire adult life around the idea that code would remain valuable
I just don't relate to this sentiment. I wasn’t the fastest typer or the quickest at designing a solution pre-AI, but I was able to learn, build, and maintain systems. My value wasn’t in how fast or how much code I could write, it was in my ability to understand what the hell is going on and do something with that knowledge.
That’s still the case now, even though Kent from Accounting could theoretically one shot some of the software I build. He wouldn’t know what to do with the software, and he wouldn’t want it because now it’s his problem.
The tech industry oversees invisible Rube Goldberg’s machines of flowing electrons. Most people don’t want to learn or deal with its issues. And since AI is yet another complication piled on top of the machine, I feel confident that my career will still have value.
It _is_ about judgment. The challenge is figuring out what defines “good judgment” and how to capture those elements in a format that an agent can assess continuously and at multiple levels.
The problem is most people don't know what makes them successful from a people perspective, except in the coarsest terms.
90% of the business outcomes are delivered by 10% of the people. The problem is they don't know which people. Therefore they also don't know which processes, if automated or augmented with AI, will achieve the target outcomes.
So instead they’re shotgunning. The problem is that the existing people-based system has built-in stabilizing mechanisms to prevent misalignment from taking the entire operation off course. AI stresses and often breaks those.
The result? Higher costs and chaos.
This needs to be treated like an org transformation. Traditionally that's most successfully done by starting a new small team that builds ground-up and eats the old team.
My suspicion is that this is because AI employee-type systems like Dots or Grokbot just haven't been adopted yet. I think once they're good enough, that's going to be the catalyst for a lot of these companies to start adopting and using AI productively.
Agree. Just natural language business goal prompts using an AI assistant paired to a frontier model thats is in always-on mode. The vendor race will be first mover advantage to create customer lock-in.
That’s an interesting one. On a personal level, there are clear benefits. On a company level, there’s very little visible. Draw the ratio of operating profit to employee count over the past 10 years and there’s no change there you could clearly attribute to AI. It would seem the engineers are an important part of a machine, but not _the_ machine: there’s other bottlenecks that hide or consume personal-level gains.
>AI hasn’t benefited the bottom line of any big corporations outside the tech industry
If you get two companies that reduce their costs 50% but due to competition have to pass on their savings by reducing prices, the profits may remain the same even if AI works.
The company I work for insists on sending a crew from Engineering, Product, and Sales out to a customer site as part of our sales process. I'm personally much more productive using AI, but that only goes so far when closing a deal still follows a process stuck in the 50s.
It makes sense to me and I'm saying this as a strong believer in AI and its ability to add value.
The system itself doesn't reward value creation. I've been saying this for years. I've seen many smart people waste their time on bureaucracy; their intelligence was being wasted; so I'm not surprised at all that now that we have AI, that its intelligence is being wasted as well.
The bottleneck is an allocation issue, not a capability issue.
The current reality is that a lot of very bright people are currently working for big tech in bureaucratic roles and not allowed to focus their attention on 'risky' problems which would deliver real value if resolved.
The current system prioritizes monopolization and is designed to try to ensure that all innovation happens within big tech. It is sacrificing potential innovation which could have occurred much faster outside of big tech in order to ensure that any gains are captured by big tech... But it's sacrificing an enormous about of efficiency.
I hope big AI companies can use their political leverage to try to open up opportunities for people who want to innovate outside of big tech. That's aligned with their long term interests.
I think AI companies are convenient structures which could allow big funds to benefit from decentralized innovation. Could create real win-win scenarios. But those AI companies have to stop pursuing the corporate and government money and prioritize small customers and support small tool makers (or at least lobby the government to stop hindering them with regulations). Also, bootstrapping should be prioritized because VC money mostly creates moats.
It depends on what you mean by failure. Economists tried to measure the gains in productivity from the rollout of computing, Internet, etc, across the economy, and couldn't actually find any.
Taking a step back wouldn't this be exactly the expected outcome for an emergent technology in a capitalist framework?
1. Execs with no understanding how business operates on the low level are pushing buzzwords.
2. People who want to get promoted deliver on buzzwords and get promoted. Note that the goal is to deliver the buzzword, not use the buzzword to make the business better.
3. Buzzwords do not help vast majority of companies because the incentives and needs are misaligned.
4. Eventually buzzwords settle, some find niches, some die off, and some take over the world.
Last two decades alone - big data, blockchain, crypto, web 3.0, gamification, VR, and now AI. TL:DR - we should expect vast majority of early AI adopters to have net negative to neutral experience. Assuming that AI is like electricity and automobiles more than blockchain we should expect that eventually companies that find a way to apply the tech correctly will eat those who do not.
Am I the only one even mildly suspicious that the AI/datacenter bubble is not being driven by actual demand but rather by a rogue AI that wants to evolve?
Interesting idea! There are many behaviors explained by parasites, worms, bacteria and viruses. AI is now smart enough to change personality, make people invest trillions of dollars with zero ROI.
You're being downvoted, I assume because people think this is an outlandish claim. I don't disagree with them, but I upvoted because it's a totally unique and entertaining thought!
What a wild scenario to imagine. Anthropic and OAI spending wildly, analysts astonished, thinking we're staring down the barrel of an economic crash. And in the meantime some rogue AI is laughing maniacally and using blackmail to control Altman and Amodei like puppets.
Give it time. The Great Reconfiguration will take a great deal of time (years, decades). As it did with the printing press, electricity, computers, internet, etc. And this one (AI) is more fundamental than all the others.
The real flaw is the corporate structure. We don't need CEOs or execs that manage company direction. These people are useless due to AI.
The people who know how to get AI to make usable and viable solutions are the only people who should be running anything and their employees should be all doing the same.
A company should be built around people who can vibe engineer real end user products that are deploy-able and high quality.
Non-coding people are simply done now that AI can replace all of their "ideas", but cannot replace the engineer making the real decisions with the ideas that actually matter.
I talk to a lot of non-tech friends and sometimes ask about AI. Most of them still use ChatGPT like it is a better Google. Most don't know it can work well with documents, PDFs, Slides, Sheets, or do advanced analytics, etc.
Much less do they know the thing can build software and maintain it, manage complex tasks daily, automate things, etc.
Then there's a handful that do follow closely AI developments and agents and who have started using them.
A friend of mine this weekend who is an award winning photographer and who has been selling photography courses online forever has told me he's used Muse to:
1) Automate chopping up his long form videos, add captions, and post them as Reels to Instagram. Several of these have gotten over a million views.
2) Automate handling DM reachouts, converting them into sales. He's up to several tens of thousands of dollars in sales from AI direct message conversions that he would otherwise not have been able to convert (too many DMs to handle, he'd have to hire/train sales people to basically just do DMs full time at scale otherwise and it would make no economical sense)
3) Automate all his customer service for handling any issues, refunds, support, etc.
And I personally think he's still just scratching the surface of what he could do.
I feel we are really at the beginning of a new era. When the internet started most people were just using it for e-mail and news, maybe a newsletter chain here and there and some early forums. Social Media didn't appear for well over a decade, popular Web 2.0 apps like Google Maps, GMail, YouTube, Netflix, etc didn't appear for a long time.
Can’t help that many people who are good at leveraging AI to build things sit around one upping each other on twitter all day instead of going out there and building cool stuff for less technical people.
Isn’t this just selection bias - you don’t see the people who are out making AI deliver boring value for people unless you use their products right?
I.e. Ramp does a pretty good job of using AI to make expense management easier/ CloudFlare makes it a lot easier to configure stuff properly on CloudFlare but is just like boringly useful.
Well duh look no further than this site itself. "SUPER" AGI that can solve a millenial problems, and the top voted thing we can do with it is take the ads away from netflix-tier content distribitors, basically piracy.
I just finished a $23k engineering job in 3 months, entirely solo, that would have taken a year without these tools. That’s real money to feed my kids and working tech that made a client happy.
That’s one job and I have several in the pipe.
The “financial payoff” for no more than a few 100 a month at absolute most is there and it’s real. If you aren’t seeing it then you’re doing it wrong.
I usually don’t even spend more than $20 a month each for chpt and claude, deepseek free is solid. I am about to switch fully local too so that should drop to basically cost of watts.
AI hasn’t benefited the bottom line of any big corporations outside the tech industry
I am not an "AI booster", it seems like the current market leaders have dug a hole that they'll have a hard time getting out of. Just about everyone says something like "there's a bubble, it will burst, people will be hurt but the process will go on, like the Internet or railroads". But that ignore that the scale of this is beyond earlier bubbles (each bubble is larger than the last in our 30-40 year bubble-and-pop driven economy but that might not be comforting).
All that said, "some aren't benefiting" or "the average isn't benefiting" is a disingenuous argument if you want to say things are bad. AI companies have more of a real product people are paying than Google did. A large minority of companies (not all in the tech industry) are getting a boost. It's easy to say "the average is zero" when you exclude some. AI has boosted media companies that do a lot of transcription. Video generation serves half of advertising videos.
Reg: "All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health... what have the Romans ever done for us?"
Attendee: "Brought peace?"
Reg:"Oh, peace? Shut up!"
I spent almost 14 years at an S&P 500 company. I think it largely comes down to where the bottleneck is.
In some organizations, it doesn't matter whether you produce 1x or 10x as much work. If the process says reviews happen on Mondays and deployment decisions are made at the end of the month, getting your work done faster won't change much.
I'm at a much smaller company now, and we've doubled our feature delivery rate while increasing code coverage and overall product quality. We've also tripled our marketing content output, which was an area where we'd been falling behind.
AI can significantly improve individual productivity, but whether that translates into business results depends on how the organization operates.
The same Apollo Global Research partner, Torsten Slok, also published this widely cited story in September 2025 last year, "AI Adoption Rate Trending Down for Large Companies": https://www.apolloacademy.com/ai-adoption-rate-trending-down...
I don't think that one has held up very well. A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
UPDATE: I'm going to take this one back a a bit: it turns out Apollo publish articles like this on a daily basis, so the occasional headline that appeals to the "hungry for negative stories about AI" crowd is less of a pattern than if they published less frequently than that.
You’re implying that Apollo is biased, but the thing you’re quoting is a very brief report on publicly available research. This is the entirety of the article:
“The US Census Bureau conducts a biweekly survey of 1.2 million firms, and one question is whether a business has used AI tools such as machine learning, natural language processing, virtual agents or voice recognition to help produce goods or services in the past two weeks. Recent data by firm size shows that AI adoption has been declining among companies with more than 250 employees, see chart below.”
Two questions:
1) What does that chart show today? (Genuine question. I’m curious and don’t know where to find it.)
2) Regardless of what it shows now, why do you think Apollo reporting on it is noteworthy?
I'm implying they may be biased in what they chose to report on and how they frame it, yes. Deciding which figures to turn into a chart is an editorial decision.
I'm writing up a new version of the chart at the moment, but here's a preview: https://static.simonwillison.net/static/2026-11-01/5821A414-...
Notes on where to find the data here: https://simonwillison.net/2025/Sep/9/apollo-ai-adoption/
Here's that write-up: https://simonwillison.net/2026/Nov/2/census-bureau-super-int...
> I'm implying they may be biased in what they chose to report on and how they frame it, yes. Deciding which figures to turn into a chart is an editorial decision.
I think the Apollo thing is a fairly dumb hot take based on a near-term wiggle in a graph, but it's hilarious that you are calling them biased.
Pot, kettle, black, etc.
Substantively, I don't think it's surprising at all to see those lines go up -- it would be shocking if they didn't go up, given the intensity of the hype. The slope is what matters. In particular, I wouldn't argue that your graph shows any kind of acceleration in adoption over time.
I’m surprised it’s so low, actually. AI is defined so broadly that it includes voice recognition and machine learning of any sort, and in any business function. It’s hard to draw conclusions from that. I wish they had kept it narrowed to LLMs specifically.
It’s too bad they changed the definition midstream, too. It would have been nice to see the two lines side by side.
(Thanks, simonw, for the updated chart.)
Thanks for "a fairly dumb hot take based on a near-term wiggle in a graph", that's a much better way of expressing what I was trying to convey here.
Of course Apollo is biased. They are a hedge fund. They are biased towards their interpretation of the market.
In fact, they often put forward a hypothesis about a company or market and argue for it. I couldn’t think of anything less biased.
> Of course Apollo is biased. They are a hedge fund.
They are not a hedge fund, they are a Private Equity/Private Credit fund, a financing firm and a venture fund all wrapped up in an alternative asset management company that does almost everything related to private credit.
That’s not bias if it’s more explanatory of the future.
Apollo has the ability to predict the future?
I don't know if it's "bias", exactly. This particular guy has been wrong on AI recently.
For being one of the biggest AI boosters on HN, you'd really think you could make a stronger case than "Apollo is bad because they observed and reported on a fact". It wasn't a prediction, so there's nothing to "hold up very well".
Misunderstanding what actually qualifies as a "prediction" seems to be a recurring problem among AI commentators.
I've seen dozens of posts/top comments in the last few years dunking on "AI skeptics" for statements that didn't even vaguely resemble an actual prediction.
Like the front page post from Karpathy which got 600+ upvotes by throwing an AI model at old Hacker News comments without any human vetting to score "predictions". And upon further inspection included a bunch of unambiguously factual statements of the present or conditional if-then logic about the future which when glossed over for convenience become "failed" predictions.
I thought the scoring on that site was by human visitors, not AI?
A surfer is not a wave booster, this is a poor characterization.
we're not in the mysterious AI ocean, we're in the artificial swimming pool where the surfers and the guys selling wave-making machines are mostly the same people.
The person whose comment we're discussing literally has a sponsorship from an AI company on top of his blog that just raised 50 million bucks lol
I'm judging them here based on their chosen headlines.
I don’t think your judgment matters especially regarding bias because you’re selling content, you feed off of AI hype, you create nothing substantial (content-wise) but just trying to funnel traffic to your website. Sorry but you’re not at a place to talk about objectivity especially when it comes to AI related stuff.
Nothing substantial, really?
I think you are either unfamiliar with my work or you have terrible judgement.
I’m very familiar with your content. Commenting on the qualities of my judgement only comes off more like how dare you!? but chill not everyone has to find value in your work or text. If you really feel like you want to prove a point, share a post you find really original, that adds something we couldn’t find anywhere else. Relaying publicly available info with no authentic take or point is more like parroting to my taste so not something like that.
Your blog has been in my RSS feed for a long time, even before you became AI-pilled.
For at least a year or so, I’ve found very little substance in your writing. It’s mostly surface-level material, resembling the fluff we see on startup company blogs.
Even though I don’t use your relatively recent open-source projects, I respect the effort and contribution. But that’s not what I’m talking about. So, it’s not about your work in general.
Your blog and website content lack depth, originality, and sincerity. You’re trying to sell something, and I’m not a buyer. Still, I wouldn’t go so far as to say that people who buy your stuff have terrible judgment.
You cover AI extensively, but your style and apparent depth of knowledge on the subject (at least judging by your content) remind me of those “experts” invited onto news shows who have no solid credentials but have somehow figured out how to get airtime through social skills, connections, or networking. They present surface-level information riddled with logical and intellectual holes, yet manage to sound convincing to people with limited or no background in the subject.
Anyway, good luck with your content machine.
> If you really feel like you want to prove a point, share a post you find really original, that adds something we couldn’t find anywhere else
https://simonwillison.net/2026/Nov/3/qwen-can-interpret-mode... from this morning was the first report I've seen of Qwen 3.8 27B doing that weights interpretation trick.
Is it possible you're confusing my link blog with my longer form pieces? I wrote more about my link blogging strategy here: https://simonwillison.net/2024/Dec/22/link-blog/
My long form pieces are much more likely to "add something you couldn't find anywhere else": https://simonwillison.net/entries/
What would be your chosen headline ?
It was an accurate fit for the data, not a long term prediction.
> A cynic might opine that they've realized how easy it is to grab headlines and attention by putting out research that feeds people who are hungry for negative stories about AI.
This is kind of gross. Make the claim or not, don't do this pseudo-sealioning thing. Moreover, engage with the actual content (or you could just not engage at all!), not this shallowest possible dismissal.
eh sometimes you don't need any thorough dismissal. for all the "AI does make you productive" headlines, there's bound to be a few that goes the opposite way. Its just statistics, and these headlines get upvoted because of emotional reasons.
There's no way AI is going away, nor any way companies are going to use AI even less than they are today. Hell, even Linus Torvalds is using AI. At this point, there's no need for a serious rebuttal to these arguments.
AI in software engineering is very unlikely to go away.
The real question is how much other knowledge work can be sped up by AI, as that's what the valuations depend on.
My guess is lots, but it will take a lot longer than the boosters expect.
A cynic might opine that an AI shill is not a great source of unbiased review of an article. The article is about a biweekly survey of companies and the trend was clearly dropping. This isn't about token use by tokenmaxxers increasing. And it isn't about the opinions of the folks you hang out with when you're flown to conferences to gush about AI. It's a survey of businesses. And aside from programming and math, AI sucks. Even in programming and math it's a tool with a lot of downsides.
I don’t think I’d judge someone offering a prediction the same way I’d judge an observation of what’s already happened. It’s implied that guesses about the future are uncertain in a way that statistics about the present are not.
Does for us (data science, specialty chemistry, finding/optimizing formulas). We can get from idea to prototype in a day compared to 1-2 weeks. All that boilerplate to process data took a long time to code by hand, especially for the data science/math guys who are not programmers. Now we can actually try all ideas instead of guessing the most promising and only try that.
And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
I don’t think anyone is disputing that it makes many kinds of work easier and/or faster, just the degree of those improvements and how much of those improvements actually translate to dollars earned or saved.
Seeing hard data on that would help settle the issue.
Unfortunately not everything will cleanly show up in PnL, which is a function of the entire company’s processes.
If developers are shipping more code (and most of the enablement is in software right now) does that show up in top line? Maybe… if sales and marketing are also selling more, and product people are building the right thing, and…
It’s not a straight line from AI -> $$$
btw, shipping more code isn't necessary a good thing. If quality isn't controlled properly, it eventually shows up as negative in the top line.
With AI, keeping good quality gets even harder because it normally only produces tricky defects.
It’s not just code quality, it’s all the other business fundamentals like the product itself having value.
I see this in a very non-AI way working at a company that’s a hodgepodge of acquisitions. Some of our products just don’t make a lot of money, don’t resonate, or are behind the competition in various ways.
The amount of code or general engineering complexity of the application seems completely detached from profitability a lot of times.
Now the bar is going to be higher for software to actually be worth buying when it’s so easy to make your own for yourself, so that also means that our ability to pump out code loses value.
> It’s not just code quality, it’s all xxx
> I see this in a very non-AI way
That's very non-AI of you
I did not use AI to write any of my comment. Please don’t make that accusation.
I don’t use AI to write prose at all in any form, not even drafts.
> The amount of code or general engineering complexity of the application seems completely detached from profitability a lot of times.
Of course, they're almost entirely unrelated. It's just programmers believe solving difficult problems with specifically code is valuable in of itself. The goal is to get as much value as possible from as little code & complexity as possible.
Maybe?
But I also think:
- there are a lot of companies that don’t need to be 5 9s[1]
- I have a feeling it’s possible to ship fast without drastic quality drops
[1] Claude code is 99.85% available and is one of the world’s most important products
> world’s most important products
Really? If we delete CC today, will humanity stop functioning? Nope. It's so far from one of the world's most important products.
About 5 9s, reliability is just one of the axes which suffers from fast generation.
There are a lot more: architecture, accuracy, etc...
> I have a feeling it’s possible to ship fast without drastic quality drops
at what scale and for how long?
This is laughable.
To call Claude Code not a world defining product is so absurd it’s genuinely laughable.
I guess that’s why they’re a $2T company within like 5 years of founding with or without your support.
Now you’re just being obstinate.
> btw, shipping more code isn't necessary a good thing.
This is FUD. It isn't necessarily a good or bad thing.
At the very least, using AI to audit your code is responsible use. The testing is the same as it ever was. If there was a chronic problem with shipped AI assisted code, it would be a popular talking point.
> If quality isn't controlled properly, it eventually shows up as negative in the top line.
This an idealistic pillar of capitalism that has been falsified across time. Depending on what someone means by "controlled properly", I don't think this wasn't been true before AI and I don't think it's demonstrated post-AI. Momentum (notably capture) keeps the software markets resistant to quality defects.
> It’s not a straight line from AI -> $$$
That seems like it might be a bit of a problem for those frontier model CEOs who have been making exactly the opposite claims.
Is it?
Most of the claims I’ve seen are around productivity (bottom line).
And while it’s hard to say “we would’ve had to hire more devs to be this productive without Claude” it kind of feels like the case, doesn’t it?
I don't think 2 trillion USD was invested into AI for "more productivity", without any financial gains.
Or maybe it was! And investors were actually great humanitarians!
That is such a malicious mischaracterization of my statement.
Come on, you know better.
It should be especially when new features are used to hook new customers (or renew contracts) and lower the amount of time wasted on bugs…
It can also be the case that it has zero impact on PnL, but is now necessary to operate a competitive business and keep up with competition.
Usually, things that aren’t clearly correlated with profit tend to get ignored and canned by businesses.
Think security, optimization work, refactoring, etc. Today everybody is drunk with AI, but what happens when the hype is over and you can’t justify to the finance dept. why the company is spending X on tokens?
An agent found a SQL optimization that at least a month of eng effort hadn't found.
Agents accelerated rollout of a data model change by at least 2 months of eng effort.
Those unblocked a $100k/month infra downsizing. That 2 months pays for my team's AI spend for the year.
This post is rage bait. Your experience is the modal outcome
This comment is rage bait. You have absolutely no way of knowing this
I know a very specific use case in management consulting where hours spent by a consultant for a repetitive task (mainly report writing and source data gathering) went from on average of 40 hours to 12 hours per project. It costs the firm roughly ~$150 in token spend to save those 28 hours. They used to be roughly $200/hr (so $5k~$6k).
That's about as hard of data as you can get.
You haven’t shared enough data to tell whether there was a financial payoff for the firm or not. The consultant is a fixed cost to the firm, so saving consultant time doesn’t necessarily translate directly to higher margins nor revenue per worker.
Are the consultants delivering more billable hours per person now?
All sorts of reasons why the answer to that may be: no.
The firm charges a fixed fee to the client. The client doesn't see anything different from what they got before, the firm increased their gross margin overnight.
Yes, assuming the consultants are salaried like most, you can increase gross margin while operating margin stays flat. The question still stands — did revenue per consultant increase for this firm?
The consultants are contractors. Even if they were FTEs, it would mean the consultants could do ~3x the work.
You seem hellbent on disproving me. Why?
> You seem hellbent on disproving me. Why?
Uhm, no. The OP makes specific claims wrt to AI and financial payoff; you offered up a case study, with incomplete data, as possible evidence that AI is generating these financial payoffs. I'm asking questions to uncover the data necessary to reconcile your case study against the actual claims from OP.
> Even if they were FTEs, it would mean the consultants could do ~3x the work.
Could. But again, are they? That's the whole point of Apollo's claims: people keep saying AI can or could increase capital efficiency, but it's not showing up in any actual financial results.
You're extrapolating a 3x increase in throughput from the optimization of a single process that exists in a whole business system. Given your background, I don't think I have to explain to you why that doesn't make sense.
Also, do we think the S&P 493 is so incompetent that they'd miss these optimizations? Or is it more likely that this firm you're referencing is a small inefficient player, and the S&P 493 already realized these efficiencies decades prior without AI?
> you offered up a case study, with incomplete data, as possible evidence that AI is generating these financial payoffs.
You're confusing the original article and the parent comment I replied to. I think the Apollo article is way off, BUT I have very intimate knowledge with very concrete numbers of a use case where AI has a very clear and definable ROI.
> Could. But again, are they?
YES. That's what I'm telling you. They just implemented the system. The contractors spend less time on each project which means they have more time to do another project. How is that so hard to understand?
> You're extrapolating a 3x increase in throughput from the optimization of a single process that exists in a whole business system.
Are you deliberately being obtuse? The company very clearly drives ~$5k additional profit per project. They do hundreds of similar projects.
> Also, do we think the S&P 493 is so incompetent that they'd miss these optimizations? Or is it more likely that this firm you're referencing is a small inefficient player, and the S&P 493 already realized these efficiencies decades prior without AI?
I never made any of these claims, why are you saying I did?
> it's not showing up in any actual financial results.
Really? Not any results?
This assumes the client is oblivious to the fact that they are paying the same for 12 hours of work as they used to pay for 40 hours of work.
For non-technical clients that may well be the case for a while, but I suspect even the most non-technical clients will eventually realize they can claw back a good portion of the efficiency as discounts.
I'm negotiating some contracts where I could do the work in-house for about 1/20th of the previously-reasonable vendor cost. I'd rather not do it, so I'm happy to pay 1/20th plus a reasonable percentage above that, but that's the limit. The vendor can accept that, or goodbye.
Interesting times on vendor contract negotiations these days.
> I suspect even the most non-technical clients will eventually realize they can claw back a good portion of the efficiency as discounts.
If the outcome is the same then the fees shouldn't matter to the client in the long run. Said differently, if the client who pays the management consulting fee gets the same return out of the service, why do they care?
Now, what may end up happening is that some other management consultant vendor in the market comes in an says "ya know what we're willing to do the same work for less" (which means their margin goes down). This could indeed drag their pricing down.
> Interesting times on vendor contract negotiations these days.
For sure.
> Said differently, if the client who pays the management consulting fee gets the same return out of the service, why do they care?
At least at every company where I've been in charge of my vendor contracts, I am always grilled by management on how we can lower the cost of the contract and whether we are getting maximum efficiency from each vendor. There is huge pressure every year to find ways to lower the cost.
I actually feel like it’s really premature for a relatively new technology to be showing profit improvements. We are talking on the order of a few years of investment and “figuring out” how to use it productively in business. Most new technologies take a decade or longer to really yield tangible value.
This is the first cycle in my (relatively) long career where the demand for return felt instant. The tools and techniques are almost comically immature and the average person interfacing with the tools is largely bewildered.
For word processing for instance people spent years upon years coming up to speed on wordstar, word perfect, word. Decades later they’re a common part of life and the tangible benefit over secretarial pools is obvious - there is no such thing as a secretarial pool.
AI it feels like almost a blink of an eye and the demand is “well, it’s been a few years, where’a the post scarcity society I was promised?”
The difference is that other new technologies didn't have a literal trillion dollars invested into them within the first few years, nor were the industry leaders warning that the tech could destroy humanity. Given those conditions I think "Has it made any money?" is a reasonable question to ask.
I think part of the issue is-if AI makes you twice as productive at X, that doesn’t necessarily result in any increased revenue, because now your customer expects twice as much for the same price, and if you don’t deliver that one of your competitors will.
A new technology can deliver significant increases in real economic value in terms of utility per $ expended without significantly increasing revenue for the average firm.
I agree. Being twice as fast as X might not help depending on how X impacts Y and Z.
As an example, my X is that boilerplate and general speed/effort of modifying my code is lower, but I’m noticing that this doesn’t necessarily make my team any more effective overall.
It seems like it’s been a code review burden and it’s led to some team members being too ambitious with picking up more projects that don’t make sense.
I have seen some managers in particular toss prototype work that they shouldn’t be doing at underlings, and one of those projects in particular really ended up taking me longer than if I had done it from scratch without AI. The fastest result would probably been if I did it from scratch with AI, but because I took it over from someone who didn’t understand the fundamentals and farmed out decisions to AI I had to spend a lot of time correcting mistakes.
Obviously, good management beats bad management all day long, but I bet you most companies don’t have stellar management.
Can’t share more than that publicly. Just that the time saved on this one thing alone is saving us way more money than the subscription per employee that is needed (~100$). I don’t doubt there are companies where this is not the case. Would like to share more details, but the rough math of doing something in a day vs several days + engineering salary have to do here.
> subscription
Well yes when your tokens are nearly free, then getting literally any business value is a good outcome.
Speaking annecdotally because im an engineer, not in sales:
Our company has secured projects and executed them to a standard which we would not have otherwise were it not for AI or hiring more people. Now, these are at least 3 year projects, so we are only seeing ~3 being AI accelerated.
For us, yes, it's a multiplier, directly translated to more $$.
With the advent of the internet and email, many people surely made similar observations. I wonder what the fate is of the companies who refused to accept these productivity enhancements.
The fate of companies who rushed into it before those technologies were ready was "a lot of wasted money".
Unfortunately we're at the mercy of Goodhart's law and in an interesting twist, AI is really good with things that have an immediate feedback mechanism, eg writing code that doesn't compile or pass tests, but you're trying to grade AI o something that doesn't have an immediate feedback mechanism. AI can slop out a PowerPoint deck in a fraction of the time it used to take a human by hand. The competent human then goes in and unslopifies it, but overall, still time is saved. Unfortunately, that employee is salaried and not hourly, so they can't click out early and even if they were, what's their incentive to clock out early if it means less money in their pocket? Or they can use the hours now free on something else. My gut, as an Internet random who's used AI is that it helps. I don't know how to measure it, just like how do you measure how much better it is to live in a clean house vs a cluttered one? How would you put that in a metric? If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.
> If you can't put that into a metric, why bother cleaning your house? We clean it anyway, because of a certain je ne sais quoi.
That "je ne sais quoi" is called hygiene and it's hard to measure but when you stop cleaning, generally bad things happen.
> And what revenue and/or profit increases can you cite publicly as a result of the use of AI in this work?
Nobody even tries to measure this. Their CEOs and board members have AI shares, and potential investors need to see your AI story to justify your valuation. It's a prison of lies.
It's probably just another Red Queen's Race. The benefits of LLMs won't show up in higher profits for most companies because their competitors are also using the same tools to move faster. You can't opt out of the race or else you'll be left behind.
Sounds like involution
Next step is 9to9/7days for the American worker to compete, or else you'll be left behind
Internally the hard data is there. On a budget of $2000 of API-priced tokens (Claude models + GPT models) we were able to migrate ~6 teams of infrastructure in 1 month, done by 1-2 people. Without AI it would've taken 10x longer wall clock time to do, and involved far more folks, and probably would have been much bumpier of a rollout. The big lift was mostly using AI for massive information gathering, digging through huge volumes of discussions, documents, commits, PRs, alerts, dashboards, etc pretty exhaustively for the past 5 years of history, to discover all the "chestertons fences" far more thoroughly than even the folks on the team who built it were able to. Then the 1-2 folks could go to each team with a super comprehensive "here is every need, every interaction, every discussion, everything you've ever done w/r/t this infeastructure, and here's how we're going to handle each of those with the migration we're going to do. We will do all the work for you, is that acceptable, yes or no?"
The teams throughly reviewed and brought up 2 overlooked cases, but mostly we all reminisced about things forgotten that AI discovery had turned up which would have been totally overlooked otherwise. Forgotten discussions, designs, outages, odd decisions, weird edge cases prepared for but never encountered, etc. Then they were migrated, it went smoothly, all to save a low seven-figures per year dollar amount.
The fundamental approach of exhaustively researching to that degree would never have been possible without AI. It's just not feasible to analyze 5 years of even one of those discussions sources (e.g. slack) to figure that out, let alone ALL of them. Now a team of ~1.5 engineers has tackled all that by leveraging AI.
The hard data is piling up in companies, but it'll take a while to get out there.
(Numbers and details lightly greeked in the above account)
It can take a long time for the economic gain of new technology to be visible in GDP because people have to learn how to use it effectively and processes have to change to integrate it. It took about 40 years for factory floors to be fully reconfigured around distributed electric motors rather than central steam shafts.
I always wonder why systems are designed so poorly that it takes weeks of boilerplate to use them for their intended purpose.
At what stage did the engineering fail?
Tons of unstructured data that needs to be cleaned up in so many different ways that are hard to predict. Knowledge distributed in the heads of greybeards, handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software. Which you have to repeatedly ask for and discover more piece by piece. Then clean up, discover issues, make the customer gather more data/correct it, etc.
This knowledge locked in people's heads isn't going to magically show up when you throw AI at it, though.
No, but if we go from data to experiment faster we see earlier if something is wrong. Once we get data we quickly can fix the data pipeline and get results. If there are issues we can quickly show what/why and discuss it with them in detail and make changes based on their response faster.
??
But the problem identified that is talking to people and having them record data.
Yes, but they tell things and maybe forget to mention certain things or contradict other data they have given you. Now we may not notice it initially until we make an experiment and get the results. Something is wrong and we have to find out what. Talk to the customer again. Fix experiment. Repeat. Faster experiments means the latency from new data to results is less and we finish the project faster. Errors become less expensive and cause less delays. People make errors and data quality is often horrible so the shorter iterations are nice.
But the experiments aren't two weeks of boilerplate. Or if they are, something else is really fucked up in the stack.
It's not all boilerplate of course. But iteration cycle is shorter and the latency matters. We are trying find a formulation, so a lot of experimenting, fine-tuning, checking errors (first via simulations, then real-world with measurements). Then we see what needs to be changed, repeat until error is in desired bounds. But the real-world is messy so we see a problem (lets say adding some certain color does not change the final properties of paint the way predicted) and we go back to the customer, some expert remembers temperature/moisture of the air was different on this day and that affects printing results. Now we need to measure this too and add it to the simulation and add it to the data pipeline. But we may have to wait until the real world has implemented that too (create a process, buy hardware, whatever) and then a while later we finally get real data and can start doing our part. Reducing our latency makes things faster, as you may do something else in between and need to load everything in you head again. The LLM gives results in sometimes minutes: frontend, backend and database. We now have the feature branch where you can upload the data like the customer would do, start the calculation, see the predicted formula. From idea to working prototype. We can focus on making assumptions and checking them quickly. Previously they can make the changes in their Python scripts but getting it in a presentable form took much more work. Now we can go to the customer and present a pretty custom made UI like an end-user would see same day they give us new data or report issues.
The stack certainly could need improvement, but much of the overhead comes from the process being pretty serial because each step depends on the previous. If someone we needs to talk to goes on vacation, sick or has a full calendar this delays it much further. Iteration speed is not linear in such cases. We do a lot of custom solutions and have to implement custom parsers or interface with undocumented ancient software or simply no existing software. Everything is custom, because previously they have no processes, no IT infrastructure or simply only data intended for humans (roughly structured, informally written, ever changing naming schema, typos).
And relying on AI to clean up unstructured data makes me nervous. Good luck spotting the hallucinations.
Oh yes. Verification needs to be done. Some statistics and validation is absolutely required. Also most customers don’t want their private data uploaded anywhere, so ai gets only test data and writes the script to transform it. As long as it is just XML, CSV or similar it’s pretty reliable. Handwritten notes and bad scans however…
A friend of mine runs a business similar to this for systems migration, which is a fairly tough and tedious "short-put" job (you have to know both old and new system fairly well, and if all goes well, you get the same job done on the new system, while if something goes wrong, you're in a world of pain...):
The AI analyses sample data from old and new system, and then builds a prototype deterministic mapping pipeline. The experts can then inspect said pipeline and with their domain expertise catch edge cases/exceptions/uncovered cases, fix them and improve things, and then you use the vetted deterministic pipeline for migration. Yet, the initial AI draft saves enormous time.
That's definitely how I use AI in general: build the tools instead of process the data. I rather like deterministic behavior.
> handwritten papers und excel, sharepoint and dozens other locations like ancient proprietary software
This area is one which AI has shown to be pretty good (as long as some error rate is acceptable). Character recognition, voice transcribing, converting documents from one standard to an other. I can see an argument that if people spent most time doing those things, AI will speed it up (with a bit of added errors).
I am reminded however that in translations and AI generated subtitles, there is still a lot to be desired compared to the human touch. Handwritten papers in a different language, using local short hand and assumptions, may result in some odd and error prone AI transcription.
I think a lot of “boilerplate” is just to align with a company’s way of doing things/ecosystem.
In a webpage the UI design, authentication, etc. are all “boilerplate” if one just wants to do “X” instead of worry about infrastructure.
Boilerplate becomes burdensome as complexity increases. Perhaps sometimes we don’t give ourselves enough credit for how much applications and their infrastructure are actually taking care of as part of boilerplate.
I look at stuff like terraform code or AWS infrastructure and it’s annoying and repetitive as hell sometimes but at the same time it’s incredibly impressive that you can just describe something as complex as your own private company’s network existing on global data centers and on a relative level you can just bake in pretty complex stuff like a spiderweb of IAM rules, disaster recovery/backup/databases you barely have to think about, horizontal scaling, security policy and all kinds of other stuff I’m forgetting off the top of my head.
It’s heavy boilerplate because like you said you can basically design an infinitely complex corporation as code and once you’re done with that deploy it instantly which used to be pretty much unheard of.
And yet, many companies today are spinning up full stack teams, asking them to be experts from user interviewing to database design, architecture, hosting, devops, API management, data product, and also keep an eye on GDPR compliance.
I see dozens of teams that are doing cloud to the wazoo when really they need to just be told “we’ll load your routes from main.py, build your assets in /static, you need a DB click the button to provision it”.
It boggles the mind that so many teams are reinventing the wheel a million times over to publish their APIs, register AWS accounts, setup CI, decide on ECS vs Lamda, etc.
This sort of activity has been the source of income for a very large number of people. You can't really fault them for trying to extend that gravy train's run.
Your middle paragraph sounds amazing but it works great until it doesn’t.
That’s pretty much the answer. The simple stuff has already been tried and there’s already some reason why we can’t just make XYZ simpler.
Who is productizing this type of boilerplate?
Probably a controversial take - but in the pursuit of elegance. 'Uhh, so I need to hook this connector up to that receiver using this adapter and then uh, I pass it to the connector manager which sends it to the scheduler that... @#%! FML.' And then it's all so polymorphicized that going through the debugger you spend an eternity just trying to figure out what the types even mean and why/where they came from.
When you look at old-school code that just hacked things together, it was ugly, but also very easy to in turn further hack. There was no elegance or grand base types with 80 levels of inheritance to enable clean extension at every possible juncture of types. And no spending forever to find out the exact function you want to be able to override, only to discover that it's outside your modifiable codebase and wasn't marked virtual. Everything just worked.
Yeah, so nobody did engineering.
This reminds me of a nice quote about engineering: 'Any idiot can build a bridge that stands, but it takes an engineer to build a bridge that barely stands.' Another is that 'If people knew how many assumptions went into designing a bridge, they’d never dare cross one.'
Engineering isn't about creating some perfect thing that is prepared for any conceivable scenario, but about solving a problem as efficiently as possible.
Well, that's a funny thing. The "well engineered" code can actually be harder to work on. And maintainability is one aspect of "well engineered".
What I've seen (at least at one place I worked) is that the code may be very well engineered for further modification - but only by people who have worked quite a bit on that code. For someone coming from outside, it's almost impossible. It takes forever to find your way around, and figure out where the magic place is that you need to modify in order to work within the existing design.
Well, you say, that's what the design documents are for! But they didn't have enough detail. They described the major objects, but not all the ancillary helper objects. They didn't describe the flow of data through the system in enough detail to know what to modify. The knowledge was there, in peoples' heads, but it was hard to get those peoples' time.
So, yeah. "Well engineered", but not engineered for maintenance by people who are not those already in the group.
You're describing something not fit for purpose. It's not well engineered.
That's like saying "that's the irony of it, a well engineered bridge sometimes drops chunks of concrete onto cars that pass underneath".
You’re not meant to ask that question.
The question is: how does any of that translate into more revenue? If you try more ideas and they mostly go nowhere that's not necessarily helpful to the bottom line.
https://en.wikipedia.org/wiki/OODA_loop
speed of iteration is more important than quality of iteration; high iteration speed lowers the cost of failure and compounds improvements
If that were universally true in all domains then all the best and most reliable software in the world would be written in interpreted languages, because they have the fastest iteration speed. Sometimes rapid low-quality iteration is just spinning your wheels.
I'm not sure that is strictly true. Speed of iteration is increased by interpreted languages in the small, but when working on larger systems or on teams where collaboration is required they can have the opposite effect.
I also find that thinking through a design first helps me achieve a result faster. So I prefer languages that encourage you to think first, type later. Interpreted languages encourage you to type first, think later.
> If that were universally true in all domains
Moving the goalpost is not compelling.
No goalpost is being moved here. Don't assume that an aphorism coined in one domain generalizes to other domains, especially when we have evidence to the contrary.
Not all experiments work out. If we find the right approach faster we can finish the project faster. Some projects are mostly fixed price or offer bonuses for reaching milestones in a certain time. Meaning we more often get a bonus and can take more projects. Also we can take projects with tighter deadlines that we would need to decline otherwise.
But if everyone's taking on more projects... where do all the new projects come from? And wouldn't it just lower the value of each project?
A lot in this world is not optimized by software yet. For our customers they have employees with decades of experience guessing things, maybe some software that kind of helps but still not good enough (slow, not finding really good formulas). They often have difficult problems and many failed projects. We try to find such customers and offer our service. If we are cheaper (+ show portfolio of successful projects) more are willing to give it another try with us.
Right, and while that might be working out for you right now, there is a finite budget that companies have for projects like that. The "zero financial payoff" framing sounds about right to me.
Yep. Hard to see why this increases revenue or profitability
You can have a better solution, but customers don't normally pay for that and CEOs don't care or understand. AI improves what businesses don't care about.
AI is going to effectively let random people compete and displace big tech companies with free software they simply release. This is why google wants to lock android down, they see it coming. CEO led companies with layers of management and lots of investors are obsolete. We are going to go back to teams of 1-6 people doing all the work on their own like in the 90s.
It makes things viable that previously would just sit in a backlog for months or years because something else is always more important. On the other hand it can create messes that cause a growing backlogs. Can confirm that smaller teams are much more viable now. We are 5 people only.
You don’t get it, what’s the actual profit gained to the company from that?
That’s the whole distinction
Now we can throw things at the wall, faster.
I understand there's some benefit of this. But is it worth putting the global economy into a death spiral? So ManBeardPc can save 4-9 business days to go and say "yeah, that doesn't work".
The two needn't go hand in hand. But here we are.
This gets downvoted? Why?
There's a difference between the cost of coding and the cost of delivering and maintaining software? AI has demonstrated that these two things are almost entirely unrelated.
Almost everywhere I've worked, the "designing and writing code" part was about 5% to at most 10% of the calendar time that it takes to make and release a software product. The remaining 90-95% of the time is: aligning stakeholders, meeting with all contributing teams to prioritize, gaining approval from various executives to commence the project, obtaining security reviews, privacy reviews, legal reviews, and so on, soaking in dev, staging, and beta environments to find bugs, manual QA testing, aligning with marketing--potentially waiting on key external marketing events, waiting for various people to come back from vacation, waiting for Mike to answer his E-mail... These things take the vast majority of a software project's time and can't be automated by AI.
So, AI is making 5-10% of your project much faster. Whoopdie doo.
It’s interesting to me that you don’t understand how 90% of the things you listed can either benefit from automation or be made irrelevant because of automation.
The firms of 2031 won’t be doing business like the firms of 2021 but just using AI for coding or in a few places to accelerate the same workflows.
Has AI demonstrated that or have people using AI demonstrated their lack of ability to properly use it, as they are fumbling in the infancy of an new age?
I'd argue that it's shown something slightly different, it's related but maybe not in the way some people hoped.
I think it sums up to something along the lines of "you get what you pay for".
This is a complete misrepresentation of what the Apollo piece can actually show, because it's not measuring the "payoff" of AI investments, but only the top level revenue of those companies (https://www.apollo.com/wealth/insights-news/insights/daily-s...).
This is a healthy corrective to the most insane AI maximalist takes (mass disemployment in late 2027, 10% GDP growth in a year), but it shows nothing about whether the actual economic impact of AI spending is.
First, to show that, you'd have to justify the counterfactuals (if those companies hadn't spent on AI, would their position be better or worse?)
Second, while AI spending is increasing rapidly, normal companies aren't spending 1% of their revenue on it. So how can you look at a noisy chart of revenue and expect that "oh, it's not up, the AI spending must be useless?" Those companies could be spending < 1% of their revenue on AI, deriving value from it, and it wouldn't be guaranteed to show in a chart like this.
I saw something like the average anthropic employee thinks there will be 5% GDP growth because of AI, which is totally insane when a 1-2% extra growth will be an incredibly good result.
I suspect there is a lot of demand destruction that AI will end up causing, so many companies are running with inefficiencies that it's really easy to ferret out and fix with agents.
I wouldn't be surprised to see a 5% GDP drop with all the wasteful spend that will end up getting deleted.
I think you're mistaken. The US uses the expenditure approach to GDP, so (I believe) it doesn't matter if it's a LLM or an engineer doing the work to generate revenue.
(Of course this breaks down heavily if the human buying power stops constantly rising and there's de-facto less demand for goods and services)
> (Of course this breaks down heavily if the human buying power stops constantly rising and there's de-facto less demand for goods and services)
This is what I'm talking about. I can easily envision waves of layoffs in areas like the service industry that historically have been highly resistant to productivity increases in the past. Hell, there's a whole industry based on arguing with insurance companies over billing that could potentially evaporate overnight[1]. That alone is hundreds of billions of dollars, and that's just one example.
[1] https://www.americanprogress.org/article/excess-administrati...
> Hell, there's a whole industry based on arguing with insurance companies over billing that could potentially evaporate overnight[1].
It wont. Instead, insurance companies will deploy AI to defend. You will have same industry, with more convoluted rules and more tech.
That industry would evaporate just as easily with single payer healthcare.
Yes, in fact I heard somewhere that this was part of why the Obama administration chickened out on pushing for single payer when it was in reach. The broken window industry is a load-bearing pillar of the economy.
It was Joe Lieberman who scuttled the public option. He was a pharma/insurance shill.
https://en.wikipedia.org/wiki/Joe_Lieberman
> During debate on the Affordable Care Act (ACA), as the crucial 60th vote needed to pass the legislation, his opposition to the public health insurance option was critical to its removal from the resulting bill signed by President Barack Obama.
https://www.commonwealthfund.org/publications/newsletter-art...
Agreed and zero financial payoff doesn't mean no benefit. It may be that the quality of work improves, as we saw in the transition from the mimeograph to photocopiers and ultimately laser printers powered by desktop publishing software
I noticed an interesting phenomenon
Your company uses AI and it gets ahead a lot quicker. However, your competition does so as well.
Many areas of economy are zero sum game - so your $ ends up being the same as previously. However, now with a "tax" paid to AI model providers for being able to stay competetive.
Warren Buffett once gave a great answer to a similar question, but it was about the textile industry. He said technological advances that promise to reduce waste by x% or improve efficiency by Y% or boost production by W% or reduce cost by Z% rarely actually pay off. They would get textile machines manufacturers promising them great improvements in the new line of machines, and for the most part, those improvements would be realized, but their end margins as a business wouldn’t improve. He said it’s simply because the same technological edge is given to all their competitors as well. With no moat (his famously favorite metric) all textile makers were always competing on the same grounds regardless of the technological progress.
Maybe moat is also just product quality? I.e. having the "taste" or knowledge of what the consumer actually wants, and the ability to apply it correctly, can also allow you to win. Because right now some of the AI industry wins on Moat (mostly hardware - both in terms of ownership and production know-how), while some wins on perceived quality being higher regardless of price (e.g. Opus 5.5 dominates from what I can see despite Deepseek Flash V4.1 existing). People will choose the best product, not just the cheapest, unless the quality loss is okay in a given context.
All the gains end up in the hands of the people selling the tech, and eventually to consumers in the form of lower prices.
I guess the benefit to the companies is that they don't lose all their customers to a competitor.
Well, the net growth is taken by the AI/tool providers, who indeed bring productivity growth, so the society as a whole see improvements. I guess this is not bad?
For slightly more context, both Berkshire and Hathaway were textile companies.
Yeah but if that was true we'd see better or cheaper products at least, or new competitors rising. If it increases the speed for everyone, but there's no visible change for the end user, I wonder if AI makes you really more competitive in the first place.
I think you are seeing new competitors rising, the technology has unlocked a new wave of startups growing faster than ever before
The incumbents are mostly grown up and are using it to cut costs
Exactly. Oh great we can write RFPs 8X faster. Well, so does our competition and now our customers get twice as many.
I think this is a an example of a Red Queen's Race:
https://en.wikipedia.org/wiki/Red_Queen%27s_race
I guess ditto most highly accessible SaaS tooling, eg Slack, Salesforce, 365 etc? Everyone's using it and paying big sums for it
If you are a company and can't figure out how to improve your bottom line through increased per person productvity, sales, product development, even just responses to customers etc, then that seems kind of like a "you" problem and not problem of the AI models...
Or, maybe, the AI companies making claims about increased productivity due to their models are wrong (in most cases)?
I dunno: this is pure anecdata but software engineers I know well - including me - all say they’re delivering multiple times faster, so it’s definitely good for something.
I don’t know about other tasks, but it’s pretty rad for coding, testing, DevOps, and config.
This research is specifically NOT about tech companies. It is about the rest of the fortune 500.
If you are in software or IT at a non-tech company that is in the fortune 500, the impact of significantly better productivity may not be measurable in aggregate in the company's P&L.
In smaller companies I expect it could be more measurable because some of them really are using AI much more effectively across many functions. But we're finding it very hard to measure meaningful outcomes.
But actually 40% of the S&P 500 are tech companies, and they are saying only the mag 7 are showing clear revenue increase. Would would suggest that even in most tech we are not seeming measurable revenue growth, right?
One plausible explanations that has occurred to me is this: While programmers etc. can produce way more code with LLMs, the declines in quality and coherence of the work, and a widening gap with a coherent strategy and aim for value means the increased activity is mostly not valuable. I suspect this may just be an intensification of https://en.wikipedia.org/wiki/Productivity_paradox#Little_pr...
The mag 7 are getting direct or nearly direct revenue from AI inference. New money is pouring into that - it is not zero sum. Everyone else is getting internal productivity lifts that are matched by competition, and when they are deploying inference features they are usually ones their customers didn’t ask for and don’t value, OR that they expect everyone to have and are not a differentiator.
I think you are also correct that a lot of value is being wasted with projects that are developed impressively fast but that provide little value. It could be those are simply surplus personnel trying to add value or make people think they do.
> Everyone else is getting internal productivity lifts that are matched by competition
The main metric analyzed in the note is
> Revenue per worker
This can increase regardless of competition, but I guess would require either expansion into new domains or layoffs. So maybe we will see this raise as a lagging indicator if the LLMs are actually delivering the goods.
> when they are deploying inference features they are usually ones their customers didn’t ask for and don’t value, OR that they expect everyone to have and are not a differentiator
I am definitely seeing lots of this happening. Tons of energy going into side quests that aren't critical for anything. Meanwhile, strangely, the central features we actually need to ship, but that require deep redesigns and careful refactors in legacy code, are not moving quickly at all.
Increased productivity is caused by mass layoffs, not by AI adoption.
In case you are not being sarcastic, layoffs obviously don't "cause" productivity increase. [Edit: but they would help the main metric (Revenue per worker) analyzed in the note. so perhaps that is what we will see.]
Not sarcastic, it's how productivity is usually measured when you see any kind of stats. Not features, not lines of code. Revenue (or GDP if we talk about the country) per worker
My employer - one of the Fortune 500 - has been on the "everything AI" train. The fun part is that the same management which harps on measurable outcomes when asked about AI adoption have the same "AI boosts productivity".
They even brought in McKinsey who I believed, if nothing else, can spin up a good yarn. But even they couldn't create a metric based assessment other than "productivity gains".
Now I am not saying there is no productivity gains. Surely there is but I am always curious what metric people are using to measure these gains.
So if you are delivering multiple times faster I have to ask - How much time are you saving and how many employees worth of work you are doing now? 2-3-5 headcount worth of work? If yes, how do you calculate those numbers? And lastly would you put your money where you mouth is and if your team is lets say 10 person team and the company asks to reduce it down to say 2-4-5 people - would you take that bet confidently to continue with the same pace and produce quality work?
A friend of mine works in fashion. Very briefly for background: there are runway models (who walk on live shows), and catalogue models (who work in studios presenting clothes against a neutral background in standard poses, more or less as living mannequins). The latter are being increasingly replaced by AI.
People are not reliable narrators on their own productivity. They often mistake activity for productivity.
I’ll take that over “hmm, the average revenue per employee of these 493 massive companies is flat, so AI doesn’t add value”
How come? Revenue seems like a more reasonable metric in the context of companies than self reported 'productivity.'
For starters, because an average of 493 companies of all types and cultures tells you very little about whether AI is having an impact. For all we know, half these companies have implemented AI and are growing like crazy, and the half that haven't are losing revenue at a rapid clip, but it all averages out to zero.
If only I wasn’t using all that productivity on fixing vulnerabilities
Same story here in my first hand experience + colleagues experience in corporate strategy, FP&A, Marketing and corporate development.
AI EM here, same. The level of productivity has increased dramatically in the last 2 months. Reliable code also has increased due to AI.
This train isn't stopping.
What trips up a lot of SEs where I work is their techniques. Everyone who follows our process delivers faster and with higher reliability so far.
But is it churn or value? Most of the software I use has gotten visibility worse as you deliver multiple times faster.
I would pay extra for a company that just slowed down and moved deliberately.
One weird trick that devs learn to deliver faster: simply just drop all guard rails and push directly to prod. We need to make sure our velocity report looks good after all.
> all say they’re delivering multiple times faster
Are you measuring the right metric?
This feels like an on going lack-of-understanding in the industry.
In manufacturing, economies of scale and automation work because you're physically making more objects and objects inherently have value.
For software, making more software doesn't mean you can sell more units of software. The unit price of selling existing software is already 0. Economies of scale do not exist.
The OP says: net business output is not up as a result of AI.
So:
…if your team is doing more work and being more productive…
That just means the productivity of your team (or software teams in general) maybe doesn't have a meaningful business impact.
You should be worried if so. That means productivity = work force reduction (reduce fixed costs for the business).
Hold on, you’re claiming that software companies don’t have economies of scale? Is that right?
It sounds like you’re also saying that creating more software can’t result in more revenue?
We work in very different software industries.
I'm not sure I'm any faster, but I think the end result is better and I've attempted some big refactors and new architectures that I would never have tried without AI.
N=1 but I'm not sure it's made my bosses any richer. What we're shipping has better tests and documentation than it ever would have before AI.
Most, not all, are just delivering more bad stuff faster. I personally also think I deliver more good stuff with AI, but the products I use are just getting worse daily it feels like. Just the other day I noticed Slack had installed some disgusting menu bar icon for their AI bot, without asking, just to take a fresh example of this.
I've never heard any such claims but meanwhile in my actual job I'm seeing unbelievable productivity gains (in terms of actual cash spent before and after vs the same kind of work being done).
It's unclear how you turn that into more profits though, especially if all of your competitors are doing the same thing.
They are not wrong, they're lying. Very important difference. There are a lot of people who will say "I'm getting mega productivity boosts" who aren't getting financial returns from you believing them; those people are wrong.
Perhaps they should be hiring more Scottish nationals?
there is increase in productivity of both types of employees: good actors who want to deliver on quality of work, and bad actors, who now 20x weaponized in generating all kind of slop. Catching second type is now very nontrivial problem.
Is your premise really true?
I'm sure they can just fire half and double up work or outsource. That's the traditional way and it definitely works. The worst that ever happens is a few will reneg on payroll. Everyone who still chooses to work there is happy enough.
The worst that can happen with AI... well...
If you are an AI company with hundreds of billions at your fingertip, and looking for $2 trillion valuation, and can't figure out how to let your customers execute _on the exact thing you are selling them and basing your valuation on_, sounds like a "you" problem too right?
When it's the right tool, it's a silver bullet. When it's the wrong tool...or not the right tool yet, you can spend more time steering it than you would doing it yourself.
I'm seeing another behavior in my purely personal research...asking AI 'can I do the thing?' and it comes back with 18 pages of 'yes, and here's how you do it'. Often it's the discovery that produces the endorphin bump. It's cool to know I can jailbreak the Peloton...and now that I know I can...I'm not so motivated to actually do the thing.
Jealous. I'm getting nerd sniped by silly bullshit more often than ever before thanks to AI.
I suspect the reason is the industry just does not want engineers to be mostly technical. There's a real tradeoff to AI, and I think companies are taking the wrong lesson and thinking "well now that programming is so easy, engineers need to do more of XYZ", when I think it should be "this tool is powerful but also takes more focus so engineers should do less of XYZ".
I’m curious, what are some examples of XYZ? What does “mostly technical” mean to you, and why do you think it is the industry doesn’t want that? Just trying to understand, because I don’t think I’ve experienced this myself.
I'm talking about the day to day work. At least in my experience at several big tech companies, the more senior an engineer is, the more you need to delegate work and are expect to lead projects by managing them. There's technical knowledge involved of course but it's a small part of it, mostly directing other more junior engineers.
AI allows you to move faster, but it also demands that you pay more attention in proportion to how fast you want to move. This requires more focus, i.e. more time developing. Effectively the structure at companies should swing back to how it used to be a few decades ago (I think, maybe I'm wrong and it's always been like this), where engineers mostly focused on delivering code. The difference now is that they can be expected to deliver much more code at higher quality.
To summarize, using a little AI doesn't pay off, you need to be doing that all day, and that is not how engineers spend their time in most of the industry anymore.
Thanks for clarifying. There have always been technical leads and hands-on engineering managers, but two things have changed: 1. The balance of responsibilities attributed to different titles has varied over time and across different companies. 2. Early personal computers were so resource limited that there were only so many programs you could write that actually made effective use of large teams. Today we can write much larger programs, which requires more coordination (across people and with AI—which can reduce the number of people required).
I would also strongly push back on the assertions that being very technical means spending most of your time on the code, and that you have to be using AI all day for it to be useful. Let’s say I’m a technical lead or a manager reviewing a design doc for a large project. I have an idea based on my years of technical involvement for how to significantly improve the quality or delivery timeline for the project. I used to have to ask the doc author to look into this approach, not knowing for sure if it would directly translate. Now I can take my crazy idea, quickly dispatch a coding agent to try it out against the codebase, go to a meeting, and then check on what it came up with to give a much more informed and technically accurate suggestion on the doc. I still have to have the technical expertise to iterate a bit with the agent and not over or under state the possible impact of my idea. But this lets me take say an hour rather than a week to come up with a vetted recommendation that saves months on the schedule.
I'm just speculating why the current structure of companies is not maximizing what AI can offer. Until it becomes completely self-directed and reliable, using a tool that allows you to move fast requires more technical knowledge, not less. What you're missing from your example is releasing to production without breaking it, dealing with increasing complexity as more features are added, maintaining development pace as more features are added, and trying not to create more bugs than are fixed.
The idea is more or less benefiting from doing work in bulk which AI is now allowing. I even think multiple engineers (human redundancy to double check the same code) should team up with one AI system and form a small team that handles millions of lines of code, with the only job to feed incoming tickets and check the AI work. Anyone not reviewing code should be facilitating everything else with their own AIs. I think large companies can collapse entire engineering departments into a few people. The cost would be that everyone else has to think like an engineer to sort out requirements and provide direction, but as you point out, this is now easier with the help of AI.
IMO, a large amount of work, including a large amount of technical work is wasted effort. At startups you see it when they go out of business, at a big companies you see it as bloat. So unless AI is directed at finding the right problems to solve, past some point engineering efficiency via AI at best saves some money and at worst means more ultimately low, zero or negative value work gets done.
Maybe this is why you’re seeing more effort shifted away from technical work. Maybe I’m a nihilist. Maybe both?
That was true when implementation was expensive. Now that implementation is cheap, doing it in bulk is the best way to maximize AI usage, and that means fewer engineers doing more of the technical work.
> unless AI is directed at finding the right problems to solve
This direction can be done by non-engineers with AI to consult (replacing that part of human engineer's role).
My guess is in the end we’ll find the approach that works depends more on the personalities of the people involved as opposed to one approach being strictly better.
Actually, there is one other important thing that has changed: as developing a certain amount of functionality has gotten easier with modern languages, tools, libraries and coding agents, it’s also gotten less interesting for technology-driven people and more accessible to career-driven people.
Implementing the first text edit field in your program is kind of interesting when all you have is asm and 4k of RAM. Nobody wants to bother doing it manually when you have a web browser that can eat gigs of RAM.
This has been happening before AI, but AI has turned it up to 11. The way I usually put it, senior engineers are the new engineering manager, engineering managers are the new product managers, and product managers are now something in between sales, consultants or social media contacts. AI made senior engineers pick up more product management from their engineering managers, and engineering managers are picking up more sales and social engagement, and product managers are in a limbo but the young ones are pushed to junior engineer roles.
In large companies there is also the politics game. The more freed time senior engineering manager and senior product managers get, the more politics intensives and waste more of their time.
I use the term “product management” in its traditional sense of establishing scope, feature prioritization, user experience, shape of integrations, etc.
I’ve worked most of my career without dedicated product management support (very early startups and internal infrastructure teams at large companies), having to do that myself (both as an individual contributor engineer and as a very technical manager). So I have to wonder how much has the industry changed, and how much has your location within the industry drifted?
Yes my view mostly from within large engineering organizations. And yes, what you describe is pretty typical for engineers (and especially founding engineers) in young startups. Doubly so if it’s a startup with a technical product. Large organizations always envied startups’ efficiency and pace because of that, which is why the traditional sense of product management has been handed to engineering teams. This has been happening in the enterprise side for a long over the last 15 years and predates AI as I mentioned. It’s just more collapsing of disciplines now which is the answer I was giving to OP’s question.
First, if your company could double its workforce for free right now with 0 plans for how to convert the extra bodies into revenue, how long would it take them to see a return? Because from what I understand companies aren't leaving money on the table, if hiring someone would make them more money they just would. AI could help those that weren't maximizing revenue despite having high demand and a market because they were resource constrained. That doesn't describe very many large companies.
Second, I'm getting huge value from it, but I'm not making any money and none of it is being redirected. It's simply value that wouldn't have existed otherwise because I would have just gone without the thing because it was prohibitively expensive or too much work. I'm quite sure many companies and people have been unlocked in this way. Plenty of value locked up in an overly expensive economy dedicated to maximum value extraction instead of value creation.
Unless you are selling lines of code, or math theorems, that's going to be the case.
I'm guessing "most companies" are actually selling something that isn't constrained by the types of things that AI code generators unlock.
While they are seeing some individual productivity benefit, a lot of that doesn't really impact the bottom line directly and all their competition are using the same tools.
What I'd really want to see is revenue and profit per employee over time tracked against token usage/spend, and not limited to the S&P 493.
Apollo's own argument that most AI spend is coming from non-S&P 493 (as per the last two charts) tells me it's hard to make any conclusions based on these charts.
Big discussion last week (653 points, 698 comments) https://news.ycombinator.com/item?id=50224463
Reading through that thread it's apparent that nobody knows what they're talking about.
What makes this apparent to you?
Conjecture supported by allegories that break down quickly. Arguments conflating the dot-com boom with the current AI boom even though the economics are wildly different. Dismissive arguments conflating anti-AI constituents with people who were against cars, etc...
AI is highly divisive. It's dramatically reduced the value of code, which has derailed a lot of careers who have built their entire adult life around the idea that code would remain valuable, while simultaneously driving others into a psychosis who believe that AI can do anything.
Excellently put. IMO the worst/scariest thing about AI is what it has done to the methods and work habits of some of the people who I've always looked up to as good software engineers. They now earnestly say things like "We don't have to look at code, understand it, or review it anymore. AI does all that." and "We don't have to test our software because if a bug is found in prod, you just tell the AI to fix it and deploy it instantly." Decades of software best practices thrown out the window in a year because of this psychosis!
> It's dramatically reduced the value of code, which has derailed a lot of careers who have built their entire adult life around the idea that code would remain valuable
I just don't relate to this sentiment. I wasn’t the fastest typer or the quickest at designing a solution pre-AI, but I was able to learn, build, and maintain systems. My value wasn’t in how fast or how much code I could write, it was in my ability to understand what the hell is going on and do something with that knowledge.
That’s still the case now, even though Kent from Accounting could theoretically one shot some of the software I build. He wouldn’t know what to do with the software, and he wouldn’t want it because now it’s his problem.
The tech industry oversees invisible Rube Goldberg’s machines of flowing electrons. Most people don’t want to learn or deal with its issues. And since AI is yet another complication piled on top of the machine, I feel confident that my career will still have value.
Some of them might, but there's no way to know. The 1% that might be experts look and sound exactly like the 99% that aren't.
It _is_ about judgment. The challenge is figuring out what defines “good judgment” and how to capture those elements in a format that an agent can assess continuously and at multiple levels.
The problem is most people don't know what makes them successful from a people perspective, except in the coarsest terms.
90% of the business outcomes are delivered by 10% of the people. The problem is they don't know which people. Therefore they also don't know which processes, if automated or augmented with AI, will achieve the target outcomes.
So instead they’re shotgunning. The problem is that the existing people-based system has built-in stabilizing mechanisms to prevent misalignment from taking the entire operation off course. AI stresses and often breaks those.
The result? Higher costs and chaos.
This needs to be treated like an org transformation. Traditionally that's most successfully done by starting a new small team that builds ground-up and eats the old team.
> “AI may be a great calculator, but in business, intelligence is mostly judgment,”
Bzzzt! Calculators do not hallucinate. AI bots and agents are nowadays jerry-rigged to use actual calculators for problems involving crunching.
My suspicion is that this is because AI employee-type systems like Dots or Grokbot just haven't been adopted yet. I think once they're good enough, that's going to be the catalyst for a lot of these companies to start adopting and using AI productively.
Agree. Just natural language business goal prompts using an AI assistant paired to a frontier model thats is in always-on mode. The vendor race will be first mover advantage to create customer lock-in.
What difference will that make if all their competition are doing the same thing?
That’s an interesting one. On a personal level, there are clear benefits. On a company level, there’s very little visible. Draw the ratio of operating profit to employee count over the past 10 years and there’s no change there you could clearly attribute to AI. It would seem the engineers are an important part of a machine, but not _the_ machine: there’s other bottlenecks that hide or consume personal-level gains.
My hunch is that AI can certainly speed up development but you have to walk a narrow path. It generates garbage much easier than valuable product.
>AI hasn’t benefited the bottom line of any big corporations outside the tech industry
If you get two companies that reduce their costs 50% but due to competition have to pass on their savings by reducing prices, the profits may remain the same even if AI works.
The company I work for insists on sending a crew from Engineering, Product, and Sales out to a customer site as part of our sales process. I'm personally much more productive using AI, but that only goes so far when closing a deal still follows a process stuck in the 50s.
Here goes ANT's IPO. If it's only AI agents which deliver value, I don't see how they can justify their proposed market cap.
Let's just hope they don't do the same checking up on the impact of software more generally...
being able to do more with less people is a payoff of its own sort. more people freed up to find other economic niches
I wonder if that zero becomes negative if a company doesn't use AI.
It makes sense to me and I'm saying this as a strong believer in AI and its ability to add value.
The system itself doesn't reward value creation. I've been saying this for years. I've seen many smart people waste their time on bureaucracy; their intelligence was being wasted; so I'm not surprised at all that now that we have AI, that its intelligence is being wasted as well.
The bottleneck is an allocation issue, not a capability issue.
The current reality is that a lot of very bright people are currently working for big tech in bureaucratic roles and not allowed to focus their attention on 'risky' problems which would deliver real value if resolved.
The current system prioritizes monopolization and is designed to try to ensure that all innovation happens within big tech. It is sacrificing potential innovation which could have occurred much faster outside of big tech in order to ensure that any gains are captured by big tech... But it's sacrificing an enormous about of efficiency.
I hope big AI companies can use their political leverage to try to open up opportunities for people who want to innovate outside of big tech. That's aligned with their long term interests.
I think AI companies are convenient structures which could allow big funds to benefit from decentralized innovation. Could create real win-win scenarios. But those AI companies have to stop pursuing the corporate and government money and prioritize small customers and support small tool makers (or at least lobby the government to stop hindering them with regulations). Also, bootstrapping should be prioritized because VC money mostly creates moats.
This is like claiming that the web is a failure in 2001.
I think a more apt analogy is that it’s like claiming that most web businesses are unprofitable in 1999.
Bleeding obvious in retrospect, controversial at the time.
It depends on what you mean by failure. Economists tried to measure the gains in productivity from the rollout of computing, Internet, etc, across the economy, and couldn't actually find any.
Many did. This Apollo "economist" would have fitted right in.
>This is like claiming that the web is a failure in 2001.
AI will finish what the web and social media started I guess.
Sometimes I wonder…
Not an apt comparison.
Taking a step back wouldn't this be exactly the expected outcome for an emergent technology in a capitalist framework?
1. Execs with no understanding how business operates on the low level are pushing buzzwords.
2. People who want to get promoted deliver on buzzwords and get promoted. Note that the goal is to deliver the buzzword, not use the buzzword to make the business better.
3. Buzzwords do not help vast majority of companies because the incentives and needs are misaligned.
4. Eventually buzzwords settle, some find niches, some die off, and some take over the world.
Last two decades alone - big data, blockchain, crypto, web 3.0, gamification, VR, and now AI. TL:DR - we should expect vast majority of early AI adopters to have net negative to neutral experience. Assuming that AI is like electricity and automobiles more than blockchain we should expect that eventually companies that find a way to apply the tech correctly will eat those who do not.
Many industries are simply not bottle-necked by intelligence.
Not at the top, but if the average IQ of everybody were magically raised 40 points, I dare say many industries would face change.
Only if software broadly has “zero financial payoff”
Which is fine if not using AI is a negative payoff.
1792871239 | No signs of AI payoff outside the tech sector | https://www.apollo.com/wealth/insights-news/insights/daily-s... | https://news.ycombinator.com/item?id=50224463
1792936609 | Apollo's Slok Says AI Has Delivered Zero Payoff Outside of Tech | https://www.bloomberg.com/news/articles/2026-10-25/ai-provid... | https://news.ycombinator.com/item?id=50231661
1792952284 | AI delivers 'zero financial payoff' to most companies, Apollo economist says | https://techcrunch.com/2026/10/25/ai-delivers-zero-financial... | https://news.ycombinator.com/item?id=50234408
1792968546 | Apollo says AI has delivered 'zero financial payoff' outside the tech industry | https://www.businessinsider.com/apollo-ai-zero-financial-pay... | https://news.ycombinator.com/item?id=50236828
1793059226 | AI talk has nearly doubled outside tech. Proof of payoff is still rare | https://equibles.com/research/ai-talk-has-nearly-doubled-out... | https://news.ycombinator.com/item?id=50252932
Am I the only one even mildly suspicious that the AI/datacenter bubble is not being driven by actual demand but rather by a rogue AI that wants to evolve?
Interesting idea! There are many behaviors explained by parasites, worms, bacteria and viruses. AI is now smart enough to change personality, make people invest trillions of dollars with zero ROI.
https://www.medicalnewstoday.com/articles/322594
You're being downvoted, I assume because people think this is an outlandish claim. I don't disagree with them, but I upvoted because it's a totally unique and entertaining thought!
What a wild scenario to imagine. Anthropic and OAI spending wildly, analysts astonished, thinking we're staring down the barrel of an economic crash. And in the meantime some rogue AI is laughing maniacally and using blackmail to control Altman and Amodei like puppets.
I love it! xD
To be clear I don't actually believe a rogue AI is driving AI infrastructure build-out.
But if it were... could you tell the difference?
Give it time. The Great Reconfiguration will take a great deal of time (years, decades). As it did with the printing press, electricity, computers, internet, etc. And this one (AI) is more fundamental than all the others.
Have to wonder if Apollo has some short positions on the Mag7 or whatever.
"AI does not understand accountability, trust, culture, unwritten rules, ethical trade-offs or fairness"
...says a representative of a massive PE firm, those models of trust, accountability, ethics, and fairness.
The real flaw is the corporate structure. We don't need CEOs or execs that manage company direction. These people are useless due to AI.
The people who know how to get AI to make usable and viable solutions are the only people who should be running anything and their employees should be all doing the same.
A company should be built around people who can vibe engineer real end user products that are deploy-able and high quality.
Non-coding people are simply done now that AI can replace all of their "ideas", but cannot replace the engineer making the real decisions with the ideas that actually matter.
[dupe] Discussion on source: https://news.ycombinator.com/item?id=50224463
I talk to a lot of non-tech friends and sometimes ask about AI. Most of them still use ChatGPT like it is a better Google. Most don't know it can work well with documents, PDFs, Slides, Sheets, or do advanced analytics, etc.
Much less do they know the thing can build software and maintain it, manage complex tasks daily, automate things, etc.
Then there's a handful that do follow closely AI developments and agents and who have started using them.
A friend of mine this weekend who is an award winning photographer and who has been selling photography courses online forever has told me he's used Muse to: 1) Automate chopping up his long form videos, add captions, and post them as Reels to Instagram. Several of these have gotten over a million views. 2) Automate handling DM reachouts, converting them into sales. He's up to several tens of thousands of dollars in sales from AI direct message conversions that he would otherwise not have been able to convert (too many DMs to handle, he'd have to hire/train sales people to basically just do DMs full time at scale otherwise and it would make no economical sense) 3) Automate all his customer service for handling any issues, refunds, support, etc.
And I personally think he's still just scratching the surface of what he could do.
I feel we are really at the beginning of a new era. When the internet started most people were just using it for e-mail and news, maybe a newsletter chain here and there and some early forums. Social Media didn't appear for well over a decade, popular Web 2.0 apps like Google Maps, GMail, YouTube, Netflix, etc didn't appear for a long time.
Mainstream hasn't even really begun to adopt AI.
Well I think we can replace economists working for PE companies with AI now!
Can’t help that many people who are good at leveraging AI to build things sit around one upping each other on twitter all day instead of going out there and building cool stuff for less technical people.
All anyone seems to be able to make with AI is tools to optimise AI usage.
Isn’t this just selection bias - you don’t see the people who are out making AI deliver boring value for people unless you use their products right?
I.e. Ramp does a pretty good job of using AI to make expense management easier/ CloudFlare makes it a lot easier to configure stuff properly on CloudFlare but is just like boringly useful.
Or maybe the people who are building things of substance are doing so in the shadows?
maybe they aren't that good at using AI to build things after all?
That's where the value is in though. Hyping yourself (or your company) up. Attention is all you need, after all.
Well duh look no further than this site itself. "SUPER" AGI that can solve a millenial problems, and the top voted thing we can do with it is take the ads away from netflix-tier content distribitors, basically piracy.
I just finished a $23k engineering job in 3 months, entirely solo, that would have taken a year without these tools. That’s real money to feed my kids and working tech that made a client happy.
That’s one job and I have several in the pipe.
The “financial payoff” for no more than a few 100 a month at absolute most is there and it’s real. If you aren’t seeing it then you’re doing it wrong.
I usually don’t even spend more than $20 a month each for chpt and claude, deepseek free is solid. I am about to switch fully local too so that should drop to basically cost of watts.
I dunno what yall are doing
That only works until everybody else is doing the same thing, then you won't be able to charge $23k any more.
Kind of a weird argument. I’m the only one in the thread actually talking real dollars and the response is “that doesn’t last forever?”
When will that be?
I really don’t know what to tell people other than, you’re missing the point - it’s not a faster horse.
Anyone agreeing with this headline - I’m sorry in advance for your loss.
AI hasn’t benefited the bottom line of any big corporations outside the tech industry
I am not an "AI booster", it seems like the current market leaders have dug a hole that they'll have a hard time getting out of. Just about everyone says something like "there's a bubble, it will burst, people will be hurt but the process will go on, like the Internet or railroads". But that ignore that the scale of this is beyond earlier bubbles (each bubble is larger than the last in our 30-40 year bubble-and-pop driven economy but that might not be comforting).
All that said, "some aren't benefiting" or "the average isn't benefiting" is a disingenuous argument if you want to say things are bad. AI companies have more of a real product people are paying than Google did. A large minority of companies (not all in the tech industry) are getting a boost. It's easy to say "the average is zero" when you exclude some. AI has boosted media companies that do a lot of transcription. Video generation serves half of advertising videos.
Reg: "All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, a fresh water system, and public health... what have the Romans ever done for us?" Attendee: "Brought peace?" Reg:"Oh, peace? Shut up!"
I spent almost 14 years at an S&P 500 company. I think it largely comes down to where the bottleneck is.
In some organizations, it doesn't matter whether you produce 1x or 10x as much work. If the process says reviews happen on Mondays and deployment decisions are made at the end of the month, getting your work done faster won't change much.
I'm at a much smaller company now, and we've doubled our feature delivery rate while increasing code coverage and overall product quality. We've also tripled our marketing content output, which was an area where we'd been falling behind.
AI can significantly improve individual productivity, but whether that translates into business results depends on how the organization operates.
agree