Here we go. The “AI is overrated” backlash is in full swing. MIT just released a study that finds 95% of organizations exploring GenAI are getting zero return:

Despite $30–40 billion in enterprise investment into GenAI, this report uncovers a surprising result in that 95% of organizations are getting zero return. The outcomes are so starkly divided across both buyers (enterprises, mid-market, SMBs) and builders (startups, vendors, consultancies) that we call it the GenAI Divide. Just 5% of integrated AI pilots are extracting millions in value, while the vast majority remain stuck with no measurable P&L impact. This divide does not seem to be driven by model quality or regulation, but approach seems to be the determining factor. [emphasis mine]

Pretty much what I predicted in several of my recent posts about product development and AI. Transitions are hard and the vast majority of organizations have real trouble navigating them. Clayton Christensen’s classic “The Innovators Dilemma” is one framing of this problem. In my career, the practical drivers have often been:

Leaders build habits and expertise in the world they grow their careers in. With a fairly predictable periodicity, that knowledge becomes out of date and often actively misleading. This was why educating Mark about mobile was part of Facebook’s mobile transition.
Organizations reflect previous investment theses, actively reducing the knowledge intersections needed to drive the next wave of innovation. AI pilots, separate AI research teams, and the hyperfocus that comes from monocultural investment all compound this problem.
Lack of either top-down commitment, bottom-up capabilities, or both. Organizations aren’t built to change, so change will absolutely fail — and experiments that depend on change will fail — if leaders aren’t pushing and teams and employees have the knowledge and capability to succeed in the new world. AI is perfect storm of erratic executive support and nobody really knowing what an AI product is.

Don’t believe the anti-hype either

Even as the backlash starts, organizations are clearly making AI-driven pivots. Take this Stanford study, which reports an interesting mixed set of employment results around AI. On the positive side, it notes “Our third key fact is that not all uses of AI are associated with declines in employment. In particular, entry-level employment has declined in applications of AI that automate work, but not those that most augment it.” Very aligned with my belief that AI will ultimately drive a new category of partnered products. However, there is a very troubling signal:

Our first key finding is that we uncover substantial declines in employment for early-career workers (ages 22-25) in occupations most exposed to AI, such as software developers and customer service representatives. In contrast, employment trends for more experienced workers in the same occupations, and workers of all ages in less-exposed occupations such as nursing aides, have remained stable or continued to grow.

Of course companies are finding AI wins and cutting here. Of course, particularly in computer science, this is a terrible idea since today’s junior programmers are your most important investment in the future. They’re comparatively inexpensive, most exposed to emerging technology and trends, and key canaries that tell you your company culture is working for your members. Companies betting on major cuts here are going to deeply regret it down the road.

It’s about the previously impossible products

As I led with at the top, NewsArc couldn’t exist without LLMs. Older, classical ML techniques — classifiers, NLP, prediction models — were not sophisticated enough to understand news, to embrace the shared experience and constant change that makes news so unique. As I’ve mentioned before, the most exciting time to create products is during times of significant transformation. Why?

Because you can create the previously impossible!

NewsArc is a perfect example. Take our “shhhh” feature, which allows you to mute a news event until a major change happens. This isn’t muting a publisher or a keyword, it’s muting the entire, evolving news event. What’s a news event? A news event is the combination of claims and stories that emerges as a distributed consensus among journalists and publishers as stories emerge, blend with each other, and evolve over the course of hours, days, or more.

It’s the kind of concept incredibly obvious and consistent to a reader, but previously impossible to accurate capture with a computer. LLMs don’t make it easy, but they do make it possible. And once you can accurately track a news event, your whole sense of what’s possible in a product changes.

We’re undervaluing AI

Which is why they hype still doesn’t really capture what’s going to happen with AI and new products. Sure, AI is going to make some people much better at their jobs while some will get displaced, but the real transformations — what the MIT study found companies weren’t finding yet — are the places we can rethink a customer or market need. Where we can imagine previously impossible solutions.

That’s product development and the great product development companies are going to use AI in more impactful ways than anyone is predicting.