Of all the lessons I’ve taken away from my time at Facebook — how to successfully pivot a massive, public company, Schrep’s “loose opinions, strongly held”, the limitations of feeds, the power of simplicity — the hardest to convey is this:

Product companies need product infrastructure teams

It’s a lesson neither the Navy nor my early career taught me. My era of game development — 90’s arcade and console — had a lot of throw away everything after every game. Building Second Life, we were all working on all the things. Valve, id, and Epic were in the process of figuring it out, but nobody built around this idea like Facebook.

It started with User Interface Engineering, a team that emerged from the initial, catastrophic javascript misadventures that grossly slowed Facebook web load times and reduced stability. By 2009, UIE was an incredibly focused and disciplined group who made sure that Facebook was the same product — and fast — on every browser, everywhere in the world.

From UIE to ProdInfra

During the mobile transition, two trends led to UIE growing and transforming into an even more remarkable organization: product infrastructure. First, native mobile — and the APIs required — meant we needed to think more broadly about products than just user interfaces. Second, since many infrastructure and release engineers were more familiar with native code than most Facebook SWEs, we were aggressively stealing borrowing people from infra and release. They ended up closely collaborating with UIE on some of our hardest problems (GraphQL, React, our homegrown (and less capable) typed Javascript).

During this time, UIE renamed itself to ProdInfra, and it really clicked for me what we had: a team of engineers self-selecting for two often-contradictory skills:

1. Deep empathy for users, design skills, and a fearlessness around taking on the hardest problems

2. Deep technical expertise and a commitment to delivering infrastructure that other teams and engineers could rely on for the years ahead

Most hiring pipelines that try to find one of these tend to filter out the other. Looking at the UIE/ProdInfra team, you could immediately see that many were hired elsewhere at Facebook—SWEs, designers, data scientists, release, and others—but the kind of challenges a product infrastructure leader seeks tended to lead them to each other.

The Google version

It took me almost two years at Google to figure out how to leverage this insight. Google—of course—has a ProdInfra team. Google, like many huge companies, has every team you can think of if you look hard enough. It had the same kind of talented, multifaceted members you’d hope for. The challenge was that Google’s exponentially more complicated infrastructure, product diversity, and org structure made it very difficult for product infrastructure to get the kind of wins they need to overcome the natural “not invented here” resistance that every tech company has. Moreover, Google’s fundamental ethos around letting every team—every engineer—have maximum latitude compounded the challenge.

Fortunately, while I was helping the incredible Luiz sketch out what would become Core Engineering, we noticed that a group of teams—Material, parts of Identity, Mobile Engineering, Prod Infra—were an awkward fit, looked more like each other, and were facing similar challenges. I had been badgering Sundar about the missed opportunity for product infra, so as Core Engineering morphed into Core under Jen, he suggested I use this new org as a laboratory to explore how to make it work at Google.

It meant juggling two full-time jobs, but what an opportunity! The question was what to name it. I went with Core Experience, mostly because nobody knew what that meant, and it created time and space to maneuver. As we were forming, Apple’s ATT changes hit, so we were in the thick of it from the start.

So, what do product infrastructure teams do?

Product infrastructure teams do what it says on the tin. They create infrastructure around building products, through the lens of making products better. Usually this means making product development easier, faster, safer, more observable, etc. And not just a little.

Unless you are already a world-class product development org—and I know, we all think we are—product infrastructure is your path to huge gains in productivity, efficiency, and the ability to understand, create, and deliver the right product changes.

This might be switching to more modern technology, having genuine observability across your critical user journeys, or reducing how many languages a developer needs to be an expert in to ship a change. Every product and product organization has unique challenges, inefficiencies, and needs—product infra is where you marshal the skills and capacity to truly fix them.

Like those who obsess about observability (more on that in second), prod infra is about making it so all of your engineers can do great work in your products.

I get it, how do I build one?

Start with the signal characteristics prod infra team members need: the rare combination of user empathy and product sense and deep technical expertise and infrastructure mindset.

Do you already have employees who combine those? If so, take a quiet moment to celebrate, and figure out how to get them working together with enough airspace, goals, and authority to make your products and product development better.

Be prepared to run a change management process around them. Or make them the owners of your v2 product. Expect them to collide with how you have always done things.

If it was easy, your existing stack and teams would have solved these challenges already. Different isn’t always better, but better is always different.

I would, but I just got convinced to build an o11y team

I, for one, would never disagree with Charity and much of the most recent wave of observability has been about creating teams who specialize in it. Yay! To me, while o11y and prod infra teams are often drawing from the same talent pools, the o11y team members skew 10% more towards SRE mindsets while prod infra members skew 10% more toward designers. As you grow and scale, you’re going to need both. Facebook certainly did.

ProdInfra and AI

AI makes it even more important to have highly capable prod infra teams. Whether you’re just trying to leverage AI — nobody benefits more from regular, predictable, and clear product development infra than coding agents — or working on novel, frontier model features, it is incredibly challenging but rewarding to find developers who can work at the intersection of user-needs and deeply technical R&D.

I’ll come back to that in a future post.