Some changes to o16g. Enough traffic to move the manifesto off to its own page and start producing daily synopses. The site adds and categorizes about 50 relevant articles a day.

The great debate: demo or production?

In a glorious world of professionals using AI assistance to rapidly prototype — or, for that matter, to be vibe coding ideas to look at — what happens to all that code? As I noted earlier this week, some people like to use the backlog to bury code like this. I don’t like that idea at all.

However, it’s fair to be deeply critical of just dumping vibe coded PRs into a codebase willy-nilly. Sure, we can all imagine a future where all of our codebases are so well tested, so observed, so partitioned, isolated, and feature-flagged that we can turn any problematic part on or off. Where our build system is so smart that piles of never used feature end points are pruned and contribute neither to payload size nor to risk surface.

But, obviously that’s not where we are today. And that incredibly cool demo backed by comically ugly code is sitting as a PR, waiting to be reviewed.

Pop quiz, hotshot, what do you do?

(Burning bus in background totally not a reference to adding demo code into production code base. Probably.)

Production code vs demo code

In some ways, this split is pretty straightforward.

Production code: code you expect to be a permanent part of your codebase. It meets all of your expectations for testability and o11y, and you can demonstrate that shipping it delivers positive change to your users. Code you’d frame and proudly hang on your wall. Code you gleefully send out for review, knowing it will sail through and your team will sing songs of its simplicity, virtue, and beauty.

Sure, we all know not every PR fits that, but it is what we aspire to. Then, what about demo code?

Demo code: code used to explore an idea, answer a question, settle a debate, or prove a point. Never intended to be deployed to customers or in the production codebase.

Cool, all good. Code reviews are likely a waste of time here. Unless, of course, you are smart and running a monorepo. Then, where did the demo code go?

> ls
> ...
> src/lib/awesome_fasterdb.ts

Oops. How long before someone starts relying on this awesome bit of code that maybe isn’t obviously demo code?

Still, there isn’t a ton of discipline needed here. Make sure the repository has a home for demos. Use tools and culture to enforce it. Easy peasy.

But what about when you want users to experience that demo? Or to deliver it to colleagues in production code?

Production code vs deployed code

An idea I think is worth exploring is the difference between production code and deployed code.

Production code: same as before. Maybe even more awesome because if there’s one thing we’ve all noticed, nothing looks as amazing as our beautiful, artisanal, handwritten code when we compare it to AI slop.
Deployed code: code that you can prove is low risk enough to add to the production build.

Hmm, that’s a pretty different standard. And “low risk” is doing a lot of work. Feature flags you really trust? Slow rollout to 1%? Zero chance of customer data loss? Isolated code path only 3 early adopters ever look at? Proven ability to revert within 1 minute of error detection?

There are a lot of different ways to keep risk low enough or protections high enough to add code with a far lower degree of trust, certainty, and review into your deployed systems.

Permanent code vs temporary code, tooling for agentic explorations

OK, well and good, but what happens when — like Troy — deployed code demos start stacking and interacting? What happens when someone else starts depending on it?

Ruh-roh.

Tooling is part of the answer here. The good thing about revision control and having your deployed experiment tied to a feature flag — it is tied to a feature flag, right? — is that you could automate removal of this code from production and archiving it in your demo spot. It would also be pretty trivial to have tests looking for dependencies from outside the demo.

This is a start of the conversation, but one that needs to move quickly if we’re going to really handle the scale of rapid prototyping and AI-assisted coding.