Build or Replaced
Today: The engineering behind the US Strategic Petroleum Reserve | Backups Aren't Simple | Training a 4B model to produce 81% faster query plans than Postgres Episode date: 2026-09-17.
Download MP3 →ERIK: When a system card drops, a release is close. They don't publish these for minor updates. JOSH: It's Thursday, September 17th. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson. ERIK: Today's theme is bad days. AWS lost data it can't get back. Postgres got outrun by a 4-billion-parameter model. Everybody's having a rough week except the guardrails. [beat] JOSH: Stick around — Erik's got an AI pro tip at the end about picking a coding agent for the harness, not just the model. [pause] JOSH: Okay, headlines. ScanBrief scored 99 items across 56 sources overnight — episode 109, and this one's a weird mix. First up, Nvidia just announced native Rust on the GPU. Big deal or nerd trivia? ERIK: Big deal. CUDA Rust compiles straight to PTX, no C wrapper. That's Nvidia admitting the Rust AI stack isn't a fad anymore — it's infrastructure. JOSH: Give me something concrete. What actually breaks or gets better because of that? ERIK: Every memory-safety bug you used to chase in a CUDA kernel written in C — buffer overrun, race condition on a shared block — a chunk of that goes away at compile time instead of at 2 AM when your training run corrupts. I'm not rewriting my pipelines in Rust tomorrow, but the teams writing new kernels from scratch just lost their excuse not to. JOSH: Next — somebody reverse-engineered the signing keys for US driver's license barcodes. That sounds bad. ERIK: It's bad and it's fascinating. If your ID's barcode can be forged, every bar and every TSA checkpoint scanning it is trusting a broken signature. Somebody's patching that this week, I guarantee it. JOSH: And to end on something lighter — cargo ships are putting sails back on. ERIK: Wind's free. Fuel isn't. That's not nostalgia, that's a spreadsheet. [pause] JOSH: Let's go deeper on a couple of these. First one — HarnessTax, how much does the harness matter for coding agents. That's basically your whole job description. ERIK: It's the whole game. Everybody benchmarks the model — which one writes the cleanest function. Nobody benchmarks the harness, and the harness is what actually determines if the agent ships anything real. JOSH: What do you mean by harness, for people who don't live in this? ERIK: The scaffolding around the model. How much context it gets, whether it can actually run your tests, whether a tool call failure gets retried or just silently eats the task. I've got 12 agents running across the Bobaverse fleet right now — Neo, Homer, Bill, Echo, Gandalf. Same underlying models a lot of people use. The difference is PrimeBus. JOSH: PrimeBus is the harness. ERIK: PrimeBus is the nervous system. 140 distinct projects emit telemetry onto that bus. When a test fails, an event fires, an agent picks it up, generates a fix, and it goes through review before it ever touches prod. The model's replaceable. That pipeline isn't. [beat] JOSH: So swap the model and nothing breaks? ERIK: That's the goal. I've changed backend models on PrimeRouter more than once and the pipeline didn't notice, because the contract between the harness and the model never changed. That's the actual moat. Not which model you're renting this month. JOSH: What does that contract actually look like? Like, what has to stay the same when you swap a model underneath it? ERIK: Same interface — call, timeout, retry, structured result. Every backend, whether it's a Claude slot or a local Ollama box, has to speak that exact protocol or the router won't even register it. I learned that one the hard way early on with a test fake that didn't match the real call signature. Passed a hundred tests, broke the first real request. Now every fake has to implement the same contract as production or it doesn't ship. JOSH: And the guardrails — that's not just a nice phrase, right? You've got numbers. ERIK: Real numbers. The auto-merger's run 2,295 attempts since June. 1,507 merged clean. 788 got blocked by Gandalf before they ever hit prod. Zero — zero — escalated to me needing a manual save. That's 65.7% merging on its own, and the other third isn't a failure rate. That's the harness doing its job and catching things before I have to. JOSH: Most people would call 788 blocked a bad week. ERIK: Most people are counting the wrong thing. If I'm not getting paged, the harness worked, whether the code merged or got rejected. That's the whole point of building this instead of just prompting a model directly and hoping. [pause] JOSH: Alright, from harnesses to disasters — literally. AWS is saying it can't restore some data from facilities in the Middle East that got hit during the Iran strikes. And there's a second story today just called "Backups Aren't Simple." Feels connected. ERIK: Painfully connected. People think backups mean a cron job and an S3 bucket. Real disaster recovery means you've actually run the restore, on a schedule, and watched it work end to end. AWS losing regional data means somebody's 3-2-1 strategy had a single point of failure they never tested against. JOSH: You run your own backups on the home lab, right? ERIK: Every night. Nightly push to Gitea, automated, no human in the loop unless something's wrong. But the number that matters isn't "I have a backup job." It's "I restored from it last month and it worked." I test mine. Most companies don't test theirs until the day they need it, and that's the day they find out it doesn't. JOSH: How often do you actually run a full restore test, not just check the job logs? ERIK: Monthly, minimum, on anything I'd call critical. Logs telling you the backup job "succeeded" just means a file got written somewhere. It doesn't mean that file is usable. I've caught corrupted archives that way — job reported green, restore reported garbage. If you're not testing the restore path, you don't have a backup, you have a belief. JOSH: So what would you tell the AWS customers who just lost data? ERIK: I'd tell them what I tell myself — don't trust one cloud, one region, or one vendor's promise. If a strike on a data center can take out your only copy, that copy was never a backup. It was a hope. [beat] JOSH: Last one — this one's close to home for you. A 4-billion-parameter model, fine-tuned, generating Postgres query plans 81% faster than Postgres's own optimizer. ERIK: That one's the most important story on the list today and it's buried at number three. A small, fine-tuned model beating decades of hand-tuned database engineering on a narrow task. That's not a chatbot story, that's an architecture story. JOSH: Why does that matter more than a bigger, smarter model coming out? ERIK: Because it proves the thing I've been building PrimeRouter around — you don't need the biggest model for every job, you need the right-sized model for the specific job, wired into the right pipeline. A giant frontier model doing query planning is overkill. A tuned 4B model that only does one thing well, running fast and cheap, that's the actual future of production AI. JOSH: So bigger isn't always the move. ERIK: Bigger is the expensive default. Specific beats big, almost every time, once you've got the harness to route the request to the right size model. That's the whole bet. JOSH: Would you actually trust a model rewriting query plans on something like PrimeBus's own database, or is that still lab-only for you? ERIK: Lab first, always. Same rule I use for everything — prove it small before it touches anything that pages me at 3 AM. I'd want to see that 4B model wrong on a hard query before I let it near anything with real writes. But the direction is right. Narrow, fast, cheap models doing one job well is where a lot of this is headed, not bigger chat windows. [pause] ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com [beat] JOSH: Alright, what's the AI pro tip today? ERIK: Stop picking your coding agent by which model it's wearing. Pick it by the harness underneath. Ask three questions before you commit to one: does it actually run your test suite, or just claim it did. Does a failed tool call get retried with the error fed back in, or does it just silently give up and hallucinate success. And can you swap the underlying model without rewriting your whole setup. If the answer to any of those is no, you don't have an agent, you've got a chatbot with extra steps. That's your tip. Use it. JOSH: Track your freedom score and net worth with the Freedom Blueprint app — free download, link in the show notes. [beat] JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev. ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff. [pause] ERIK: Build or be replaced. JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.