Build or Replaced
Today: Our position on open-weights models | Netflix employee fired for sharing personal details in retreat trust exercise | Benchmarking Opus 5 on SlopCodeBench Episode date: 2026-07-28.
Download MP3 →JOSH: It's Tuesday, July 28. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson. ERIK: Open models are the pressure valve. If the big labs close the door, builders find another door. JOSH: Stick around — Erik's got an AI pro tip at the end about using cheap models as test pilots before you spend frontier-model money. [pause] JOSH: First headline. Anthropic's Dario Amodei says the company does not want to ban open-weight models. Erik, why did that matter today? ERIK: Because the argument got lazy. Open weights are not automatically dangerous, and closed models are not automatically safe. The right fight is security, evals, and controlled release, not pretending only giant vendors can be trusted. JOSH: Second headline. A five-hundred-dollar RL fine-tune of a 9B open model reportedly beat frontier models on catalog review. Wait, really? ERIK: Yeah, and that should make every builder sit up. Narrow task, good data, clear reward signal. That's where small models punch way above their weight. JOSH: Third headline. Rootless containers are back in the feed. Is that just Linux people arguing again? ERIK: Partly. That's their cardio. But rootless containers matter because the boring security layers are what let automation run without turning your server into a crime scene. [pause] JOSH: The big story today is open weights. Anthropic saying, basically, we don't support banning them. What's your read? ERIK: My read is that they had to say it plainly. There's been this weird vibe where every conversation about open models turns into, "Should anyone be allowed to download weights?" That's the wrong question. ERIK: The right question is, what can the model do, who can access it, what safeguards exist around distribution, and how do we test it before release? ERIK: Non-dangerous open-weight models are good for builders. Full stop. They give you control. They give you a way out when pricing changes. They let you run local. They let you build workflows where your data doesn't leave your box. JOSH: But the scary version is still scary, right? ERIK: Of course. A capable open model with dangerous cyber or bio capability is not the same thing as a coding assistant you can run on a Mac. Pretending those are the same category is how bad policy gets written. JOSH: How does that hit your world? ERIK: Hermes is exactly why I care. That's my local agentic LLM backend on the Mac M3. It runs qwen3-coder, and it's not trying to be a magic oracle. It handles local tasks, code review prep, small agent calls, and cheap routing before I burn expensive tokens. ERIK: That changes the design. I don't need every request to go to Claude. Some jobs are local. Some jobs need Claude. Some jobs need GPT. PrimeBus decides where the work goes. JOSH: So open weights are part of the routing plan. ERIK: Exactly. Model selection becomes infrastructure. Same way you don't run every packet through the same firewall rule if the traffic is different. You classify the job. ERIK: ScanBrief scored 92 items across 54 sources today. That pipeline doesn't need a frontier model on every step. Fetching, dedupe, rough scoring, clustering, those can be cheap. The final judgment and script framing can use the stronger model. JOSH: That's a money thing? ERIK: Money, latency, privacy, and control. Pick all four. ERIK: If you're building real systems, you want options. Closed model for the hard reasoning. Open model for local code edits. Small model for filters. Embeddings for memory. Selenium for browser work. NATS for events. Terraform for repeatable infra. Kubernetes if you enjoy meetings. [beat] JOSH: That sounded personal. ERIK: Kubernetes is fine. People are the issue. [pause] JOSH: The five-hundred-dollar fine-tune story feels related. A 9B model beats frontier models on catalog review. How? ERIK: Narrow problem. That's the whole trick. Catalog review is not general intelligence. It's pattern recognition, policy matching, and consistent judgment over repeated examples. ERIK: Give a smaller model the right examples and a tight reward signal, and it can beat a giant general model that is trying to be good at everything. JOSH: So this isn't saying small models are better than frontier models. ERIK: No. It says most companies are wasting money asking giant models to do tiny jobs. ERIK: If your task is "review this product title and decide if it violates our catalog policy," you don't need a philosopher. You need a very consistent clerk who never gets tired and can cite the rule. JOSH: That sounds like automation work more than AI magic. ERIK: It is. The AI part is maybe twenty percent. The system around it is the rest. Queue, schema, test set, evals, approval path, logs, rollback, cost tracking. That's where the actual engineering lives. ERIK: PrimeBus is a good example. The auto-merger has run 2295 attempts since 2026-06-05: 1507 merged, 788 blocked by Gandalf, 0 escalated to Erik. That's not because the model is perfect. It's because the pipeline has guardrails and the guardrails are allowed to say no. JOSH: The blocked number is the point. ERIK: Exactly. People hear blocked and think failure. Wrong. Blocked means the system caught something before it hit prod. That's the win. JOSH: How would a normal team use the five-hundred-dollar lesson? ERIK: Start with one annoying decision your team repeats every day. Ticket routing. Log classification. Config diff review. Lead scoring. Security exception triage. Something boring enough that everyone hates it, but important enough that mistakes cost money. ERIK: Then build the dataset. Fifty good examples is better than five thousand messy ones. Label what good looks like. Label what bad looks like. Write the rule in plain English. Make the model return JSON. Test it like code. JOSH: And then fine-tune? ERIK: Maybe. Not first. ERIK: First, prompt it. Then try retrieval. Then try a cheaper model. Then run evals. Fine-tune only when the same failure keeps showing up and the task is stable. ERIK: People jump to fine-tuning because it sounds serious. Half the time they need a better prompt, a cleaner schema, and one angry engineer writing tests. [beat] JOSH: One angry engineer is a renewable resource. ERIK: Sadly, yes. [pause] JOSH: You mentioned lead scoring. Does this connect to your real-estate automation work? ERIK: Completely. Real Estate Automation is a narrow-decision machine. Pull the lead, score the property, score the seller signal, decide the next outreach step. ERIK: I don't need a giant model writing poetry to a homeowner. I need the system to say, "This lead has equity, age-of-ownership looks interesting, outreach should be postcard first, email second." Then it logs the decision. JOSH: And if it's unsure? ERIK: Route it. HumanRail pattern. Low confidence goes to a person. High confidence goes through. Medium confidence gets another model opinion. ERIK: That's how you make AI useful without pretending it's a person. Give it a lane. Give it a brake pedal. Make every action observable. JOSH: Is that where teams mess up? ERIK: They give the model a vague job. "Help with operations." What does that mean? Nothing. Give it a specific contract. ERIK: "Given this ticket, classify the service, severity, likely owner, and next runbook. Return valid JSON. If confidence is under point seven, escalate." Now we can work with that. JOSH: That's much less glamorous. ERIK: Good. Glamour gets paged at 2 AM. [pause] JOSH: Third deep story. Rootless containers. Why should builders care? ERIK: Because the future is agents running commands, and agents running commands need less power, not more. ERIK: Rootless containers let workloads run without root privileges on the host. That doesn't solve every security problem, but it removes a whole class of bad outcomes. JOSH: This is about agent safety? ERIK: Agent safety, CI safety, backup jobs, scrapers, local tools, all of it. If Claude writes a script and PrimeBus runs it, I want that script boxed in. ERIK: Same thing with Selenium jobs. Browser automation is messy. Pages change. Downloads happen. Weird JavaScript runs. Run it in a container with tight permissions. Give it a scratch directory. Kill it after the job. Log the result. JOSH: Where does PrimeRestorer fit here? ERIK: PrimeRestorer is the boring adult in the room. It handles backup orchestration for DBs, system state, and S3 offsite. When you're letting agents touch code and infra, backups stop being a checkbox. They become part of the workflow. ERIK: Before a risky change, snapshot. Before a migration, dump. Before a deploy, confirm restore path. If you can't restore it, you don't own it. JOSH: That's a hard line. ERIK: It has to be. People talk about self-healing systems like the system just magically fixes itself. No. It needs known-good state, telemetry, a rollback path, and permission boundaries. ERIK: The production server has 132 services running right now. That is not the place for cowboy automation. Every service needs logs. Every automation needs a scope. Every agent needs a leash. JOSH: And rootless containers are one of those leashes. ERIK: Yep. Not the only one. But a good one. ERIK: Use rootless containers for untrusted tasks. Use separate service accounts. Keep secrets out of the container unless the task needs them. Mount read-only when possible. Emit events to NATS. Make the agent ask for a human when the blast radius is too big. JOSH: This feels like the opposite of "give the agent your laptop and hope." ERIK: That's because hope is not an architecture. [beat] JOSH: Put that on a sticker. ERIK: Only if the sticker has a rollback plan. [pause] JOSH: There's also a Go garbage collector story today. Watching the collector move through the heap. Does that matter outside Go nerd circles? ERIK: It matters because runtime behavior becomes visible. When you can watch memory move, you understand pauses, allocation patterns, and why your service acts weird under load. ERIK: Builders should care because AI systems are not just prompts. They are long-running workers, queues, parsers, browsers, model clients, and databases. Memory leaks still exist. CPU spikes still exist. Bad retry loops still exist. JOSH: Even with all the AI. ERIK: Especially with the AI. Agents are very good at generating activity. Activity is not progress. ERIK: If your agent retries the same broken API call two hundred times, congratulations, you built a tiny denial-of-wallet machine. JOSH: That's painfully specific. ERIK: I've seen enough logs. ERIK: The fix is not complicated. Put budgets on loops. Put timeouts on calls. Put max tokens on requests. Put circuit breakers on services. Make the worker report why it stopped. JOSH: How does that show up in PrimeBus? ERIK: Everything emits telemetry. 138 distinct projects have emitted telemetry to PrimeBus. That gives me a map. When something is noisy, I can see it. When an agent is stuck, I can see it. When Gandalf blocks a merge, I know why. ERIK: That's the difference between automation as a pile of scripts and automation as a system. Scripts hide. Systems talk. [pause] JOSH: Pulling this together, what's the builder's take today? ERIK: Open models are not the enemy. Unbounded automation is the enemy. ERIK: Use open weights where they make sense. Use frontier models where the task deserves it. Fine-tune small models only after you've proven the task is narrow and repeatable. ERIK: Then wrap the whole thing in boring infrastructure. NATS events. Containers. Service accounts. Backups. Evals. Logs. Human approval when needed. ERIK: That's how you build something that runs while you sleep without giving it the keys to the building. JOSH: And ScanBrief found all this before breakfast. ERIK: ScanBrief did the sorting. The builder still has to think. Annoying, but apparently required. [pause] ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com [pause] JOSH: Alright, what's the AI pro tip today? ERIK: Before you pay for the expensive model, run a cheap model as the test pilot. ERIK: Take the same task and make the cheap model produce a plan, a schema, or a first-pass classification. Then send only the hard cases to Claude or GPT. ERIK: Example. Ticket comes in. Small model tags service, severity, and owner. If confidence is high, it files the result. If confidence is low, Claude reviews it. If Claude is still unsure, route to a human. ERIK: You save money, you get faster response time, and you also get a confidence signal you can measure. ERIK: Do not make it vague. Use JSON. Require confidence. Log the model name. Store the input and output. Review the misses once a week. ERIK: That's your tip. Use it. [pause] JOSH: We also drop daily market picks and automation tips on YouTube — search Build or Be Replaced. [pause] JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev. ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff. [pause] ERIK: Build or be replaced. JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.