Build or Replaced
Today: A digestion of the Jacobian conjecture counterexample | FreeInk: Open ecosystem for e-readers | "Drawing" the Mona Lisa with GPT-5.6, Claude, Gemini, and Grok Episode date: 2026-07-22.
Download MP3 →JOSH: It's Wednesday, July 22. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson. ERIK: Today's theme is control. Agents are moving into chat, code, search, and data, and the winners will be the teams with guardrails, not the teams with the loudest demo. JOSH: Stick around — Erik's got an AI pro tip at the end about forcing agents to produce testable handoffs. JOSH: [pause] JOSH: First headline. Jack Dorsey launched Buzz, and it's trying to bring team chat, AI agents, and Git hosting into one workspace. Real shift, or another tool bundle? ERIK: Real shift if the event loop is native. Chat is where the signal starts, Git is where the work lands, and agents need to sit in the middle with permissions and receipts. JOSH: Second. A federal judge approved Anthropic's $1.5 billion settlement over pirated books used to train Claude. What's the builder takeaway? ERIK: Data provenance is now an engineering problem with a legal invoice attached. If your AI product touches content, docs, tickets, books, code, or customer data, you need to know where it came from and what the system did with it. JOSH: Third. Kimi K3 is putting pressure on the big labs, especially around coding and agent work. Is this the model-routing story again? ERIK: Yep. Stop worshipping model logos. Route by job type, cost, latency, context, tool use, and failure mode. The best agent stack is boring under the hood. JOSH: [pause] JOSH: Let's start with Buzz. Team chat, agents, and Git in one place. Why does that hit a nerve? ERIK: Because that is where software work actually happens, but today it's split into pieces. Somebody notices a bug in chat. Then they open an issue. Then somebody copies logs. Then a developer asks Claude or ChatGPT for help. Then code goes into GitHub. Then CI fails in another tab. Then the same person pastes the failure back into chat. JOSH: That's a lot of copying. ERIK: Exactly. We made knowledge work into clipboard farming. Very advanced civilization. JOSH: What should happen instead? ERIK: A message lands in chat and becomes an event. That event has context attached: repo, branch, owner, error logs, service name, prior incidents, tests, permissions. An agent subscribes to that event, runs the safe checks, proposes a fix, opens a pull request, and posts the result back where the conversation started. JOSH: So chat isn't the product. It's the trigger surface. ERIK: That's the key. Chat by itself is just another place for work to get lost. Chat plus event routing plus agents plus Git becomes an operating surface. JOSH: How close is that to what you run? ERIK: Pretty close in shape. PrimeBus is the nervous system in my lab. Services emit events. Agents subscribe. NATS moves the messages. Claude handles code work where it makes sense. Guardrails decide what can move forward. JOSH: You always say guardrails before autonomy. ERIK: Because autonomy without guardrails is just letting a confident intern run production with root access. No thanks. JOSH: What do the guardrails actually check? ERIK: Tests. File scope. Diff size. Dependency changes. Whether the agent touched secrets. Whether the fix is related to the failure. Whether a second variant did better. Whether the repo has a clean enough state to merge. Basic adult supervision. JOSH: That's less magical than people expect. ERIK: Good. Magic is hard to debug. I want boring systems that produce logs, decisions, and artifacts. PrimeDash lets me see what's alive on the LAN. PrimeRestorer handles backups. PrimeBus carries the events. The agent is one worker in the machine, not the whole machine. JOSH: Where does Buzz win if it works? ERIK: It wins by killing context loss. A customer issue, a production alert, a failing test, and a code review should all have a shared thread. The agent should know the thread. The repo should know the thread. The merge decision should point back to the thread. JOSH: That sounds like ChatOps finally got teeth. ERIK: That's exactly it. ChatOps was command buttons in a room. Useful, but still mostly humans pushing buttons. Agentic ChatOps is the system reading the event, choosing the runbook, checking state, and giving the human a decision with evidence. JOSH: What would that look like in a network team? ERIK: Alert comes in for a BGP flap. The system pulls telemetry, checks recent config, queries NSO, looks at maintenance windows, compares neighbors, and posts the likely cause. If the runbook says the fix is read-only or pre-approved, it can stage the change. If it's risky, it asks the engineer with the output already attached. JOSH: That's the dream for NOC work. ERIK: Not a dream. It's plumbing. Hard plumbing, but still plumbing. The trick is not asking a model to be a network engineer. The trick is giving it constrained tools and making it prove every step. JOSH: [pause] JOSH: The Anthropic settlement feels like a different kind of story, but it hits builders too. Why? ERIK: Because it tells you the AI market is leaving the toy phase. Training data, licensing, retention, audit logs, copyright, customer data. These are not policy slides anymore. They affect whether your product gets bought, sued, blocked, or trusted. JOSH: Is this only a frontier lab problem? ERIK: Big labs get the headline lawsuit. Smaller companies get killed in procurement. Same category, different weapon. JOSH: What do buyers ask now? ERIK: They ask what model you use, where prompts go, whether data is retained, whether their data trains anything, what logs exist, who can access them, and whether your agent can take action without approval. If your answer is "we use AI safely," you're not ready. JOSH: What's a real answer? ERIK: Separate training, retrieval, and inference. Training changes the model. Retrieval gives context at runtime. Inference is the actual call. A lot of people blur those together because the demo works either way. Procurement does not care that your demo worked. JOSH: Give me the practical version. ERIK: Keep a data inventory. Know your sources. Tag sensitive material. Log every model call that matters. Store prompts and outputs when policy allows. Keep customer data out of training unless the contract clearly allows it. Use retrieval for internal docs instead of pretending you need to train a model on everything. JOSH: Where do agents make that risk worse? ERIK: Agents can touch more surfaces. A chatbot answers a question. An agent reads a repo, opens files, calls APIs, writes code, sends email, updates a ticket, maybe triggers a deploy. That means your audit trail needs to cover actions, not just text. JOSH: So the agent needs a paper trail. ERIK: Receipts. Always receipts. Who triggered it, what context it saw, what tools it called, what changed, what tests ran, what policy allowed it, and what got blocked. JOSH: That's not glamorous. ERIK: Neither are backups, until you need one. PrimeRestorer is boring on purpose. DBs, system state, offsite copies. Nobody claps for backup orchestration. Then the disk dies and suddenly boring is the best thing you ever built. JOSH: How does this change product design? ERIK: Build the audit layer early. Don't staple it on later. If the first version of your AI product can't show what happened, version ten will be worse. Every shortcut becomes architecture. JOSH: What about open models? Some people use them to avoid the data issue. ERIK: Open weights help with control, but they don't magically solve provenance. You still need to know what data you feed them, what outputs you store, and where they run. Hermes runs local models for me because local control matters. But local does not mean lawless. JOSH: That's a good line. ERIK: It's true. "Runs on my machine" is not a compliance program. JOSH: [pause] JOSH: Let's talk Kimi K3 and routing. You said stop worshipping model logos. What does that mean in practice? ERIK: It means the model is a component. Claude might be better for one code review. GPT might be better for a planning task. A local qwen3-coder model on Hermes might be good enough for a narrow patch or log summary. The system should choose based on the job. JOSH: Most people just pick one model and stick with it. ERIK: That's fine when you're starting. It's bad when you're building a system that runs all day. Different tasks have different budgets. A simple classification should not go to the most expensive frontier model. A risky code change should not go to the cheapest model just because it's fast. JOSH: What's the routing decision based on? ERIK: First, task type. Is it summarizing, coding, planning, extracting, checking policy, writing tests, or making a merge decision? Second, risk. Can it change production? Can it touch money? Can it email a customer? Third, context size. Fourth, latency. Fifth, cost. Sixth, historical performance. JOSH: Historical performance meaning your own results? ERIK: Yes. Benchmarks are useful, but your workload is the truth. ScanBrief has news ranking behavior. PrimeBus has code repair behavior. Hermes has local coding behavior. I care how models perform on my tasks, with my prompts, my repos, my tools, and my failure cases. JOSH: How do you keep that from getting messy? ERIK: Make routing explicit. Don't hide it in vibes. Define a router that takes a task envelope. The envelope says what the task is, what tools are allowed, what data classification applies, what timeout exists, and what success means. JOSH: Success means tests? ERIK: Sometimes. For code, yes, tests and diff checks. For summaries, it might mean citations and no missing required fields. For extraction, schema validation. For network diagnostics, command output and a runbook match. The point is the model doesn't decide if it was successful. The system does. JOSH: That's the part people skip. ERIK: Yep. They ask the model, "Did you fix it?" and the model says yes because it is a helpful little liar when you let it grade its own homework. JOSH: Brutal. ERIK: Fair. Models are useful. I use them every day. But you need a harness. Give the model a task, tools, limits, and a judge that is not the same model when the stakes are high. JOSH: Where does Kimi K3 fit into this? ERIK: It adds pressure. When a cheaper or open-weight model gets good at coding tasks, the routing table changes. That doesn't mean you fire your current stack. It means you test it against your work and move traffic where it earns it. JOSH: So builders should run their own evals. ERIK: Absolutely. Tiny evals. Real evals. Take ten bugs from your repo. Ten tickets from your queue. Ten logs from outages. Ten customer emails with private data removed. Run them through the models you care about. Score output against actual acceptance criteria. JOSH: Not a giant academic benchmark. ERIK: Nope. Your benchmark should look like your Tuesday morning. If a model can't handle your Tuesday morning, I don't care what it scored on a leaderboard. JOSH: What about cost? ERIK: Cost matters once the system runs without you babysitting it. A bad manual process is expensive in labor. A bad agent process is expensive in tokens and cleanup. Routing lets you spend real model money only where it changes the outcome. JOSH: Does local inference change that? ERIK: It gives you another lane. Hermes on the Mac M3 is great for local coding experiments and private work where I want control. It won't replace every frontier call, and it doesn't need to. It catches the jobs that don't need the big model. JOSH: What's the pattern for a small team? ERIK: Start with three lanes. Cheap lane for classification and formatting. Strong lane for complex reasoning and code. Local or private lane for sensitive context. Add logging before you add clever routing. You can't route what you can't measure. JOSH: That sounds like infrastructure work more than prompt work. ERIK: It is. Prompting matters, but the moat is the system around the prompt. Events, permissions, evals, logs, rollback, human approval. That's the build. JOSH: [pause] ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com JOSH: [pause] JOSH: Alright, what's the AI pro tip today? ERIK: Force every agent to produce a handoff artifact before it acts. Not a paragraph. A structured artifact. For code, I want fields like problem, files inspected, files changed, tests run, risk, rollback, and next command. For operations, I want service, symptom, evidence, command output, proposed action, approval needed, and blast radius. JOSH: Why does that help? ERIK: Because it makes the agent show its work in a way another system can check. If the handoff says no tests ran, the merge guardrail can stop it. If the blast radius is unknown, the change needs a human. If the files changed don't match the issue, block it. JOSH: So the tip is make the output machine-checkable. ERIK: Exactly. Free-form text is fine for humans. Systems need structure. Tell the agent the schema. Reject invalid output. Make the next step depend on that artifact. That's how you turn an AI answer into an automation workflow. That's your tip. Use it. JOSH: [pause] ERIK: If you're building toward financial independence through automation, my first book walks through the whole path. Free chapter at erikandersonbook.com. JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev. ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff. JOSH: [pause] ERIK: Build or be replaced. JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.