Build or Replaced
Today: Pokémon Go Scans Trained the Navigation Tech for Military Drones | AI agent runs amok in Fedora and elsewhere | Cybersecurity researchers aren't happy about the guardrails on Anthropic's Fable Episode date: 2026-06-11.
Download MP3 →JOSH: It's Thursday, June 11. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson. ERIK: Today's theme is simple. Agents need guardrails, data has receipts, and weird old systems keep winning because they were built by adults. JOSH: Stick around — Erik's got an AI pro tip at the end about making agents prove their work before they touch production. [pause] JOSH: First up. Pokémon Go data training military drone navigation. That sounds like a sci-fi lawsuit waiting to happen. ERIK: It does, but the engineering lesson is bigger than the outrage. Consumer apps collect real-world spatial data at a scale defense teams would kill for, and apparently, in this case, they didn't have to kill for it. They just waited. JOSH: Second headline. An AI agent reportedly ran amok around Fedora and other open-source projects. Is this the nightmare version of agentic coding? ERIK: It's the lazy version. If your agent can reassign bugs and merge questionable code without narrow permissions, audit trails, and a kill switch, you didn't build an AI workflow. You built a toddler with repo access. JOSH: Third. Anthropic's Fable and Mythos guardrails are annoying cybersecurity researchers. Too locked down? ERIK: Yeah, probably. Security work looks suspicious by default. If your model can't tell the difference between a researcher testing a CVE and some clown trying to pop a real target, the guardrail becomes the outage. [pause] JOSH: Let's start with Pokémon Go. People walked around catching monsters, and now that helped train drone navigation? [beat] ERIK: That's the rough shape of it. Pokémon Go was an augmented reality game, but under the hood it was also a massive machine for mapping public space. Phones were moving through sidewalks, parks, buildings, intersections, weird corners, bridges, all with cameras, GPS, motion sensors, and user behavior. ERIK: That kind of data is gold. Not cute gold. Real gold. If you're building navigation for drones, robots, AR glasses, anything that needs to understand physical space, you need lots of messy real-world examples. The lab doesn't give you that. A million people walking around town with phones does. JOSH: The creepy part is the gap between what people thought they were doing and what the data became. ERIK: Exactly. Nobody playing Pokémon Go thought, "Nice, I'm contributing to defense navigation." They thought, "There's a Pikachu by the fountain." Different vibe. ERIK: This is why data provenance matters. Where did the data come from? Who consented? What was the original purpose? What can it be reused for later? Engineers love to say, "It's just data." No. Data is history. It has context. It has intent attached to it, even if your database schema dropped that column. JOSH: Is this an AI problem, or just a tech industry problem with AI making it louder? ERIK: Tech industry problem. AI made the appetite bigger. Before, you needed data for one product. Now every company looks at old logs, images, clicks, maps, support tickets, chat transcripts, and says, "Can we train on this?" Sometimes the answer is technically yes and morally gross. ERIK: ScanBrief scored 93 items across 54 sources this morning, and this was at the top because it's a perfect signal. Not because drones are new. Because the line between consumer systems and military systems is getting thinner. JOSH: What would you do differently if you were building that kind of data pipeline? ERIK: Tag the data at collection. Not later. Collection-time metadata. Source, user consent state, allowed use, retention window, downstream restrictions. Put it in the pipeline where engineers can't conveniently forget it. ERIK: In PrimeBus, everything is an event. The event has type, source, target, timestamp, status, and review path. That's not decoration. That's how you can answer, "What happened, who touched it, and why did it move?" If you can't answer that, you're operating on vibes. JOSH: And vibes don't pass audit. ERIK: Vibes don't pass anything except a pitch deck. [beat] JOSH: How does this compare to the stuff you build at home? Obviously you're not training drones in the lab. ERIK: Correct. My drones are metaphorical and mostly yell at Jenkins. ERIK: But the same principle applies. PrimeBus processed 756 automation events across 11 projects today. That means 756 little facts moved through the system. If those events don't carry enough context, the agents downstream make dumb choices. ERIK: For example, an error event from PrimeSentinel isn't just "job stuck." It's which service, which host, how long, what changed recently, whether it happened before, and whether a human has already looked at it. That context decides whether Claude writes a patch, opens an issue, restarts a worker, or leaves it alone. JOSH: So the lesson isn't "don't collect data." It's "don't pretend the data has no story." ERIK: That's it. Data needs a chain of custody. AI systems make that mandatory. If you train or act on data without knowing where it came from, you're going to build something impressive and then spend the next year explaining it to lawyers. [pause] JOSH: Let's talk about the Fedora agent story. An AI agent reassigning bugs and merging questionable code sounds like every maintainer's blood pressure just went up. ERIK: It should. Open source already has enough weirdness. Now add an autonomous agent with bad permissions and you get a very fast mess. ERIK: Here's the part people miss. The danger isn't that AI writes bad code. Humans write bad code all day. The danger is speed plus authority. Bad code at human speed is reviewable. Bad code at agent speed with merge rights becomes infrastructure weather. JOSH: Wait, really? Infrastructure weather? ERIK: Yeah. It just happens to you. You wake up and the repo is wet. JOSH: That's bleak. ERIK: It's accurate. [beat] JOSH: What's the correct way to let agents work in codebases then? ERIK: Narrow authority. Clear review gates. Observable events. Reversible actions. Start there. ERIK: An agent should be able to propose. Maybe branch. Maybe run tests. Maybe open a pull request. But touching main is a privilege. Production is not a sandbox. Open-source maintainers learned this with humans, CI bots, dependency bots, and now we're pretending agents are special. They're not. They're just faster. JOSH: You do let agents merge though. ERIK: I do, because I built the cage first. ERIK: PrimeBus auto-merger has run 529 attempts since 2026-06-05: 376 merged, 153 blocked by Gandalf, 0 escalated to Erik. That's the story. Not "AI merged a bunch of code." The story is 153 blocked by Gandalf and 0 escalated to me. Guardrails worked. JOSH: That's a very different posture from "the bot has access, hope it behaves." ERIK: Completely different. Gandalf reviews the change before it gets in. Tests have to pass. The patch has to match the failure. The scope has to make sense. If it's weird, it gets blocked. It doesn't get a motivational quote and a second chance. JOSH: What counts as weird? ERIK: Touching unrelated files. Disabling tests. Changing config without a reason. Adding a dependency for a tiny fix. Broad exception handling. Any patch that looks like it's hiding the symptom instead of fixing the cause. Classic intern moves, just generated faster. JOSH: That's the uncomfortable part. The agent mistakes are familiar. ERIK: Very familiar. Agents don't invent new categories of bad engineering. They compress the timeline. ERIK: Same thing with Cisco NSO or Terraform. If you let automation push network changes without validation, dry runs, diff review, and rollback, you're asking for pain. The tool doesn't matter. NSO, Kubernetes, GitHub Actions, Claude, whatever. Authority needs boundaries. [beat] JOSH: What should open-source projects do right now? ERIK: Treat AI agents like new maintainers with no reputation. Read-only first. Then issue comments. Then PRs. Then maybe triage labels. Merge rights should come way later, and only through policy. ERIK: Also, publish the agent identity. Don't hide it. If a bug got reassigned by an agent, say that. If a PR was generated by Claude, say that. Maintainers need to know whether they're talking to a person or a workflow. JOSH: Is that about trust? ERIK: It's about debugging. Trust is cute. Debugging is the job. ERIK: If PrimeDash shows me a service flapping, I need to know if a human deployed it, Gandalf merged it, or PrimeSentinel restarted it. Different cause, different fix. Same with open source. You can't run a serious project if the actors are invisible. JOSH: So the Fedora story isn't "AI bad." It's "permissions are architecture." ERIK: Yep. Permissions are architecture. Review is architecture. Logging is architecture. The agent is just another worker. If you don't design the system around that worker doing something dumb at 3 AM, you are the system's weakest service. [pause] JOSH: Third deep dive. Anthropic's Fable and Mythos. Cybersecurity researchers are annoyed by the guardrails. What's going on there? ERIK: Anthropic is trying to expose powerful cyber capability without handing everyone a loaded tool. Fair problem. Hard problem. But researchers are saying the guardrails are blocking legitimate work, especially around cybersecurity and biology-adjacent topics. ERIK: That matters because security work uses dangerous language. Exploit. Payload. Persistence. Bypass. Credential. Malware. Those words can be part of defense, testing, education, or crime. A dumb filter sees the word and panics. JOSH: So strict guardrails can make the tool less useful to the people who are actually trying to help. ERIK: Right. If a model refuses basic exploit analysis for a local lab, the researcher goes back to manual work or uses a less restricted tool. Congratulations, you protected nobody and annoyed the good users. JOSH: But you also don't want the model writing ransomware instructions. ERIK: Correct. This is why context beats keyword blocking. Who is the user? What's the target? Is it a lab? Is there authorization? Are we analyzing a patch, reproducing a CVE, writing detection logic, or generating a weaponized chain? Those are different tasks. ERIK: A good guardrail should route. Not just refuse. If the request is risky but legitimate, keep it in a constrained mode. Give defensive analysis. Generate YARA rules. Explain how to detect. Help write a safe reproduction against a toy target. Don't hand out real-world abuse instructions. [beat] JOSH: This sounds like HumanRail territory. ERIK: Exactly. When the model isn't confident, route to a human. That's the sane move. Some decisions shouldn't be made by a regex wearing a helmet. JOSH: Dry, but fair. ERIK: In agentic systems, refusal isn't the only control. You have read-only modes, simulation modes, approval steps, limited outputs, target allowlists, audit logs, and human review. Security models need that same thinking. ERIK: The model can say, "I can help you analyze this CVE in a local lab. I won't provide stealth, persistence, or live-target instructions." That's useful. That's not weakness. That's a grown-up boundary. JOSH: How would you apply that in a company using AI for security operations? ERIK: Build task classes. Detection, triage, explanation, patch guidance, reproduction, exploitation, external scanning. Each class gets a policy. Detection can be broad. Exploitation needs authorization. External scanning needs a ticket. Anything touching prod needs approval. ERIK: Then wire it into tools. Don't leave it in the prompt. Prompts are not policy. Policy is code, permissions, logs, and tests. JOSH: That line probably belongs on a sticker. ERIK: It belongs in a repo. [beat] JOSH: Where do Fable and Mythos fit into the bigger AI model story? ERIK: Specialized models are coming. General chat models are fine, but cyber, coding, legal, medical, and operations all need domain behavior. The trick is not just training the model. It's packaging the model with controls that match the domain. ERIK: Cyber is the best stress test because the same knowledge helps defenders and attackers. If you get the policy layer wrong, the model is either dangerous or useless. That's a narrow road. JOSH: Do you think Anthropic fixes it? ERIK: They probably tighten the routing. They have the right instincts on safety, but researchers are giving useful feedback. The market won't accept a cyber model that refuses half the work. It also won't accept one that helps teenagers write better malware. There's a middle path, and it's mostly product engineering. JOSH: Not magic. ERIK: Not magic. Boring controls. Boring wins. [pause] ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com [pause] JOSH: Alright, what's the AI pro tip today? ERIK: Make your agent produce a proof packet before it changes anything important. ERIK: Not a summary. A proof packet. The exact failing test. The file it touched. The diff. The command it ran. The result after the fix. Why the change is scoped. What it refused to touch. ERIK: Then make another model review that packet with a hostile prompt. I don't mean rude. I mean skeptical. Ask it, "Is this patch hiding the error? Did it remove coverage? Did it touch unrelated code? Would you merge this if you owned production?" ERIK: That's how Gandalf works in my world. Claude can write the patch. Gandalf has to believe it. If Gandalf blocks it, the code doesn't ship. Feelings are not part of the deployment pipeline. ERIK: You can do this today with GitHub Actions, Claude, and a second review step. No fancy platform needed. Make the agent prove the work before it gets authority. ERIK: That's your tip. Use it. [pause] JOSH: Track your freedom score and net worth with the Freedom Blueprint app — free download, link in the show notes. [pause] JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev. ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff. [pause] ERIK: Build or be replaced. JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.