Mon–Fri · 6 AM ET
← All Episodes
EP  • 00:11:54

Vulnerability reports are not special anymore | Build or Be Replaced

Today: Vulnerability reports are not special anymore | In memory of the man who put red and green squiggles under words | FUTO Swipe – A new swipe typing model Episode date: 2026-06-24.

Download MP3 →

Transcript

JOSH: It's Wednesday, June 24. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson.
ERIK: Today's theme is boring systems winning, because boring systems are the ones still running at 3 AM.
JOSH: Stick around — Erik's got an AI pro tip at the end about making agents write their own failure receipts.
[pause]
JOSH: First headline. Vulnerability reports are getting treated like regular issues now. Is that good or bad?
ERIK: Both. Security reports used to get special ceremony. Now maintainers are buried, and a vuln ticket competes with every other broken build, flaky test, and angry user.
JOSH: That's not comforting.
ERIK: Nope. But it's honest. If your process depends on humans noticing one special GitHub issue faster than the other thousand, that's not a process. That's a wish with a CVE number.
[beat]
JOSH: Next up, a quieter one. Tony Krueger, the Microsoft Word engineer behind the red and green squiggles, passed away.
ERIK: That feature changed software. Real-time spell check sounds normal now, but at the time it moved correction from a separate task into the flow of writing. That's automation done right. It doesn't hold a meeting. It just shows you the problem while your brain is still in the sentence.
[beat]
JOSH: Third headline. Meta paused an employee-tracking program after an internal data leak. What do you make of that?
ERIK: If you track people, the tracking data becomes production data. Then it needs access control, retention policy, audit logs, blast-radius thinking, all of it. Companies love surveillance until they have to secure the surveillance.
[pause]
JOSH: ScanBrief scored 98 items across 54 sources today. Those were the top signals. Which one is the real builder story?
ERIK: The vulnerability report one. Easy. Because it exposes a lie in software teams. Everybody says security is priority one, but then they route security through the same tired human queue as everything else.
JOSH: What's the better pattern?
ERIK: Treat the report like an event, not a message. Big difference. A message says, hey, someone should look at this. An event says, this happened, and a system starts moving.
[beat]
ERIK: In my world, PrimeBus processed 978 automation events across 16 projects today. That's not a dashboard vanity number. That's code reviews, failures, deploy checks, backups, alerts, little decisions moving without me staring at Slack like it's a campfire.
JOSH: So a vulnerability report should kick off a workflow.
ERIK: Exactly. First step, classify it. Is it dependency, auth, injection, data exposure, network edge, secrets, or supply chain. Then pull context. What repo. What service. What owner. Is there a public exploit. Is it internet-facing. Do we have logs showing hits. Do we have tests that cover the path.
JOSH: That's a lot for a maintainer to do manually.
ERIK: That's why it doesn't happen. People are not lazy. The system is lazy. It waits for one burned-out maintainer to become a security triage engine after dinner.
[beat]
JOSH: But open source maintainers don't all have PrimeBus sitting there.
ERIK: They don't need my setup. Start smaller. GitHub issue label triggers a workflow. Workflow runs dependency scan. Workflow posts a structured summary. If there's a patch, it opens a branch. If there isn't, it creates a minimal reproduction template. If the report is vague, it asks for missing fields automatically.
JOSH: That sounds like taking the drama out of it.
ERIK: Good. Drama is expensive. The report should become structured data as fast as possible.
[pause]
JOSH: Where does AI fit without making it risky?
ERIK: AI should not be the final judge on a vuln. It should be the first clerk. Read the report. Extract claims. Map affected files. Search for similar code paths. Draft a test. Draft a patch. Then a policy engine decides what can merge.
JOSH: Policy engine meaning Gandalf in your case.
ERIK: Yeah. Gandalf reviewed and merged 10 code changes to production overnight. That's the part people miss. The agent can write code, but the guardrail decides what is allowed to land.
[beat]
ERIK: Since June 5, the PrimeBus auto-merger has run 1338 attempts. 876 merged, 462 blocked by Gandalf, 0 escalated to me. That blocked number is the story. Guardrails doing their job is what lets you sleep.
JOSH: Wait, really? Zero escalated?
ERIK: Zero. Because the system knows when to say no. That's what most agent setups are missing. They build the part that acts, then forget the part that refuses.
JOSH: So for security, refusal is a feature.
ERIK: Huge feature. If the agent proposes touching auth middleware and deleting a failing test, that's not clever. That's a clown car with a pull request. Block it. Ask for a narrower fix. Make it prove the vuln and prove the patch.
[pause]
JOSH: The second story I want to hit is the Word squiggle one. It feels tiny, but you lit up when you saw it.
ERIK: Because it's the perfect automation lesson. The red squiggle didn't replace the writer. It replaced a dumb interruption. Before that, spell check was a mode. Stop writing. Run spell check. Click through a dialog box. Lose your thread.
JOSH: And now it just sits under the word.
ERIK: Right. Non-blocking feedback. That's the magic. It doesn't make you stop. It doesn't steal the keyboard. It gives you a signal at the right time, in the right place, with low friction.
[beat]
JOSH: How does that map to infrastructure?
ERIK: Everything. Network automation should feel like that. If I push an NSO service package and a device template has a bad variable, I don't want a weekly report. I want the system to underline the broken part while the change is still fresh.
JOSH: The red squiggle for Terraform.
ERIK: Yes. Terraform plan drift. Kubernetes manifest risk. NSO dry-run mismatch. NATS consumer lag. Selenium test flake. Whatever. Don't make engineers go hunting. Put the signal where the work is happening.
JOSH: That's a cleaner way to say observability.
ERIK: Observability got bloated. Half the time it means a hundred charts nobody reads. The squiggle model says, surface the smallest useful warning directly in the workflow.
[pause]
JOSH: Give me a real example from your systems.
ERIK: The PAS website-pipeline is a good one. It builds and maintains agency websites, but the important part isn't the page generator. The important part is the review loop. Broken link, bad image, layout shift, missing form action, Lighthouse issue, deployment error. Those should not become a giant task list.
JOSH: They become squiggles.
ERIK: Exactly. Small visible corrections. The agent can fix a missing alt tag. It can rerun a screenshot test. It can check the form endpoint. But if it sees payment code or DNS, it slows down and asks for review.
[beat]
ERIK: Same pattern with PrimeRestorer. Backups are boring until they're not. The system shouldn't send me a novel. It should say, this database backup passed, this S3 offsite copy passed, this one restore check failed, and here's the exact command that failed.
JOSH: That's the difference between a warning and a useful warning.
ERIK: Yes. Most alerts are just guilt with timestamps.
JOSH: That's painfully accurate.
ERIK: You want the squiggle. Not the guilt.
[pause]
JOSH: There's also the FUTO Swipe story. A million swipe-typing examples released under MIT. Why should builders care?
ERIK: Data is the moat people ignore until they don't have it. A swipe keyboard model needs real gesture paths, not just words. The release matters because it gives builders a benchmark and training material they can actually use.
JOSH: Is that an AI story or an open data story?
ERIK: Both. Models are hungry, but the good stuff is domain-specific. Generic text is everywhere. Real user swipes are not. Network configs, repair tickets, NOC runbooks, deployment logs, customer emails, failed test diffs. That's where useful automation comes from.
[beat]
JOSH: So the lesson is collect your own data?
ERIK: Collect it cleanly. Big difference. Dumping logs into a folder is not a data strategy. You need schema. You need timestamps. You need source. You need outcome. Did the fix work. Did the customer reply. Did the restore pass. Did the change roll back.
JOSH: That's where ScanBrief fits?
ERIK: Yeah. ScanBrief isn't just grabbing headlines. It scores, dedupes, ranks, and keeps the source trail. Today it scored 98 items across 54 sources. That gives me a daily signal layer I can trust enough to build from.
JOSH: And HumanRail?
ERIK: HumanRail is the other side. When the agent isn't confident, route to a human. But the routing event still becomes data. What did the model think. Why did it ask. What did the human decide. Next time the system is smarter, or at least less annoying.
[pause]
JOSH: What do people get wrong when they hear "dataset"?
ERIK: They think it means giant. It means useful. A hundred high-quality examples from your actual workflow beats a million random examples scraped from nowhere.
JOSH: That's uncomfortable for teams that never write anything down.
ERIK: Good. It should be. If your company has no examples, no runbooks, no outcomes, and no clean history, AI won't magically understand the business. It'll produce confident fog.
[beat]
JOSH: Confident fog is a strong phrase.
ERIK: It's everywhere. The model writes a summary, everybody nods, nobody checks whether it maps to reality. Then someone puts it in prod and acts shocked when it invents a field name.
JOSH: How do you avoid that?
ERIK: Force every agent output to attach evidence. File path. log line. test result. ticket ID. source URL. The moment an agent can't show its work, it becomes a draft, not a decision.
[pause]
JOSH: Bring these three together for me. Vulnerability reports, red squiggles, swipe data. What's the common thread?
ERIK: Feedback loops. That's it. Vulnerability reports need a loop from report to context to test to patch to review. The Word squiggle is a loop from mistake to signal to correction without breaking flow. FUTO Swipe is training data for a loop between human motion and predicted text.
JOSH: So builders should be designing loops, not tools.
ERIK: Tools are easy. Loops are where the money is. A tool waits for you. A loop moves the work, checks the result, and leaves evidence.
[beat]
ERIK: That's why PrimeBus matters in my setup. It isn't one app. It's the nervous system. 111 distinct projects have emitted telemetry to it. That means my systems can react to each other instead of sitting around like bored interns.
JOSH: And that's how you get daily shipping without watching everything manually.
ERIK: Right. The production server has 111 services running right now. If I had to babysit that by hand, the show would be called Erik Needs a Nap.
JOSH: I'd subscribe.
ERIK: You already work here.
[pause]
JOSH: Before the tip, sponsor read.
ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com
[pause]
JOSH: Alright, what's the AI pro tip today?
ERIK: Make your agents write a failure receipt every time they can't finish a task.
JOSH: What goes in the receipt?
ERIK: Four fields. Goal. What it tried. Exact blocker. Next safe action. That's it.
[beat]
ERIK: Don't let the agent say, I couldn't complete the task. That's useless. Make it say, I tried running the test suite, dependency install failed because package X needs Node version Y, next safe action is update the runtime or pin the package.
JOSH: So it's not just logging.
ERIK: No. It's a handoff contract. The next agent, or the human, should be able to pick it up without redoing the investigation. Put those receipts on a queue. NATS, Redis, Postgres, whatever you already run. Then route by blocker type.
JOSH: Give us the prompt.
ERIK: Add this to your agent instruction. When blocked, do not apologize. Produce a failure receipt with goal, attempts, blocker, evidence, and next safe action. Include exact commands, file paths, and error text. If the next action changes production, mark it requires human approval.
[beat]
ERIK: That's your tip. Use it.
[pause]
ERIK: If you're building toward financial independence through automation, my first book walks through the whole path. Free chapter at erikandersonbook.com.
[pause]
JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev.
ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff.
[pause]
ERIK: Build or be replaced.
JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.