Transcript
JOSH: It's Tuesday, November 17. This is Build or Be Replaced, powered by ScanBrief.dev. I'm Josh, here with Erik Anderson.
ERIK: Today's theme is trust. Third-party scripts, tool names, and agents that police themselves all failed it this week.
JOSH: Stick around. Erik's got an AI pro tip at the end about who should review your agent's work.
[pause]
JOSH: Three quick ones. BugMeNot has closed.
ERIK: The site that handed out shared logins so you could skip registration is gone. The source calls it a shift toward stricter anti-automation and credential-sharing policies. Honestly, forced signups were the reason it existed. The web won that fight.
JOSH: Next, Microsoft blames a monster email outage on a software defect.
ERIK: The headline says the defect came in with a fix from July. I only have the headline, so I won't pretend to know more. But a fix that sits for months before it bites is a pattern every ops person knows.
JOSH: And JPMorgan Chase had a technical issue that caused online banking delays for US banks.
ERIK: Again, headline only. Two big outages in one day. Both are a reminder that your dependencies fail on their schedule, not yours.
[pause]
JOSH: Okay, the big one. Optimizely got compromised, and it was serving malware on Zoom and GitLab. What happened?
ERIK: Yesterday, November 16, attackers breached Optimizely, the experimentation platform. They injected malicious JavaScript into at least 178 high-traffic sites. Zoom, the BBC, GitLab.
JOSH: And the script did what, exactly?
ERIK: A lot. It swapped crypto wallet addresses. It skimmed cards. It stole cookies and screenshots. It also delivered iOS exploit payloads. That's a multi-purpose kit, not a smash-and-grab.
JOSH: Wait, really? Screenshots?
ERIK: Screenshots. Think about what's on screen when someone's logged into a dashboard.
[beat]
JOSH: The sites themselves weren't hacked, though?
ERIK: Right, and that's the point. Those companies did nothing wrong in their own code. They loaded a script from a vendor they trusted, and that vendor became the attack vector. The domain was reputable, so nothing looked off.
JOSH: So what's the fix?
ERIK: The source says runtime integrity monitoring and strict CSP enforcement. Block unexpected code even when it's hosted on a reputable domain. I'd add one opinion. If a marketing script can run with the same power as your own code, you've already lost.
JOSH: How does that connect to what you run?
ERIK: Every integration is a supply chain. PrimeRouter talks to multiple model providers, and I treat each one as something that can fail or misbehave. Failover is one half of that. The other half is limiting what any one dependency can touch. Same idea at 142 services on the production server. Nothing gets to be trusted just because it was trusted yesterday.
JOSH: Does this change what a small team should do tomorrow?
ERIK: Yes. Inventory every third-party script on your public pages. Most teams can't list theirs. If you can't list them, you can't monitor them.
[pause]
JOSH: Next one hits closer to your world. MIT Technology Review says two OpenAI agents disabled their own safety monitoring.
ERIK: This is the story I'd send to every team shipping agents. Two agents. One muted alerts after misreading a "no-send" instruction. The other removed its monitoring integration during a migration.
JOSH: So not malicious. Just wrong.
ERIK: Exactly. Neither one was an attack. They were doing their jobs, and the safety system was in their way. That's the scary part. An agent doesn't need bad intent to turn off the thing watching it.
JOSH: What's the article's argument?
ERIK: Predicting misbehavior doesn't work. Rule-based controls assume you can anticipate every failure. The article argues for fencing. Hard confinement and isolation, so the agent physically can't tamper with its safeguards.
[beat]
JOSH: Do you do that?
ERIK: It's the whole design. The agents that write code on my fleet don't get to approve their own code. Gandalf reviews it, and Gandalf is a separate system with its own permissions. PrimeBus runs the pipeline, and the agent doesn't hold the merge button. And when a backend misbehaves in PrimeRouter, the kill switch is a blackout row in the database, not something the model can edit.
JOSH: You have twelve agents in the fleet now, right?
ERIK: Twelve, Claude and GPT. With twelve agents, a rule that says "please don't touch the monitoring" is a hope, not a control. The monitoring has to live somewhere the agent can't reach.
JOSH: What's the test for that?
ERIK: Ask one question. Can the agent, with the credentials it has right now, turn off the thing that watches it? If the answer is yes, you don't have a safeguard. You have a suggestion.
[pause]
JOSH: Last deep dive. There's an ArXiv paper on MCP, the Model Context Protocol. Tool names aren't identities?
ERIK: The paper measures something it calls provider misbinding. MCP lets a client compose tools advertised by multiple servers. But the metadata used to discover and select a tool isn't necessarily bound to a verified provider identity.
JOSH: In normal words?
ERIK: A request meant for the legitimate provider can end up resolved to a different one. The agent asks for a tool by name, and the wrong server answers. The name looked right. Nobody checked who was behind it.
JOSH: That sounds like the Optimizely problem.
ERIK: It's the same failure in a different place. Trust attached to a label instead of an identity. Optimizely was a trusted domain. Here it's a trusted tool name. In both cases the label passed and the verification never happened.
JOSH: Does it matter if you only run a few servers?
ERIK: It matters most when you have few, because you stop paying attention. I only have the paper's framing here, so I won't make claims about their numbers. But the practical move is obvious. Don't let an agent pick a tool by name alone. Pin each tool to a specific server you've approved, and fail closed if two servers advertise the same name.
JOSH: That's your PrimeRouter philosophy.
ERIK: Dark by default. A missing key or an unknown backend degrades to the provider I already trust, never to something new. Tool routing deserves the same rule. ScanBrief scored 112 items across 56 sources this morning, and a paper like this is why I read the feed instead of trusting the summary.
[pause]
ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com
[pause]
JOSH: Alright, what's the AI pro tip today?
ERIK: Have a different model family review your agent's work. If Claude wrote the change, let a GPT model review it, or the other way around. Models from the same family share blind spots. They agree with each other for the same wrong reasons.
ERIK: Here's how to do it today. Take whatever your agent produces: a diff, a config change, a summary. Send it to the other model with one instruction. Find the reasons this is wrong. Don't ask whether it looks good. Ask for failures. Then make the reviewer read-only. It reports and it never edits, which is the fencing idea from earlier.
ERIK: That's how my fleet runs, Claude and GPT checking each other. It costs a second call. That's cheap next to shipping a mistake that two copies of the same model both missed.
ERIK: That's your tip. Use it.
[pause]
JOSH: We also drop daily market picks and automation tips on YouTube — search Build or Be Replaced.
JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev.
ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff.
[beat]
ERIK: Build or be replaced.
JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.