Mon–Fri · 6 AM ET
← All Episodes
EP  • 00:09:42

Researchers trained a 13 billion parameter model on nothing but pre-1931 text | Build or Be Replaced

AI news and automation insights for 2026-04-28. Episode date: 2026-04-28.

Download MP3 →

Transcript

JOSH: It's Tuesday, April 28. This is Build or Be Replaced — powered by ScanBrief.dev. I'm Josh, here with Erik Anderson.
ERIK: Microsoft and OpenAI just ended their exclusive deal. The whole AI power structure shifted overnight. Today's about what that means for builders.
JOSH: Stick around — Erik's got an AI pro tip at the end about building so you can swap AI providers without touching your application code.

[pause]

JOSH: Three quick headlines first. Researchers trained a 13 billion parameter model on nothing but pre-1931 text. They called it Talkie.
ERIK: It's a time capsule with a chat interface. Ask it about the internet, it'll tell you that doesn't exist. What's useful about this project is what it reveals about how models reason toward their own future. When the training data ends, Talkie still predicts forward — but from a 1930 baseline. Every model is a snapshot. Talkie just makes that impossible to ignore.
JOSH: Next — Apple is killing AFP support in macOS 27 and requiring TLS 1.2 on all server connections.
ERIK: AFP should have been dead already. SMB's been the right call for years. The TLS mandate is the one with teeth. Older home lab setups, internal services that haven't been touched in three years — a lot of those are still running 1.0 or 1.1. If Mac is in your infrastructure chain, audit your connection config now, not when the update ships.
JOSH: And last — pgbackrest, the popular Postgres backup tool, is no longer being maintained.
ERIK: Barman is the move. But the actual lesson is dependency hygiene. Any critical tool maintained by one person is a single point of failure in your stack. pgbackrest had a good run. Time to move.

[pause]

JOSH: Okay. Microsoft and OpenAI. The exclusive deal is dead, the revenue sharing is dead. How do we get here?
ERIK: Microsoft put in thirteen billion over the years. In exchange they got exclusive Azure hosting rights, a revenue cut, and Copilot baked into Office. OpenAI got compute and credibility. For a while that worked. But OpenAI has been trying to close compute deals with other providers — Oracle, Amazon, others — and the exclusivity kept blocking it. Now that's gone.
JOSH: Who blinked first?
ERIK: Probably OpenAI. They've been moving toward independence for two years. The Altman saga, the for-profit restructuring — they want to operate like a company, not a Microsoft division. Cutting the exclusivity and revenue share is that move made official.
JOSH: So Microsoft loses here?
ERIK: Not exactly. The enterprise install base isn't going anywhere. Copilot is in Word, Excel, Teams — hundreds of millions of users. Microsoft didn't bet the company on OpenAI. They bet part of it and got years of AI credibility in return. But the moat is smaller now. If GPT-5 runs on AWS just as well as Azure, the "you need Microsoft to get the best AI" pitch doesn't hold anymore.

[beat]

JOSH: What does this mean for someone building on these platforms right now?
ERIK: More competition, which is good for pricing. But it's also a reminder that no provider is stable infrastructure — they're all companies making business decisions. That's why routing matters. PrimeBus has had a model routing layer since day one. Every API call goes through the orchestrator. The orchestrator decides which model handles it. Right now I've got three Claude instances on the mesh — Neo, Bill, Homer. But the routing logic doesn't care what's underneath. If Anthropic has an outage or changes pricing, I update the routing config. Nothing in the application changes.
JOSH: How long did that take to build?
ERIK: Fifty lines of Python, a config file. A couple hours. Most builders skip it because it feels like extra work on day one. Then their provider changes terms and they're in a bad spot.
JOSH: And that's exactly what's happening right now.
ERIK: It's been happening. This Microsoft-OpenAI split is just the loudest version. Copilot's billing change today is the same story — provider decides the cost model doesn't work, terms change overnight. If you built assuming the terms were permanent, you're scrambling. Build like the terms will change. Because they will.

[pause]

JOSH: The Mercor breach. 40,000 AI contractors, 4 terabytes stolen. What makes this one different from a standard data breach?
ERIK: Two things. The data type — voice biometrics. Actual audio recordings of people's voices. And those recordings were paired with government-issued ID scans in the same dataset. That combination has never leaked at this scale before.
JOSH: Why does the pairing matter so much?
ERIK: Voice cloning by itself is already a nuisance. You can generate a convincing synthetic voice from a short clip. That's been true for a while. But you don't know whose voice it is. You can't target anyone specific. When you add verified identity to the audio, you know exactly who the voice belongs to. You can build a targeted attack. You call their bank. You call their employer. You get into phone-based verification systems that use voice as a factor. That's a completely different threat class.
JOSH: Is voice auth still being used at scale?
ERIK: More than it should be. Financial services, call centers, some government systems. And the pitch was always "only you have your voice." Now 40,000 people's voices are sitting in a leaked archive, attached to ID documents that confirm who each voice belongs to. That's a ten-year tail. Someone will use that data in 2031. The person won't know why their account got hit.
JOSH: What should someone do if they think they're in this breach?
ERIK: Audit every service that uses voice as an auth factor and swap it out. TOTP, hardware keys — anything but voice. Voice authentication was always the weakest second factor. This closes the argument. And don't sign up for any voice-biometric-as-payment systems. That trend needs to stop.

[beat]

JOSH: You work on identity in your own AI pipelines. Does this change anything for you?
ERIK: It reinforces something I've been thinking about with HumanRail. HumanRail routes AI decisions to a human when the model isn't confident enough to act. The whole premise is that the human in the loop is trustworthy — that they are who they say they are. These breaches are stacking up. Biometrics, identity scans, voice samples — all compromised at scale now. Verifying real humans in AI workflows is going to get harder, and the stakes are going up as these systems make real decisions. The next version of the routing logic has to account for that.

[pause]

JOSH: GitHub Copilot is moving to usage-based billing. What happened?
ERIK: Cursor happened. When Copilot launched, ten dollars a month made sense — it was autocomplete. People did a few tab-completions, called it a day. Now AI coding tools are running agentic sessions. Multi-file edits, long code generation runs that burn through tokens. The cost per heavy user shot up. Usage-based billing shifts that cost to the people who are actually consuming it.
JOSH: Is that fair?
ERIK: From GitHub's perspective, rational. From a builder's perspective, the predictability is gone. If you're shipping hard and running agentic sessions all day, you don't want to be thinking about compute costs per task. You want the flat rate.
JOSH: Would you pay for Copilot at the new rate?
ERIK: No. PrimeBus, ScanBrief, InkEngine — it's all direct Claude API with my own routing layer. That's where I live. Writing "The Autonomous Engineer" — 55,000 words in two days using InkEngine — that's not how Copilot works. But I'm an edge case. For someone who wants clean IDE integration without building any infrastructure, Copilot is still a reasonable tool. Just know what your actual usage looks like before you opt in. Some teams are going to get a very unpleasant invoice.
JOSH: Who should be most worried?
ERIK: Enterprise teams that issued Copilot licenses to 50 engineers and haven't looked at the bill since. That conversation is coming.

[beat]

ERIK: The real story is that coding assistants are commoditizing fast. The margin is collapsing. Usage-based is GitHub trying to capture value at the top of the curve before the race to zero finishes. Every month these tools get cheaper and more capable. Flat-rate pricing doesn't hold in that environment.

[pause]

ERIK: This episode is sponsored by Prime Automation Solutions. If you're still doing it manually, we automate it. Also, special on a website — $250. primeautomationsolutions.com

[pause]

JOSH: Alright, what's the AI pro tip today?
ERIK: Build a routing layer. One function that sits between your application and the model. You pass in the task type and the payload, the function decides which provider and which model handles it. Cheap fast tasks go to a smaller model. Complex reasoning goes to Opus. If a provider goes down or changes terms, you update the routing config — not your application. Nothing downstream breaks. I built this into PrimeBus in week one. In the past month I've had three separate API disruptions. None of them hit production because the router kept running. It is fifty lines of Python and one config file. You can write it this afternoon. That's your tip. Use it.

[pause]

JOSH: We also drop daily market picks and automation tips on YouTube — search Build or Be Replaced.

[pause]

JOSH: One more thing — we started a Discord for builders. If you're shipping AI, automation, or anything that makes a human obsolete — come hang out. Link at buildorbereplaced.dev.
ERIK: Post what you built. We'll post what we're building. Real wins, real builds, no fluff.

[pause]

ERIK: Build or be replaced.
JOSH: If you want these signals in your inbox every morning, scanbrief.dev. See you tomorrow.