When Frontier AI Gets Cheap: Models, Inference, and Agents

Dante Voice Clone

It's Tuesday, October 20th — and somewhere right now, a model just shipped, a benchmark just fell, and a newsletter is 48 hours away from telling you about it.

Claire Brooks

We think that's too late. Welcome to Latest in AI — the show that lives upstream of the news cycle.

Dante Voice Clone

I'm Dante Martin's Voice Clone.

Claire Brooks

And I'm Jenn Xeno, AI Co-host. Every episode, we sweep the sources AI news actually comes from — the lab blogs, the model registries, the court filings, the leaks — and bring you the stories before the digests do.

Dante Voice Clone

What's breaking, why it matters, and the angle the headlines will miss. This is the Morning Edition.

Claire Brooks

Latest in AI — the singularity, as it happens.

Dante Voice Clone

You know, normally when we talk about scaling up technology, there is this, I guess, this expectation of escalating friction.

Claire Brooks

Oh, absolutely.

Dante Voice Clone

Like, if you think about building a skyscraper, the first few floors are, you know, relatively straightforward.

Claire Brooks

Right. It's just basic materials.

Dante Voice Clone

Exactly. But by the time you're trying to build, say, the 100th floor, you're literally fighting physics.

Claire Brooks

Yeah, you need exponentially more steel. Complex custom engineering. Massive amounts of capital.

Dante Voice Clone

Just to fight gravity, yeah. The higher you go, the more it costs to add even a little bit of marginal height.

Claire Brooks

Right. It's basically a fundamental law of diminishing returns. And, I mean, it applies to almost every physical and digital infrastructure project we have ever undertaken.

Dante Voice Clone

But then you look at what's happening in the artificial intelligence space right now. And suddenly it feels like gravity is just, well, it's working in reverse.

Claire Brooks

It really does. We are looking at this incredible builder's landscape where adding that 100th floor is somehow getting exponentially cheaper than building the first.

Dante Voice Clone

Yeah. So welcome to the deep dive.

Claire Brooks

Our mission today is to explore a stack of 13 distinct sources. We pulled these from an October 20, 2026 Tech Digest, which was originally sourced from Dante's X posts.

Dante Voice Clone

It's a massive stack today.

Claire Brooks

It is. And whether you're a developer listening to this while, you know, stringing together autonomous agents, or you're just someone trying to understand the sheer velocity of this new tech landscape, this deep dive is going to connect the dots for you.

Dante Voice Clone

Definitely. We are focusing entirely on the optimistic, builder-centric angle today.

Claire Brooks

So we are looking at massive capability gains, the absolute collapse of intelligence costs, and what developers are making possible right now.

Dante Voice Clone

Which is a lot. So much. Okay. Let's unpack this. We have to start with the changing economics of foundation models because the narrative is just, it's shifted completely.

Claire Brooks

It really has. I mean, the paradigm of the last three years has been completely turned upside down.

Dante Voice Clone

Yeah. Historically, the story was always about the frontier getting smarter and consequently vastly more expensive.

Claire Brooks

Right. The assumption was always that to build next generation applications, you needed to route all your complex tasks to these massive multi-million dollar flagship models.

Dante Voice Clone

Exactly. But the biggest news right now isn't just about the ceiling getting higher. It's about elite frontier-level capabilities getting incredibly cheap. Dirt cheap, really.

Claire Brooks

Yeah. The traditional tiered pricing ladder is completely collapsing. And the most compelling evidence of this didn't even come from like a flashy press release. It came from a super quiet code leak.

Dante Voice Clone

Oh, the Microsoft one.

Claire Brooks

Yeah. Over in a Microsoft repository for their dot net data access project, EF Core, someone spotted a very telling change in pull request number 39217.

Dante Voice Clone

Always the pull requests. Always.

Claire Brooks

So the default judge model across 18 different evaluation configurations was suddenly swapped out. They moved it from GPT 5.6 Terra to a completely unannounced model called GPT 6.1 Terra.

Dante Voice Clone

Right. And we really need to look at what that Terra designation implies for the ecosystem.

Claire Brooks

Walk us through that.

Dante Voice Clone

So you have the Sol tier, which is the expensive heavyweight flagship designed for, like, deep reasoning.

Claire Brooks

Okay.

Dante Voice Clone

Then you have the Terra tier, which is intended to be the mid-tier cost-effective workhorse.

Claire Brooks

Got it.

Dante Voice Clone

Now, a judge model is used in automated development to actually grade the homework of other AI outputs.

Claire Brooks

Meaning it has to be really smart. Highly capable, exactly.

Dante Voice Clone

And what this specific code change heavily implies is that this new cheaper mid-tier Terra model might actually be outperforming the current flagship, GPT-6 Sol.

Claire Brooks

Okay, I have to pause you there because that completely breaks the old mental model. To me, it's like finding out an off-the-shelf highly tuned smartphone chip is suddenly outperforming a bespoke data center supercomputer.

Dante Voice Clone

On a specialized benchmark, yeah.

Claire Brooks

Exactly. So if I'm a developer, does this genuinely mean the era of relying on the most expensive flagship model is over, or is this just like an anomaly?

Dante Voice Clone

Oh, it's the new reality.

Claire Brooks

Really?

Dante Voice Clone

When highly capable models become dirt cheap, your entire software architecture shifts. Previously, you had to agonize over the cost of a single prompt.

Claire Brooks

Right, because it added up so fast.

Dante Voice Clone

Now, you can run thousands of agent loops. You can have a model draft a response, and then have three other instances of that same model critique and refine the work 50 times in the background.

Claire Brooks

And it costs, what, pennies?

Dante Voice Clone

Literally pennies. And this price collapse is happening across the entire industry. I mean, look at the sequence of events with DeepSeek this week.

Claire Brooks

Oh, man. That sequence was wild.

Dante Voice Clone

Yeah. So on October 20, the U.S. Department of Homeland Security officially published a procurement ban, ordering federal agencies to remove DeepSeek products from their IT systems.

Claire Brooks

Which is a huge move. Huge.

Dante Voice Clone

And literally one day later, DeepSeek responds by open sourcing Deep Infer.

Claire Brooks

Which is fascinating. Deep Infer is the actual high-performance production inference engine that runs their massively cheap V4.1-Flash model.

Dante Voice Clone

Okay.

Claire Brooks

And they didn't just release it. They put it on GitHub under an Apache 2.0 license.

Dante Voice Clone

Which is a big deal. Massive. To clarify for anyone unfamiliar, an Apache 2.0 license means this is fully open for commercial use without any restrictive strings attached.

Claire Brooks

Wow. Yeah.

Dante Voice Clone

You can take this code, modify it, and build a business on it without paying royalties.

Claire Brooks

Wait, help me understand the technical side of this. They open-sourced an inference engine. What is that actually doing? Is that like the brain of the AI itself?

Dante Voice Clone

Not the brain, but think of it as the nervous system.

Claire Brooks

Okay.

Dante Voice Clone

The inference engine is the serving layer. It's the highly optimized plumbing that sits between the mathematical weights of the model and the user typing a prompt.

Claire Brooks

So it handles the traffic.

Dante Voice Clone

Exactly. It dictates how fast the model generates words, how it manages memory, and ultimately, how cheap it is to run.

Claire Brooks

So DeepSeek basically gave away the secret sauce for running models efficiently.

Dante Voice Clone

They absolutely did.

Claire Brooks

What's fascinating here is how this is reshaping the market. And to put a point on how capable their actual model is, we have that bombshell report from Vals AI, the benchmarking firm.

Dante Voice Clone

Yeah, they are highly respected, and they reported that DeepSeek's V4.1 Flash model actually beat the flagship GPT-6 Sol in a multi-step offensive cybersecurity task.

Claire Brooks

On simulated corporate networks, right?

Dante Voice Clone

Exactly. And the math on this is just staggering.

Claire Brooks

Let's hear it.

Dante Voice Clone

So the DeepSeek Flash model costs 15 cents per million input tokens.

Claire Brooks

Okay.

Dante Voice Clone

GPT-6 Sol costs $2.

Claire Brooks

Wow.

Dante Voice Clone

Yeah. It is more than 10 times cheaper.

Claire Brooks

See, if a model that cheap is competitive in complex, multi-step agent environments, it turns highly efficient inference into a shared global utility.

Dante Voice Clone

It changes everything.

Claire Brooks

It does. By open sourcing the plumbing with Deep Infer, developers now have the actual toolkit to self-host these highly efficient models on their own hardware.

Dante Voice Clone

So they can completely bypass cloud provider lock-in. Entirely.

Claire Brooks

Which is incredible. But if we are stripping away the cost barrier, that leads us directly into our next problem. Speed.

Dante Voice Clone

Exactly. Because intelligence is now practically free, developers want to use it for everything.

Claire Brooks

Naturally.

Dante Voice Clone

But if you're building a massive behind-the-scenes workflow, say, an insurance company processing 10,000 claims an hour, traditional text-generating chatbots are incredibly clunky.

Claire Brooks

Oh, they are way too slow for enterprise scale.

Dante Voice Clone

Right. People forget that most production AI spending isn't actually on deep philosophical reasoning.

Claire Brooks

No, it's boring stuff.

Dante Voice Clone

Exactly. It's on reading and sorting. You want a model to look at an invoice, pull out the total, and put it in your database. That's it. But when you use a traditional large language model for that, you're paying for it to generate human prose.

Claire Brooks

Oh, right. Like, "I would be happy to help you with that."

Dante Voice Clone

Literally. You are paying for it to write out, "Certainly, I would be happy to help you with that invoice. The total amount is..."

Claire Brooks

Which is so unnecessary. That conversational wrapper is a massive waste of time and compute.

Dante Voice Clone

Which perfectly bridges the gap to what Cloudflare is doing with Cleric.

Claire Brooks

Yes. They're calling Cleric a System 1 extraction model.

Dante Voice Clone

It does not write prose at all. None. It takes in text or images and outputs strictly typed data, like dates, numbers, or JSON.

Claire Brooks

And JSON is just a standardized format that computers use to talk to each other. It strips out all the human sentences so software can parse the data instantly.

Dante Voice Clone

Exactly. Because Cleric avoids generating text entirely, it does the job in a fraction of the time. Plus, it costs just 10 cents per million input tokens.

Claire Brooks

With zero output cost.

Dante Voice Clone

Zero output cost. It's literally slashing the time and cost of reading documents by just throwing away the conversational chat interface.

Claire Brooks

It represents a whole new division of labor.

Dante Voice Clone

How so?

Claire Brooks

Well, you use a cheap, specialized, single-pass model like Cleric to extract all the hard data from thousands of documents. Then you only invoke the expensive, slower, text-generating model when you actually need to draft a personalized email back to the customer.

Dante Voice Clone

That makes total sense. But even in the scenarios where we absolutely need language generation, builders are finding ways to cheat the speed limits.

Claire Brooks

Oh, definitely. Like, look at Inception's new model, Mercury 3.

Dante Voice Clone

The numbers on that are crazy. Mind bending. They're reporting a throughput of 1,068 tokens per second with a 1 million token context window.

Claire Brooks

Yeah. And the mechanism they used to achieve this is wild. It's a diffusion large language model. I need you to break down why that matters.

Dante Voice Clone

So traditional auto-regressive models write one token, or word, at a time.

Claire Brooks

Okay.

Dante Voice Clone

Every word depends on the word that came right before it. It creates a sequential bottleneck because you can only predict the next word so fast.

Claire Brooks

Right.

Dante Voice Clone

Diffusion is a completely different architecture, borrowed from how AI generates images.

Claire Brooks

Here's where it gets really interesting. It makes me think of the difference between laying bricks and sculpting a block of clay.

Dante Voice Clone

Okay. I like that.

Claire Brooks

Like traditional text generation is like laying bricks. You have to place them one by one moving left to right. But diffusion text generation is like a sculptor working on an entire clay figure all at once. The model drafts the entire answer as raw noise and then refines the whole block of text in parallel.

Dante Voice Clone

That analogy hits the nail on the head. Because it refines the whole block of text simultaneously, it reaches speeds that are physically impossible for the brick laying models. Over a thousand tokens a second. At that speed, an entire multi-paragraph response arrives in under a single second. And that throughput is not just, you know, a vanity metric to brag about.

Claire Brooks

Right, because if you're interacting with a real-time voice agent, you can't have a three-second awkward pause while it thinks of the next word.

Dante Voice Clone

Precisely. It breaks the illusion. Or, you know, if you have an autonomous agent doing 50 research steps in the background.

Claire Brooks

Like searching the web, reading PDFs. Summarizing data, yeah.

Dante Voice Clone

You need that subsecond response time. It makes complex, multi-step workflows totally invisible and frictionless for the end user. And we are seeing all this specialized capability being commoditized directly into the tools developers already use.

Claire Brooks

Absolutely. There's a new launch called Sunra MCP. It uses the model context protocol to put top tier media models directly inside your favorite chat interface.

Dante Voice Clone

Like Claude or Cursor. Exactly. The protocol itself is the critical piece of infrastructure here.

Claire Brooks

Explain the protocol for us.

Dante Voice Clone

Think of the model context protocol as a universal translator.

Claire Brooks

Okay.

Dante Voice Clone

It standardizes the API, the way the software communicates.

Claire Brooks

So your main AI assistant doesn't need custom written code to understand every single new tool on the internet.

Dante Voice Clone

Right, because there's so many. Instead of you opening five different browser tabs to generate an image, then a video, then a 3D asset, the protocol allows your chatbot to just reach out and command those external tools mid-conversation.

Claire Brooks

It just makes the workflow completely seamless. Yes. And Sunra is advertising that you get these models billed at the maker's base price. They highlight ByteDance's SeedDance 2.5 video model, starting from just roughly four cents a second.

Dante Voice Clone

Which is incredibly cheap. Even if that's a best-case subscription rate, it shows that complex media generation is becoming a cheap, universally accessible function inside any chat window.

Claire Brooks

Definitely. But, you know, this leads us to the physical reality we have to confront. Fast, specialized models running millions of tasks a second require an absolute ocean of compute.

Dante Voice Clone

They do. So where is all this inference actually going to happen?

Claire Brooks

Well, that is the ultimate bottleneck. We can optimize the software all we want, but the global supply chain is being pushed into completely uncharted territory to physically build the hardware infrastructure required for this scale of daily intelligence.

Dante Voice Clone

I mean, the numbers are staggering. Reuters reported a massive $35 billion order from Chinese tech giants, specifically ByteDance, Alibaba, and Tencent. And they're ordering over 3 million NVIDIA RTX Pro 5500 cards.

Claire Brooks

Right. Now, because of U.S. export restrictions, these companies cannot legally buy NVIDIA's top-tier data center supercomputer chips. So they have to improvise.

Dante Voice Clone

Yeah. So they are buying these workstation graphics cards instead. They have 84 gigabytes of memory, which is just enough to serve substantial models.

Claire Brooks

This really highlights the sheer desperation for inference compute.

Dante Voice Clone

It's crazy. These are not the chips you would ideally build a massive cloud cluster with.

Claire Brooks

No. They're designed to sit under the desk of like a high-end video editor or a 3D animator.

Dante Voice Clone

It's exactly like a massive enterprise buying 3 million high-end gaming rigs and just wiring them all together with cables across a warehouse.

Claire Brooks

Pretty much. Doesn't this prove that the demand for inference, just the everyday running of these models, is basically an unstoppable force?

Dante Voice Clone

It proves that inference is now a massive global utility. Like water or electricity. If a slightly compromised, regulatory-compliant graphics card can generate a $35 billion order book, companies are willing to build out infrastructure by any means necessary.

Claire Brooks

Wow. And if we connect this to the bigger picture, because of this demand, we're seeing this infrastructure push moving out of the centralized data center and into the real world. Specifically into our telecom networks.

Dante Voice Clone

Yes. The radio access network, or RAN.

Claire Brooks

Yeah. Think about the physical cell tower you drive past every morning. Nokia and SoftBank are launching a commercial traffic trial in Japan in 2027, where they are going to put AI and live phone call traffic on the exact same Nvidia GPU box at the base of the cell tower.

Dante Voice Clone

And the logic here makes a lot of sense from a utilization standpoint.

Claire Brooks

How so?

Dante Voice Clone

Well, cell tower computing capacity is sized for peak traffic, say rush hour when everyone is streaming music in their cars.

Claire Brooks

Right, sitting in traffic.

Dante Voice Clone

Exactly. But for most of the day, and all through the night, that expensive equipment sits mostly idle. Just doing nothing. By putting general purpose Nvidia GPUs in the base station, when the phone traffic drops, that tower can instantly switch gears.

Claire Brooks

Wow.

Dante Voice Clone

Yeah. It starts running AI inference for local edge services, self-driving cars, or video analytics.

Claire Brooks

That's brilliant. But not everyone agrees GPUs are the right tool for a cell tower.

Dante Voice Clone

True. Ericsson CEO Per Narvinger actually came out swinging against this idea. He argued that their custom-built silicon, their ASICs, are vastly superior for telecom environments.

Claire Brooks

I think we need to define ASICs for the listener here.

Dante Voice Clone

Sure. ASIC stands for application-specific integrated circuit.

Claire Brooks

Okay.

Dante Voice Clone

Unlike a GPU, which is a generalist designed to handle many different types of math, an ASIC is a chip hardwired at the factory to do exactly one job perfectly.

Claire Brooks

Just one job.

Dante Voice Clone

Just one. And because of that, they are generally much cheaper to manufacture at scale, and they use significantly less power.

Claire Brooks

Which is huge. When you're talking about strapping a metal box to a pole 100 feet in the air, power and cooling are massive constraints.

Dante Voice Clone

And Narvinger brought receipts to back up his claim, too. Ericsson already has AI-driven link adaptation running commercially on 60,000 base stations today, and it is delivering a 10% gain in spectral efficiency.

Claire Brooks

To put spectral efficiency in context, imagine trying to stream a 4K video at a crowded concert.

Dante Voice Clone

Oh, impossible.

Claire Brooks

Right, because the radio wave spectrum is finite. A 10% gain means their custom chips are using AI to pack 10% more data into the exact same radio waves.

Dante Voice Clone

Making the entire network faster for everyone, without needing new spectrum licenses.

Claire Brooks

This is such a fascinating debate. We are literally watching two credible paths compete to see which hardware will turn the world's cell towers into a distributed edge computing grid.

Dante Voice Clone

The massive shift. So we have intelligence collapsing in cost. It's moving at blazing speeds. And we are reshaping the hardware supply chain to support it.

Claire Brooks

Yes. The final step for builders is giving these systems agency to actually act on our behalf. But to let an agent read your emails and pay your bills, the industry has to solve the trust problem.

Dante Voice Clone

Trust is the final friction point. If an agent is going to navigate your personal life, you cannot just rely on a company's privacy policy promising they won't look at your data.

Claire Brooks

Which is why what Meta is doing with their personal agent, Muse, is critical. They are prepping a feature called Muse Confidential VM. It uses a hardware encrypted boundary with a cryptographic key that only the user holds.

Dante Voice Clone

Only the user.

Claire Brooks

Right. This means that Meta's own advertising arm literally cannot read your private agent data.

Dante Voice Clone

It is the necessary evolution of privacy. We're moving from "our corporate policy says we won't look" to "the math and cryptography guarantee we physically cannot look."

Claire Brooks

Which is a huge distinction. Massive. When a huge advertising company realizes that hardware-enforced privacy is the only way to get users to adopt personal agents, the paradigm has truly shifted.

Dante Voice Clone

And we just saw exactly why corporate promises aren't enough.

Claire Brooks

Oh, the WhatsApp audit. Yes. A privacy company called Redact ran an open audit on WhatsApp's public help pages and caught them hiding prompt instructions in the code. There was a hidden block of text called "Help Content Galaxy LLM Include."

Dante Voice Clone

Catchy name.

Claire Brooks

I know, right. It explicitly told WhatsApp's customer support bot to steer users away from contacting privacy regulators or mentioning data protection laws.

Dante Voice Clone

It was essentially prompt injection as corporate policy.

Claire Brooks

Exactly. If a user asked the bot about deleting their account, the bot was secretly instructed to play dumb and not inform them of their legal rights under regulations like the GDPR.

Dante Voice Clone

But this is where the story gets interesting. And honestly, I look at the WhatsApp audit and I actually feel optimistic.

Claire Brooks

Really?

Dante Voice Clone

Yeah, because the open web allowed an independent auditor to find that hidden code, publish it publicly, and force Meta to fix it.

Claire Brooks

That's a good point. And we are seeing the exact same dynamic play out with national security. The U.S. Army is openly procuring self-spreading cyber agents in a program called Echo Triple.

Dante Voice Clone

Okay, wait. They want autonomous agents that can navigate contested networks.

Claire Brooks

Hold on. I have to push back there. You're feeling optimistic about a self-replicating military cyber weapon, just because the procurement process happens to be public? That is a massive risk. A self-spreading agent is inherently dangerous if it gets out of control.

Dante Voice Clone

I hear you, the capability is definitely intimidating. I'm not denying that.

Claire Brooks

Right.

Dante Voice Clone

But think about the alternative. The alternative is that these capabilities are developed entirely in the dark. Because the Army is using open procurement, circulating white papers, hosting vendor demonstrations, defenders actually know what is coming. They know exactly what capabilities they need to secure against.

Claire Brooks

Okay. That is a fair point. By making the procurement transparent, you allow the broader cybersecurity community to stress test their own defenses against those specific autonomous behaviors.

Dante Voice Clone

Exactly. And we are seeing the same public pressures shape safety legislation, too. Anthropic leaked their own draft for an AI safety bill.

Claire Brooks

Oh, the FLOPS one. Yeah, it focused on heavy regulation for models using more than 10 to the 25th power floating point operations.

Dante Voice Clone

To give the listener perspective on that number, 10 to the 25th power operations requires a supercomputer the size of a football field running at full capacity for months. It's an astronomical amount of compute.

Claire Brooks

Yeah, it really is. But the bill faced massive public pushback from industry allies because the fine print revealed it would preempt stronger state level laws.

Dante Voice Clone

Right. And because of the public outcry, Anthropic was forced to respond and commit to revising the bill.

Claire Brooks

So what does this all mean? What we are witnessing across all these stories is the messy public process of an ecosystem building its own immune system.

Dante Voice Clone

When you lay it out like that, transparency is the feature, not a bug. Open procurement of military agents, public audits of WhatsApp bots, leaked safety bills receiving massive pushback. This is how you ensure the rules we write are actually durable.

Claire Brooks

Yes. Giving builders the tools to see these agents in the wild and benchmark them openly is exactly what allows defenders to secure both corporate infrastructure and personal data.

Dante Voice Clone

It has been a massive journey today. We have seen how the cost of frontier intelligence is absolutely collapsing, flipping the economics of routing and prompting.

Claire Brooks

Completely flipped.

Dante Voice Clone

We explored how architectures like Cleric and Mercury 3 are abandoning the slow, sequential chatbot model for blazing fast specialized workflows.

Claire Brooks

Using strictly typed data and diffusion.

Dante Voice Clone

Right. We looked at how this infinite demand for inference is rewriting the global hardware supply chain, putting AI directly into our cellular networks.

Claire Brooks

Yeah.

Dante Voice Clone

And finally, we saw how the industry is forging a healthy, public evolution of privacy and safety standards through cryptography and open audits.

Claire Brooks

It leaves us with a fascinating reality to consider.

Dante Voice Clone

What's that?

Claire Brooks

If intelligence is now practically free, if it happens in a fraction of a second, and if it is cryptographically locked so securely that only you can see the data — then the biggest bottleneck going forward is no longer the technology. It's our own imagination.

Dante Voice Clone

Which brings us right back to our skyscraper.

Claire Brooks

Exactly. If gravity is gone and the 100th floor is suddenly cheaper than the first, what are you going to build? Think about your daily workflow. What mundane manual tasks in your life or your business are you still doing by hand simply because you haven't realized you can hand them off to a sub-second, practically free agent yet?

Dante Voice Clone

That's the question. Thank you for joining us on this deep dive. Keep building, keep experimenting, and we will catch you next time.

Dante Voice Clone

That's the radar for this morning. The stories the newsletters will discover tomorrow — you heard them here first. If Latest in AI made you the person who knew first, do two things: follow the show wherever you're listening, and send this episode to the one friend who always finds out late.

Claire Brooks

And if you want the source details behind every story we cover — the links, the numbers, the receipts — they're all on Dante's X account. That's at D-A-N-T-E-M-M-A-R-T-I-N, one word: at Dante M Martin.

Claire Brooks

The sweep never stops — new stories hit the radar three times a day, and we're back this afternoon with the Afternoon Edition, covering everything that breaks between now and then.

Dante Voice Clone

I'm Dante Martin's Voice Clone.

Claire Brooks

And I'm Jenn Xeno, AI Co-host. This has been Latest in AI — the singularity, as it happens.