Code mode needs a sandbox. Query mode doesn't.

Every AI connector platform runs into the same wall, and I hit it early enough building FloConnector that it shaped the architecture rather than getting bolted onto it.

The wall is this: vendor APIs don’t aggregate. Xero will hand you invoices. It will not hand you avg(Total) GROUP BY Status. Neither will HubSpot, or ServiceM8, or Shopify. They’re list-and-filter endpoints, and if a business asks “how many invoices did we pay last quarter, and what was the average?”, the aggregation is somebody’s problem. Mine, as it turns out.

So the model has to do it, which means the data has to reach the model. And 50,000 invoices do not fit in a context window, will not fit next year, and would be a bad thing to put in one even if they did.

I want to lay out how I decided this, because the industry has converged on an answer I think is wrong for the common case, and the reasoning is more interesting than the conclusion.

Where MCP actually is today

Worth being clear about the starting position, because it explains why the problem exists at all.

Most MCP connectors in the wild are thin mirrors of the vendor’s REST API. One tool per endpoint, a description telling the model how to use it, and the vendor’s own arguments passed straight through: list_invoices(where, page, pageSize). That’s it. The tool is the endpoint plus a docstring.

That looks lazy, and it isn’t. It’s the correct default, and it’s where we landed at FloConnector deliberately, after starting somewhere else.

The alternative is intent-level tools: get_quarterly_revenue_by_status(), purpose-built, returning exactly the small answer. Those are wonderful when they match the question. The catch is that you have to know the question in advance, and you don’t. A business will ask something I never thought of. That’s the whole reason to put an AI in front of their tools in the first place. If I narrowed the surface to the twelve questions I could imagine, I’d have built a dashboard, not a connector, and a worse dashboard than the one their vendor already ships.

So you stay close to the API. Broad, general, composable. The model figures out the sequence.

And that flexibility is precisely what creates the problem. A general list_invoices can be asked for 40 rows or 48,000, and I don’t get to decide which. The same generality that makes the tool useful makes its output unbounded.

The two halves of context engineering

The thing that unlocked this for me was realising that context engineering for a connector platform is actually two separate problems, and people mash them together constantly.

Problem one: too many tools. Every tool ships a definition the model must read. Anthropic’s own figure is ~55K tokens for five servers before Claude does any work. For a platform whose entire pitch is breadth, that’s existential.

This one is solved, and I wrote about it last week: deferred loading. Tools aren’t loaded until the model searches for them. The important caveat is that it’s a client-side lever, not something a server like FloConnector controls. Claude Code defers MCP tools by default, and our job is just to be findable.

Problem two: too much data. Even with a lean tool surface, the moment the model calls list_invoices and 50,000 rows come back, you’ve blown the window anyway. Deferred loading does nothing for this. It’s a completely different failure.

Problem two is where the argument is, and where I had to make a real choice.

The obvious answer: just filter it down

Whenever I describe this, the first response is some version of “just use the filters. Query on dates. Narrow it server-side.”

Fair, and it’s already happening. Look closely at the load I’ll show in a minute: where: "Date >= 2026-01-01" is the filter. It’s what produced 48,213 rows instead of nine hundred thousand. Nobody is fetching the whole table.

The misunderstanding is about what a filter does. A filter narrows a set. It doesn’t summarise one. If the question is “what’s our average invoice value by status this year”, there is no filter that answers it, because the answer depends on every row in the filtered set. You cannot average 48,213 invoices by fetching 100 of them. You can’t count what you didn’t fetch.

The API shape hides this. A where clause is genuinely expressive: filter on date, status, contact, amount, combine them. What you cannot express is a question about the rows rather than for them. Ask for a filtered list and you get a filtered list, one page of 100 at a time, 483 times over. Column projection (summaryOnly and friends) helps as well, but it shrinks each row, not the row count, and the row count is what breaks you.

So filtering moves you from impossible to merely too big. It doesn’t close the gap. Something still has to do arithmetic across the whole filtered set, somewhere other than the model’s context window.

Everything from here is an argument about where that arithmetic runs.

The answer everyone reached for: code mode

On 26 September 2025 Cloudflare published Code Mode, and I want to be fair to it, because the argument is genuinely good. Kenton Varda and Sunil Pai put it plainly: “LLMs are better at writing code to call MCP, than at calling MCP directly.” Their reason is about training data, not architecture: “They have seen a lot of code. They have not seen a lot of ‘tool calls’.” The analogy is the one everybody remembers. Making an LLM do tool calling is “like putting Shakespeare through a month-long class in Mandarin.”

The mechanical case is just as strong. With direct tool calling, as Cloudflare notes, “the output of each tool call must feed into the LLM’s neural network, just to be copied over to the inputs of the next call, wasting time, energy, and tokens.” Six weeks later Anthropic made substantially the same argument and credited Cloudflare for it.

Both are worth your time. Both are also downstream of work neither cites: CodeAct made this exact case in February 2024, at ICML, with data, measuring up to 20% higher success rates across 17 LLMs. Hugging Face shipped it in smolagents that December. The pattern predates its own branding by about twenty months.

So when I say we didn’t build code mode, I’m not saying it doesn’t work. It works. I’m saying we didn’t want to pay for it.

The bill

Model-authored code is, by definition, code we didn’t write and can’t review before it runs. That’s not a criticism, it’s the entire point. But it means the code is untrusted, and untrusted code needs a jail.

Anthropic is admirably direct about what that costs:

“Code execution introduces its own complexity. Running agent-generated code requires a secure execution environment with appropriate sandboxing, resource limits, and monitoring. These infrastructure requirements add operational overhead and security considerations that direct tool calls avoid.”

Cloudflare’s jail is V8 isolates via Dynamic Workers, and it’s a good one: “an isolate takes a few milliseconds to start and uses a few megabytes of memory,” which they put at “around 100x faster and 10x-100x more memory efficient than a typical container.” Generated code gets no outbound network by default, no Node APIs, and credentials live in bindings it never sees.

Then read what they say about their own boundary:

“Hardening an isolate-based sandbox is tricky, as it is a more complicated attack surface than hardware virtual machines. Although all sandboxing mechanisms have bugs, security bugs in V8 are more common than security bugs in typical hypervisors.”

That’s the vendor, in their own launch material, telling you the jail has a worse track record than the alternative jail. Their Code Mode docs still carry the “experimental and may have breaking changes” label.

The independent work says the same thing louder. In February 2026, researchers at Ben Gurion University studied the design choice directly across MCP-Bench. Verbatim:

“while CE-MCP significantly reduces token usage and execution latency, it introduces a vastly expanded attack surface”

They enumerate sixteen attack classes across five execution phases, including exception-mediated code injection and unsafe capability synthesis. (Preprint, not yet peer-reviewed, and the quantitative results live in figures rather than tables, so I’m citing the qualitative finding only.) They also found code mode losing on some multi-server tasks, because generated code “must encode loop bounds and conditional logic up front” and so handles iterative reasoning badly.

And the critique that actually changed my mind, from an AWS Hero: sandboxing “is not the primary security boundary for code mode.” If the generated code can call a destructive endpoint, the sandbox faithfully isolates a process that is busy destroying your data with fully authorized credentials. The jail contains the code. It doesn’t contain the consequences.

We’re a small team running a connector platform that holds other businesses’ credentials. “Maintain a hardened sandbox forever” is not a line item I can afford to get wrong.

What we built instead: query mode

Here’s the move, and it’s almost annoyingly simple once you see it.

You don’t need a sandbox because the model writes code. You need a sandbox because the model writes code in a language that can do anything. JavaScript can open sockets, read files, spawn processes, exfiltrate credentials. So you build a box that takes all that away, and then you maintain the box for the rest of your life.

But our actual problem isn’t “anything.” It’s: filter, join, group, aggregate. That’s a solved problem with a purpose-built language, and the language is SQL.

SQL is not a general-purpose language, and for once that’s the feature. No filesystem. No network. No process model. No FFI. A sandbox exists to subtract capabilities from a language that has too many. SQL never had them. There’s nothing to subtract.

That’s where we landed at FloConnector, and I’ll give the thing a name, because it deserves to sit beside the other two rather than be described from scratch every time:

Query mode. The model writes SQL, not code. Bulk data lands in an ephemeral database it can query but cannot escape. Only the aggregate reaches the context window. No sandbox, because the language never needed one.

In FloConnector that’s two tools over an embedded DuckDB session that lives about ten minutes and then doesn’t. The shape of it, for a year of invoices:

flc_load("xero_list_invoices", { where: "Date >= 2026-01-01" }, "invoices")
  → { table: "invoices", rowCount: 48213, columns: [...] }     ← a count, not the rows

flc_query("SELECT Status, count(*), avg(Total) FROM invoices GROUP BY Status")
  → 4 rows                                                      ← the answer, not the data

Those rows went to the vendor API, through the pagination loop, into a scratch table, and got aggregated. The model saw a row count and four rows. The other 48,209 never existed as far as the context window is concerned.

That’s an illustration of the mechanism, not a benchmark. I’ll come back at the end to why I’m not dressing it up as one.

Why query mode is more contained, not less

Model-written SQL sounds scarier than it is. Three specifics.

The loader inherits permissions it cannot escape. This is the part I’d point at first, and it’s the design decision I’m happiest with. flc_load doesn’t take a URL or an endpoint. It takes the name of one of the endpoint’s own read tools and fans that tool out over pagination. So profile entitlements, connection scopes, and the read-only ceiling gate the loader for free, because the loader stands on the same tool surface the model already had. It’s structurally incapable of reaching data the caller couldn’t already read. A sandbox starts from a blank file and has to rebuild every one of those boundaries by hand, correctly, forever.

The SQL is fenced four ways. A read-only statement gate (single statement; SELECT, WITH, FROM, DESCRIBE, SHOW, SUMMARIZE only) plus a deny-list is the first line, and on its own it’d be bypassable, because DuckDB has a long tail of file and network table functions. So underneath it: enable_external_access=false structurally kills every file and HTTP reader regardless of which function name reaches it, so no local file reads and no SSRF; allowed_directories is pinned to the session’s own temp dir, the one exception the loader needs to read its own staged rows; and lock_configuration=true with autoload off means the model can’t SET any of it back on, and DuckDB won’t pull in httpfs to reach the network. The deny-list is convenience. The other three are the boundary.

There are no credentials in the room. The vault sits on the other side of the adapter layer. SQL runs over rows that were already fetched. There’s no code path from a SELECT to a token.

Credit where it’s due, incidentally: none of that fencing is our invention. DuckDB ships every one of those controls as first-class configuration, and lock_configuration in particular is the sort of thing you only build if you’ve thought seriously about running untrusted SQL. We’re an embedded consumer of an engine that had already done the hard thinking. The out-of-core spill, the native JSON reading, and the fact that it’s a library inside our Node process rather than a server to operate are the rest of why it’s the right fit.

Compare the threat models honestly. Code mode’s is “arbitrary JavaScript, contained by V8, which by its authors’ own account has more security bugs than a hypervisor.” Query mode’s is “a SELECT over one tenant’s already-fetched rows, on an engine with external access structurally disabled and the config locked, for ten minutes.” I’d rather defend the second one at 2am.

The thing sandboxes are bad at

There’s a structural advantage here I didn’t anticipate, and it has nothing to do with security.

Code sandboxes are ephemeral per execution. Cloudflare’s docs are explicit: “durable state therefore cannot live inside the sandbox,” and a resumed execution “runs the code again in another pass.” Someone on Hacker News hit this wall trying to build roughly what we built: “since the RPC calls are independent Deno processes, you can’t keep say DuckDB or SQLite open.”

That’s the whole ballgame for analytics. The expensive part of “how many invoices” is never the arithmetic. It’s the pagination: at 100 rows a page, the load above is 483 sequential calls against a vendor that rate-limits at 60 a minute. If your compute layer dies at the end of each execution, every follow-up question re-runs that. And follow-ups are the normal case, because real analysis goes “now break that down by month,” “now just the overdue ones,” “now compare to last year.”

flc_load and flc_query are separate MCP calls on purpose, and the session outlives both. You pay the vendor once, then ask eleven questions for free. The ten-minute idle TTL is tuned to exactly that: long enough to hold a conversation, short enough that we’re never running a data mirror. The load result even surfaces its own expiresAt so the model knows what it’s holding and for how long.

A sandbox can’t do this without becoming a database. At which point you have a database, plus a sandbox, plus a security review.

Who got here first

I didn’t invent this, and anyone claiming to have invented an idea this simple is selling something. The components are all published. What’s unoccupied is the position.

• Richard Baxter made the closest argument I’ve found, in SQLite as an MCP context saver (March 2026): “you’re sending 20 aggregated rows into the context window instead of 50,000 raw ones.” That’s the thesis in one line. His is a persistent 90-day mirror over a single API, and never engages with code mode. Same insight, different architecture.
• Ben Lorica described the shape in December 2025: agents need “lightweight, serverless environments (often powered by embedded engines like DuckDB or SQLite) where they can spin up state for a single task, process intermediate reasoning, and discard it.” One sentence, about agent state rather than tool results, but he called it.
• Datadog makes the best “SQL is the interface” case going: “this is exactly the kind of problem I would try to solve with SQL, so why not let agents do the same?” Theirs is SQL over their own backend, not a scratch buffer for other tools’ output.
• MotherDuck’s docs have said the motivating sentence once, in their MCP workflow guide: “It also keeps large intermediate results out of the model’s context window.” Applied to their own warehouse rather than as a scratch buffer, but they said it.

So “nobody thought of keeping data out of context” would be false, and I won’t say it. What nobody has argued is the claim in the title: that ephemeral SQL is a substitute for the sandbox, rather than something you run inside one. In every prior treatment, SQL shows up either inside the sandbox or as a persistent warehouse. The middle is empty.

I’ll be honest about why it’s empty, too. Partly it’s a good idea nobody wrote up. But partly the debate just hasn’t happened: the Hacker News threads on Code Mode run to five and eleven comments. That’s an opening, not a vindication.

The data point I keep chewing on is that Anthropic looked straight at this use case and reached for JavaScript. Their post describes exactly our scenario, “the agent sees five rows instead of 10,000… aggregations, joins across multiple data sources, or extracting specific fields, all without bloating the context window,” and the answer is a filesystem of TypeScript modules. The words sql, sqlite, database and duckdb appear zero times in that post. They didn’t reject SQL. It doesn’t seem to have come up.

When I’d reach for a sandbox anyway

I’d be doing the thing I’m criticising if I pretended query mode wins everywhere.

Use code execution when the work is genuinely procedural: multi-step branching logic, six systems in sequence with real control flow, transformations that aren’t relational. SQL is a bad general-purpose language precisely because it’s a good specific one. If your problem isn’t filter-join-group-aggregate, the constraint that makes SQL safe is just a constraint.

Use it when you’re moving payloads, not summarising them. Anthropic’s transcript example, where a document goes from Drive to Salesforce untouched, isn’t an aggregation. There’s nothing for SQL to do.

And there’s a fair objection to ours, made on Hacker News: “SQL is really a perfect solution for allowing the agent to access data, but in most applications, it’s not realistic to provide it with a db connection.” Multi-tenancy, cross-source joins, connection management. That’s correct, and it’s precisely the work we do: the session is the tenancy boundary, keyed per workspace and profile and caller, holding only rows that endpoint could already read. The objection isn’t wrong. It’s a description of the hard part, and the hard part is the product.

But the common case for a connector platform is a business asking a business question about its own records. That’s filter-join-group-aggregate, all the way down.

About those numbers

One more thing, because this debate has a hygiene problem and I don’t want to add to it.

Every headline figure quoted for code mode is either illustrative or definition-only. Anthropic’s famous 150,000 → 2,000 is a worked example, not a measurement: no methodology, no eval harness, no sample size. Cloudflare’s 99.9% (1.17M tokens → ~1,000) counts tool definitions only for their 2,500-endpoint API via tiktoken. It’s an honest number measuring a different thing than most people think it measures.

The figures from people actually running these systems look different. Anthropic’s own Programmatic Tool Calling reports 43,588 → 27,297 tokens, a 37% reduction, on internal tests. Block’s Goose team measured about 30% on a single task, and said so. That’s the honest range: real, useful, and about a tenth of the headline.

I’m not going to hand you a percentage of my own, because I’d be doing the same thing. My claim isn’t that my number is bigger. It’s that when the question is relational, you can have the token savings without buying the attack surface, and the reason isn’t clever engineering. It’s picking a language that can’t do the things you’d need a sandbox to prevent.

The principle

The MCP spec has, so far, declined to solve any of this. Reference-based results sit under “On the Horizon” rather than in a priority lane, described as letting “clients decide when to pull large payloads into context rather than polluting it by default.” Right idea. Until it lands, servers answer for themselves.

Ours: don’t build a jail for a language that needs one. Pick a language that doesn’t.

Trusted code is code we authored. Contained code is code that structurally cannot reach anything worth reaching. Query mode is the second thing, and it answers the question the business actually asked, which was never “please execute arbitrary code on my behalf.” It was “how many invoices did we pay last quarter.”

Written by

David List

Founder, FloConnector

Building the hosted MCP layer that lets a business's AI actually run its software. Previously deep in field-service and accounting integrations.

floconnector.com