How FloConnector keeps 50+ connectors out of your context window

Give an AI client one connector and everything is fine. Give it fifty and something quietly breaks: the model starts every conversation already exhausted. Not because of anything you asked, but because of everything it was told it could do before you asked anything at all.

That’s the problem FloConnector had to solve before a 50-connector endpoint could be anything other than a demo. The answer is deferred tools, and one detail about where they come from matters more than it looks. Deferred loading is not part of the Model Context Protocol. The spec has declined it three times over. It arrived at the layer above: the API and the client. That distinction shapes everything a server like FloConnector can and can’t do about its own context cost.

The tax you pay before the first token

Every tool an MCP server exposes ships a definition: a name, a description, a JSON schema for its inputs, per-field docs, enums, examples. The model has to read all of it to know the tool exists. Individually it’s nothing. In aggregate it’s the single biggest line item in a modern agent’s context budget.

The numbers are worse than most people expect. A single popular server, GitHub’s official MCP, spends around 17,600 tokens of tool definitions on every request. Stack a handful of servers together and you’re in five figures before the user types a word:

For a product like FloConnector, whose entire pitch is “one endpoint, every platform your business runs on,” this is existential. The catalog is 50+ connectors across email, calendars, CRM, storage, marketing and more. Expose them all the naive way and the model walks into every conversation carrying an encyclopaedia it will use two pages of.

And the cost isn’t only tokens. When the model has to attend across a 200K window stuffed with definitions, the signal from the actual task gets diluted. This is the well-documented “lost in the middle” effect: accuracy drops when the relevant material is buried in noise. More tools, loaded eagerly, can genuinely make an agent worse at picking the right one.

Deferred tools: pay for what you reach for

On 24 November 2025, Anthropic shipped the Tool Search Tool on the Claude Developer Platform, and it flips the default on its head.

Instead of loading every definition upfront, the caller marks tools with defer_loading: true. Deferred tools aren’t placed in context. The model sees only a compact search tool plus whatever handful of critical, always-on tools stay eager. When it decides it needs a capability, it searches for it, and only then does the full definition expand into context.

{
  "tools": [
    // eager: the model sees this immediately
    { "type": "tool_search_tool_regex_20251119", "name": "tool_search" },

    // deferred: discoverable, but not loaded until searched for
    { "name": "gmail_send_email", "defer_loading": true, "input_schema": { /* … */ } },
    { "name": "hubspot_create_contact", "defer_loading": true, "input_schema": { /* … */ } }
    // …50 more
  ]
}

The platform ships two search backends out of the box: a regex matcher (for when the model roughly knows the name it wants, e.g. send_.*email) and a BM25 matcher (natural-language, better for exploratory “I need to email a customer” queries). You can bring your own with embeddings if you want.

One detail worth internalising, because it catches people out: defer_loading controls what enters the context window, not what you send. Every tool definition still goes up on every request, because the API needs them server-side to run the search over. The saving is in the model’s attention and the context budget, not in your request payload.

The payoff is dramatic:

Anthropic also reports it’s more accurate: on its internal MCP evaluations with large tool libraries, deferred discovery lifted Claude Opus 4 from 49% to 74%, and Opus 4.5 from 79.5% to 88.1%. Giving the model less to look at made it better at choosing.

Worth being straight about the provenance, though: those are Anthropic’s own numbers, on unnamed internal evals, with no published methodology or sample size. The one independent test we’re aware of is less flattering. Arcade.dev pointed the search at 4,027 tools and found the correct tool surfaced in the top results about 60% of the time, with regex missing “send an email” for Gmail. Arcade sells a competing runtime and the sample was 25 tasks, so treat the headline figure as weakly powered. The specific failures are the instructive part, and they point at something a server author can actually act on, which is the rest of this post.

The part nobody tells server authors: you don’t own this lever

Here’s where most write-ups on deferred tools go wrong, and it took us a beat to get straight ourselves.

An MCP server cannot defer its own tools. defer_loading is set by whoever calls the model: the application, or the client. In the Claude API you don’t even set it per tool for MCP; you set it on the toolset’s default_config. FloConnector is a server. It hands over a tool list and has no say in whether those definitions land in the model’s context eagerly or on demand.

The good news is that the client usually gets it right without being asked. Claude Code defers MCP tools by default: unset ENABLE_TOOL_SEARCH and everything is deferred; set it to auto and it loads tools upfront only while they fit inside 10% of the context window, deferring the overflow. So a FloConnector workspace with a dozen connectors already opens lean in Claude Code, not because of anything we do at request time, but because the client made a sensible default.

Which reframes the job. If you build an MCP server, your influence over context cost is not whether your tools get deferred. It’s whether they’re worth finding once they are. That’s a narrower job, and a real one:

• Keep the entitled surface honest. FloConnector exposes only the tools for the vendors a workspace has actually connected, selected per profile. Search works better over 40 relevant tools than 400 speculative ones, and this is the one lever that’s wholly ours.
• Write descriptions that survive the trip. Claude Code truncates tool descriptions at 2KB. A description that buries its keywords past that limit is a tool the search can’t find, which is exactly the Gmail “send an email” failure Arcade documented.
• Treat server instructions as routing, not branding. With tool search on, server instructions are how the model decides to search you at all. They matter more than they used to, and they’re capped at 2KB too.

The design rule we hold ourselves to: the model should pay the context cost of what it uses, not what it’s entitled to. Entitlement is broad on purpose. Attention is scarce on purpose. We don’t own the mechanism that keeps both true, but we own everything about whether it works.

There’s a second lever we’ve designed and not yet shipped: progressive disclosure at the vendor level, via a server-side enable_vendor meta-tool plus MCP’s notifications/tools/list_changed, so a workspace with many platforms could surface one vendor’s toolset at a time. That one would be genuinely server-side, the rare context lever a server can pull unilaterally. It’s on the roadmap, not in the product. When it ships, we’ll say so.

The adjacent idea: don’t move data through the model at all

Deferred loading solves the cost of knowing about tools. There’s a sibling technique for the cost of using them: code execution with MCP. Instead of every intermediate result flowing back through the model’s context, the agent writes code that calls tools and processes results in an execution environment, returning only what matters.

Anthropic’s illustration is stark: a workflow moving a meeting transcript between two systems, which it describes dropping from 150,000 tokens to 2,000 by never round-tripping the payload through the model. That figure gets quoted everywhere as a benchmark, so it’s worth noting it’s a worked example rather than a measurement, with no methodology and no eval harness behind it. The arithmetic is plausible for the scenario as constructed. It isn’t evidence.

The underlying principle survives the caveat, though, and FloConnector’s analytics path runs on it. Large reads go through an ephemeral, in-memory DuckDB session: flc_load paginates vendor rows into a scratch table and returns a row count, flc_query aggregates and returns the small answer. Only the aggregate reaches the model. The raw rows never enter context.

Where we part company with the code-execution camp is what language does the aggregating. The model writes SQL, not JavaScript, and that choice buys a containment story a sandbox has to build from scratch. That’s a whole post of its own, and it’s the next one.

Different problem, same philosophy: context is the budget, and you spend it on decisions, not on plumbing.

The takeaway

The instinct when you build an integration platform is to show the model everything it can do. Deferred tools prove the opposite is correct. The endpoint that quietly wins is the one that shows the model almost nothing, just a way to find capabilities, then lets it pull in the exact tool for the job, right when the job appears.

The wrinkle is that you probably don’t get to make that decision. The client does, and the protocol stayed out of it entirely: the MCP spec rejected tool filtering (SEP-1300), left dynamic tool discovery unsponsored (SEP-1821), and closed lazy tool hydration (#1978). Everything that shipped, shipped one layer up.

So the job for a server is smaller than the blog posts imply, and more concrete: be findable. Keep the surface to what the workspace actually connected. Put the keywords where a 2KB truncation can’t eat them. Then get out of the way of a client that’s already trying to do the right thing.

That’s how 50+ connectors fit through one port without drowning the thing on the other end of it.

Written by

David List

Founder, FloConnector

Building the hosted MCP layer that lets a business's AI actually run its software. Previously deep in field-service and accounting integrations.

floconnector.com