This post is for platform and infrastructure leads wiring agents into real systems. You have a few MCP servers, more than one tenant who needs them, and you want a single endpoint in front of them. My position is that all of the tenancy belongs in the proxy and none of it belongs in the backends. The rest of this post covers the decisions that follow from that and the ones I would not compromise on.
Keep the backends tenant-unaware
The clearest worked example in this space is a multi-tenant agentgateway demo. In it, three companies hit the same MCP URLs and get different tool sets: Acme sees 25 tools, Globex 13, Initech 6. The servers behind the gateway contain no authentication, authorization, quota or billing code. Those tests ran on Solo Enterprise for agentgateway, not the open-source build. The pattern is what matters here. If you build it yourself, choose an MCP-aware proxy that supports the authentication, authorization and federation capabilities you need: identity verification per tenant, policy at the level of individual tools, and fan-out across several backends behind one endpoint.
I’d adopt that split wholesale. If tenant logic lives in each backend, every new server is another place to get authorization wrong. If it lives in the proxy, adding a backend is a routing change and adding a tenant is a policy change. The same post makes a second argument I agree with: group servers by business domain, not by whatever you happened to deploy. It cites Anthropic evals showing a model scoring 49% with a full tool catalog in context against 74% with on-demand discovery. One flat namespace with every tool in it hurts the agent, and it gives you no boundary to hang policy on.
Namespace tools and deny by default
The proxy has to rename tools so that two backends can’t collide. The ClaudeMCP gateway qualifies names as server:tool, and the agentgateway demo uses target_tool. Pick a deterministic scheme and make sure your policy matches on it. The agentgateway post notes a trap here. Its policy matches the origin tool name, so a rule that omits the backend target would also allow a same-named tool on any other server in the federation. Whatever you build, write a rule that covers a backend plus a tool, never a tool name alone.
Then deny by default. In the agentgateway setup, a tenant with no billing entitlement authenticates fine and sees an empty tool list. A tools/call for an unentitled tool returns Unknown tool, so the caller can’t even confirm the tool exists. I’d treat both halves as requirements. Filter tools/list, and enforce again on tools/call, because a client can call a name it was never shown. If your tenants need finer control, MCPGateway describes policy that can constrain individual argument values, not just tool names. I would add that only once tool-level allowlists stop being enough.
Identity in, credentials out
Every policy decision should key off a verified identity. API keys can work: the ClaudeMCP gateway checks keys with a timing-safe comparison and then runs two authorization checks that must both pass. If your tenants bring their own identity providers, the agentgateway approach is cleaner. The gateway reads the token’s iss claim to pick the provider, and it pins audiences so a token minted for some other service is rejected even with a valid signature.
On the outbound side, never forward the tenant’s token to a backend. mcp-proxy states this explicitly: backend credentials are configured separately and inbound tokens are never forwarded. fold describes exchanging the client’s token per upstream using RFC 8693 or injecting service credentials, so API keys never reach agents. Either approach works. What I would not accept is a design where an agent holds a backend’s credentials.
Contain failures and cost per tenant
Shared infrastructure means one tenant can hurt another. MCPGateway names the failure modes plainly: one slow upstream stalls every agent, and one buggy agent exhausts a shared quota. Its answer is to scope rate limits, bulkheads and circuit breakers per tenant and per provider. I’d make per-tenant quotas the minimum and add per-backend resilience next.
Also decide what happens when one backend is down. The agentgateway demo’s FailOpen setting keeps serving healthy targets, while its default, FailClosed, fails the whole session. For tools spread across unrelated domains I’d fail open. For a workflow where a missing tool would mislead the agent, failing closed is the safer choice. fold also separates rate limits from budgets, which accumulate across an hour, a day or a month. That distinction matters if your tenants pay per use.
When not to do this
If you have one team and one trust boundary, a multi-tenant proxy is overhead. A plain aggregating proxy with authentication is enough. The tenancy layer pays off when entitlements differ between callers. In the agentgateway demo, a standard-tier customer had more analytics tools than the enterprise account because it bought an add-on. If nobody’s view of the catalog differs, skip the entitlement model until someone’s does.
When it does apply, build it in this order. Make the backends tenant-unaware, namespace the tools, deny by default on both list and call, key every rule on a verified identity, never forward inbound tokens, and scope quotas per tenant. Get those right before you add anything else.