The Virtual Private Token Factory: Full Utilization, More Tenants, and No Plaintext on Your Platform

For neoclouds, NVIDIA Cloud Partners, sovereign and in-country cloud operators, and AI factory builders: the virtual private token factory is how to run multi-tenancy that regulated customers can join and that a single mistake cannot turn into a leak, at the economics the factory was financed on.

Every AI factory is financed on the same argument: shared, multi-tenant capacity, running near full, serving many customers from one footprint. And every operator eventually runs into the same contradiction. The customers with the highest and steadiest token demand are banks, ministries, health systems, defense primes and insurers, and their data policies prohibit sensitive material from appearing in the clear on infrastructure they do not operate themselves. Ironically, the demand that justified the buildout is the demand the platform cannot legally or contractually accept. Nowhere is that contradiction sharper than in sovereign and in-country programs, whose entire mandate is to serve a nation’s most regulated institutions from national capacity.

The standard workaround is hardware isolation: carve off dedicated capacity per customer and run it as a private environment. It satisfies the data policy at the expense of the business case. A carveout is capacity reserved for one customer that could have served many. It idles between jobs, scales on its own schedule instead of the platform’s, and pins utilization on that footprint at 20 to 40 percent, on hardware that multi-tenant operation would run at 90 percent [1]. That gap is the isolation tax, and it compounds: every regulated customer won under this model converts more shared capacity into siloed capacity.

Booked is not busy. Before an operator buys another rack, the more valuable question is what is preventing the racks already financed from earning.

The requirement behind the carveout

The blocker was never the GPUs being shared. Multi-tenant scheduling decides who gets compute; it says nothing about what the serving environment can see while that compute runs. That is the reason regulated tenants stay off the platform, and it is just as true for the tenants already on it, who are there in plaintext.

Data is encrypted at rest and in transit, and then inference requires the stack to work with usable representations of it. At the endpoint, that happens in the clear, and from there the input is present across the full operational surface of the serving environment: request and response logs, debugging traces, prompt and KV caches, scheduler records, observability tooling, and memory on the host. Zero-data-retention policies help, but a policy is a promise the serving stack has to keep at every surface, every time, and the field record on how well that promise holds is now public.

On August 30, SemiAnalysis published the security findings from its ClusterMAX 3.0 audits: 25 providers and 32 clusters tested over four months, every finding disclosed to the provider and remediated before publication [3]. The list of what the auditors managed to do is long, and three items on it end the same way, with someone in a position to read another tenant’s inference traffic. They read data cross-tenant on inference endpoints that were serving public traffic, through misconfigured Kubernetes services. They broke out of containers and VMs on shared servers, escalated to root, and attacked the neighbors on the same machine. And they chained smaller mistakes into cross-tenant remote code execution, on providers whose tenant lists included banks, telcos, universities, AI labs and, in one case, a national intelligence agency.

The report’s own definition of the problem is the useful part. It calls a design bad when a single mistake, one unpatched CVE or one provider-side misconfiguration, leads to immediate cross-tenant exposure, and it says that proper designs have layers of security built in. Every fix it recommends is right, and operators should make all of them. Each one lowers the odds that a mistake happens. None of them changes what a mistake finds, because on a conventional serving stack whatever a failure exposes, it exposes in the clear: the prompts, documents and context of whichever tenant was unlucky that day. None of this is an argument against multi-tenancy; multi-tenancy is the business. It is an argument against the version of it in which a tenant’s prompts are one misconfiguration away from a neighbor.

A regulated institution reads the surface list and the audit findings the same way its auditors do. So it demands its own hardware, or it stays off the platform entirely. Either way, the operator loses: margin to idle carveouts, or revenue to workloads that never arrive.

The virtual private token factory

Protopia AI and Rafay have built, tested, and documented an architecture that changes what multi-tenancy means on the platform: the tenants already on shared capacity stop being there in plaintext, and the requirement that kept regulated tenants off it goes away instead of being priced around. Underneath every control the operator runs, it adds the kind of layer the report’s definition asks for, one that holds when something above it slips. It runs end to end today. Each layer contributes something specific.

Protopia AI Stained Glass Transform is the inference privacy layer. It converts sensitive inputs into protected representations inside the data owner’s own trust boundary, before anything leaves. The target model reasons over those representations directly and returns usable results. There is no decoder on the serving side, no reversal step, and no point in the pipeline where the original input can be reconstructed; with output protection enabled, responses return protected as well. On the operator’s side of the data path there is no plaintext to expose, and a security team can verify that by inspecting where the transform runs rather than by auditing every log, cache and trace downstream of it. A container escape, a misconfigured service or a stolen credential on the platform reaches the same thing every legitimate component does: representations the model can use and no one can turn back into the input.

Rafay Token Factory is the layer that turns GPU infrastructure into that business in the first place. Built on the Rafay Platform, the control plane operators already use for orchestration, multi-tenancy, governance, and self-service consumption across cloud, on-prem, sovereign, and air-gapped environments, Token Factory delivers open models, including the NVIDIA Nemotron 3 family, as OpenAI-compatible, token-metered endpoints. Authentication, rate limits, tiers, quotas, real-time metering, and billing-ready chargeback come built in. In Rafay’s own framing, token-based delivery is what separates an AI factory operator from a commodity GPU provider [4].

Compose the two and every tenant gets what behaves like its own private token service, its data crossing into the operator’s environment only as protected representations, while the operator runs one shared, governed, metered platform underneath all of them.

The name describes exactly how it works: it is Virtual because privacy is managed by software at the customer’s location, rather than requiring separate physical hardware. It is Private because what crosses into the operator’s environment is a representation the model can reason over and no one can turn back into the original input, not the operator, not its tooling, and not a neighbor that has ended up somewhere it should not be. And it is a Token Factory because it plugs directly into existing systems for billing and monitoring, meaning the operator’s workflow stays exactly the same.

Isolation, resized: share the model, not the exposure

One consequence of the architecture changes what isolation costs.

In today’s model, isolating a tenant means isolating the heaviest thing in the stack: the model serving infrastructure itself. That is racks of accelerated compute, reserved and idling, per tenant. The expense of isolation is the expense of the model.

With Protopia Stained Glass in the path, the component that touches sensitive data before protection is the transform, and the transform is computationally lightweight: a small software artifact, created once per target model and run at the data owner’s boundary, with a compute footprint that is a fraction of serving the model it protects. So for tenants who still want dedicated infrastructure for something, the something shrinks from the model to the transform. Isolate the lightweight layer per tenant, wherever the tenant wants it, in their own facility, their own VPC, or a dedicated slice the operator hosts, and leave the heavyweight model serving fully shared, running at maximum throughput.

Tenancy becomes a range rather than a yes or no, and the operator can offer all of it from one platform:

• Fully shared. Tenants run the Stained Glass Proxy in their own environments, and every request lands on common capacity as protected representations. Maximum utilization and the lowest cost per token, and a neighbor that gets somewhere it should not, the failure the audits reproduced on shared servers, finds representations rather than prompts. This is the default.
• Isolation where and when a tenant insists on it, applied only to the transform layer. The tenant gets a dedicated, auditable privacy boundary it can point regulators at, without a single GPU being pinned to its name. The factory’s serving tier never fragments.
• Hybrid and burst by economics. Because protection travels with the data, the same workload can run on-prem, in a sovereign region, or on partner capacity based on cost and availability rather than on where the data is allowed to appear.

Every option enables the accelerated compute to be pooled, which is what the economics depend on, and the isolation conversation with a customer stops being a pricing concession and becomes a configuration choice.

What the P&L sees

Rafay solved serverless delivery at the infrastructure level well before this partnership. What the privacy layer adds is the ability to run it for every tenant: the ones already on shared capacity, whose data is no longer in the clear there, and the ones who could never touch it before.

For the operator, the workloads that used to demand carveouts run on shared capacity, moving utilization on that footprint from 20 to 40 percent toward 90 percent [1], with cost-per-token falling in proportion. In a modeled twelve-customer deployment, annual infrastructure cost for those workloads drops from $413K to $47K, about 89 percent lower, with actual savings depending on workload mix and scale [2]. Return on the same invested capital rises for one reason: more of the factory is earning, more of the time, from more tenants.

For the enterprise on the other side of the endpoint, the deal improves too: a better price point, because it is paying for infrastructure shared holistically rather than time-sliced in its name, with its data never in the clear on the operator’s side. Security stops being an operator cost and becomes an operator product. Every operator claims isolation, and the audits are a reminder of how many components that claim rests on. The claim this architecture supports is different in kind: not that the platform guards a tenant’s data well, but that the data is not on the platform to be reached. Regulated customers pay a premium for that, on infrastructure everyone shares, so it lands as margin rather than overhead.

There is a reason Rafay frames enterprise adoption in cloud terms. Enterprises scaled onto AWS, GCP, and Azure because consumption came with controls, guardrails, quota management, auditability, and observability at every layer of the stack, and they will scale onto AI factories on the same condition. Token Factory brings that operating discipline to token delivery; the virtual private token factory extends it to the data itself. Together they give a regulated institution the thing it is actually buying: cloud-grade consumption with a privacy boundary it can verify.

The sovereign case: in-country is necessary, not sufficient

Sovereign and in-country platforms are the purest form of everything described above. They exist to serve ministries, national banks, healthcare systems, defense organizations, and critical industry from capacity that stays inside the country, which means they carry the most regulated tenant mix any operator faces, on GPU supply that is scarcer and more expensive than anywhere else, with economics that still depend on multi-tenancy to work at all. A sovereign operator that answers every sensitive tenant with a carveout is fragmenting exactly the national capacity it was funded to pool.

And the tenants’ auditors have moved past the first question. Keeping data in-country answers where the data sits. It does not answer what can see the data while it is processed, and inside a national facility, plaintext is still plaintext: present in the same request logs, caches, traces, and operator tooling as anywhere else, visible to whatever serving stack, staff, and third-party software the facility runs. That is the question ministries and national banks now put to their own national platforms, and the audits make it concrete: the tenant mix on the affected providers was the sovereign tenant mix by another name, down to a national intelligence agency, and none of the misconfigurations that exposed them had anything to do with where the racks were.

The virtual private token factory gives the sovereign operator a stronger answer on both counts. Data stays in-country, and it crosses into the national serving environment only as protected representations, transformed inside each institution’s own boundary before it ever reaches the shared platform. Rafay’s platform already operates in-region, private, and air-gapped environments; the transform layer completes the picture by making the shared national footprint safe for tenants that could never share before. Ministries, banks, and agencies stop being carveouts on scarce national compute and become tenants on a pooled national factory, each with a privacy boundary it controls and can show its auditors. For operators standing up capacity under national mandates across Europe, the Gulf, and Asia, that is the difference between a sovereign platform that hosts a few isolated workloads and one that serves the whole institutional economy it was built for.

Dark data, finally in the pipeline

The deepest effect is on the demand side. Every operator’s addressable market has been quietly shaped by the data that could never be sent: customer records and transaction histories in banking, clinical notes and claims in healthcare, mission and telemetry data in government and defense, proprietary code and process IP in industry. This is the dark data of the enterprise, high-value, heavily governed, and historically never fed into shared AI infrastructure at all. It is also the data behind the use cases with the strongest ROI, and most of that return is still on the table: the most interesting version of the workload was not allowed on shared capacity, and the isolated version cost more than it earned.

When that data can cross into the virtual private token factory only as protected representations, those use cases come onto the platform, and they arrive as its heaviest, steadiest token volume. The same holds inside a single enterprise: business units that could not expose data to one another on the internal AI factory stop needing internal carveouts, and the platform team serves more of the organization from the same footprint.

Agentic workflows multiply all of it. One agent task is, in effect, many model calls, the harness reasoning, calling tools, and reasoning again, and every pass is another metered request and another chance for sensitive context to land in a log or cache. The report’s own example of who is exposed first on a compromised endpoint was agentic and coding-harness traffic, which follows: an agent sends more context, more often, through the same endpoint. Protopia’s SafeCLAW extends the same pattern into agent harnesses, routing every sensitive call in the loop through the local transform, so the operator captures the full agentic workflow of exactly the customers whose agents will run hottest. The economics of these platforms depend on high degrees of multi-tenancy, and agent-scale demand does not change that; it raises the stakes on it.

The questions that decide it

None of this turns on a feature comparison. It turns on how an operator answers a few questions about where its platform is going.

• Is your constraint really supply, or is it allocation? If booked capacity is not busy capacity, more GPUs deepen the problem they were bought to solve.
• When agentic traffic multiplies token demand, will your most regulated customers meet it on your platform, or build around you? The institutions with the strictest data policies also have the deepest agent roadmaps.
• Is isolation a security requirement, or has it become a pricing decision? If the privacy boundary can be delivered in a lightweight software layer at the tenant’s edge, dedicating racks to it is a choice, and an expensive one.
• When a mistake or breach happens on your platform, what does it find? The audits say one misconfiguration is enough, and on a conventional serving stack it finds customer prompts and context in the clear. The same mistake finding nothing readable is a different product to sell.
• If your mandate is national, how much of the nation’s data can actually reach your platform today? In-country capacity that the country’s most important institutions cannot send their real workloads to is a mandate half met.

Actions to take

1. Run the booked-versus-busy numbers on your own footprint, then list the accounts your current architecture prices out. That list, repriced at shared-infrastructure economics, is what this opportunity is worth on your platform.
2. Have your platform team pressure-test the reference architecture against the SemiAnalysis findings: for each one, ask what it would expose on your platform today, then ask the same question with the transform in the path. The request path, serving-side configuration, and input and output protection are documented in the joint post on Rafay’s blog.
3. Watch the conversation where the term was coined, twenty-six minutes with both CEOs on the economics and the architecture.

Next steps

No single vendor builds an AI factory. In practice, eight to ten come together on every one, in the ecosystem pattern NVIDIA has set for the industry, and Rafay and Protopia sit at two of the choke points that decide whether a factory’s economics hold: how tokens are delivered, metered, and monetized, and whether the data that matters most is allowed to arrive. Budhani has said Rafay is building toward “the success of a thousand AI factories across the world.” The virtual private token factory is how those factories get to run multi-tenancy safely, for the tenants already on shared capacity and for the ones no shared factory has been able to serve.

Both of the events below are in Europe for a reason. EMEA is where the sovereign and in-country buildout is moving fastest, where national programs, telcos, NCPs, and regional clouds are standing up capacity under national mandates, and where the questions in this post get asked in their sharpest form. If you operate, or are standing up, shared AI capacity anywhere in the region, these conversations are for you.

We are setting up these conversations with operators now. Protopia AI and Rafay will both be at Rafay’s AI Infrastructure Leadership Summit in Barcelona, September 8 to 10, an invitation-only executive forum for operators of shared AI infrastructure. We will also be at NVIDIA GTC in Berlin, October 20 to 22.

If either works for you, tell us you are coming and we will hold time. If neither does, we will set up a working session directly against your numbers, with both teams in the room.

Get in touch

Notes and sources

1. Protopia AI analysis of AI factory utilization under dedicated-carveout versus multi-tenant operation. See protopia.ai/solutions/ai-factories.
2. Protopia AI deployment modeling, twelve-customer scenario, comparing dedicated capacity per customer against shared capacity with Protopia Stained Glass Transform. Modeled figures; actual savings depend on workload mix and scale.
3. SemiAnalysis, “Most Neoclouds Suck at Security,” ClusterMAX 3.0 security audit findings, August 30, 2026. Findings were disclosed to the affected providers under a 90-day embargo and remediated before publication.
4. Rafay’s Token Factory adds a governed, token-metered service layer on top of existing GPU infrastructure, turning raw inference into a monetizable AI service rather than capacity billed by compute time.