Inference is where your AI is most valuable and most exposed, because that’s when your model is running on previously unseen datasets to derive new learnings. Adversarial AI has pushed software defenses to their weakest point, and AI-driven zero-days now outrun the patch cycle. Enterprises, AI labs, ISVs and governments are moving AI to neoclouds for cost-efficient compute. For them, compromised infrastructure and model theft now top the risk list, slowing neocloud adoption.
The reason is simple. To run, a model loads its weights, customers’ prompts, keys and data into CPU and GPU memory, on hardware someone else runs. Anyone with admin access to that host sits one hop from your crown jewels: an operator, a contractor, or an attacker holding a stolen credential or an AI-built exploit.
Confidential inference closes that gap. The model runs in the neocloud inside a hardware-isolated trusted execution environment (TEE) that spans CPU and GPU. Weights and data effectively stay encrypted throughout the entire lifecycle, and attackers and even root users have no access to them - not limited access, or gated access - absolutely zero access. Cryptographic attestation proves this is true before models are loaded and decrypted into the TEE, given trusted configurations, keys and data to compute.
The silicon is ready: as of October 2026, NVIDIA supports confidential computing from a single H100 up to a full 72-GPU rack with B200s, B300s and the latest Vera Rubin chipset. The hard part is no longer whether the hardware is available. It is trust: how and who verifies the proof, who holds the keys, and who can pull them back; and managing this across multiple GPUs, nodes and rack-scale infrastructure. That is the problem Anjuna was built to solve. Confidential computing for the world's most valuable emerging companies, enterprises, model providers and AI-native businesses is prime-time ready. We’ve proven this in the world's most demanding enterprises. Now we bring this capability to mass AI adoption in neoclouds.
The risk: whoever controls the host controls your model
Encryption at rest and in transit stops at the moment of use. To serve a token, the stack must hold weights, prompts, outputs and keys in the clear. That is exactly what a privileged insider or a host-level attacker can read.
The result is a standoff. Model vendors won't hand weights to infrastructure they can't verify. Enterprises won't send sensitive data to a model they can't inspect. Both sides default to "keep it in-house," and the cheapest, fastest GPU capacity goes unused for the workloads that matter most.
This is not hypothetical. Look at Fireworks AI. Fireworks is a hosted inference and fine-tuning platform with tens of thousands of customers. In five weeks it disclosed two incidents, and every step ran through infrastructure, not through the model (The Hacker News, TechCrunch):
- Oct 6: Attackers using stolen internal credentials read environment variables, including API keys and tokens, inside customer evaluation jobs.
- Oct 23: A third-party cloud provider hosting Fireworks R&D machines is breached.
- Oct 24: Attackers pivot into Fireworks development machines and run their own workload, reaching billing data, datasets and incomplete model weights for some customers.
- Nov 9–11: Fireworks widens its disclosure. Notion confirms Notion AI chat transcripts from fewer than 1% of its workspaces were exposed, and stops serving open-weight models through Fireworks.
Fireworks linked the attackers to a financially motivated group that resells stolen LLM access, a scheme known as LLMjacking. Note the pattern: stolen operator credentials, a compromised infrastructure provider, a rogue workload running beside customer data. One provider's breach became its customers' breach.
In a well-architected confidential inference design, that same intrusion meets ciphertext. Weights and datasets stay encrypted outside an attested TEE. Secrets release only to workloads whose measurements match policy, so a rogue workload fails attestation and gets no keys. The intrusion still happens; the weights, datasets and secrets stay out of reach - the attack is neutralized and meaningless. No regulated data is exposed, and weights are intact. The evidence proves this to regulators and investigators.
AI has turned this from a theoretical risk into an operational one. The time between disclosure and exploitation has collapsed. The Zero Day Clock project measured a mean time-to-exploit of minus 8 hours in July 2026, down from 21.5 days in 2025, and more than 80% of exploits seen in the wild now target zero-days (SecurityBrief). Meanwhile, the median enterprise took 43 days to fully remediate known-exploited vulnerabilities, up from 32 the year before (CSA research note). Anthropic and its Project Glasswing partners found more than 10,000 high- and critical-severity zero-days in their first month of searching (daily.dev summary).
The math is simple - and alarming. Attackers now move in hours; defenders patch in weeks. Patch updates to open source repos inform attackers of a window of opportunity, and exploitation is near instant. You cannot patch your way to safety. You have to make the host irrelevant. If a compromised hypervisor, a rogue admin or a stolen root credential exposes nothing, the zero-day under it stops mattering to your model.
Why neoclouds feel this first
Neoclouds win on price per GPU-hour and on getting the newest silicon first. AI labs, model providers and ISVs want that capacity. But when a buyer weighs a neocloud for a closed model or regulated data, the first questions are rarely about throughput. They ask:
- Who on your staff has root on the host my model runs on?
- How mature is your security program, and can you prove it?
- After a successful attack, what controls are in place to prevent impact to my models?
- If you are breached tomorrow, are my weights in scope?
A neocloud that answers with policies and SOC reports is asserting trust. A neocloud that answers with hardware attestation, checked by someone other than itself, is proving it. That difference decides which workloads it can win. After Fireworks, expect these questions on the first call, not the last.
Confidential inference turns "trust us" into a product feature. For the neocloud, that means:
- Winning workloads it can't today. Premium closed models and regulated data that vendors and buyers won't put on shared infrastructure. That demand currently goes to the vendor's own hosted API or stays on-prem.
- Differentiation beyond price. "Verifiably operator-blind" is something a neocloud can sell, and something a hyperscaler that also sells competing models struggles to claim with the same neutrality.
- Smaller blast radius. If operators can't see weights or prompts, a breach or a rogue admin becomes a much smaller event, and the compliance story with enterprise customers gets simpler, as do investigations and audits post-incident.
- No privacy tax. On current-generation GPUs, confidential mode runs close to native speed (details below), so customers no longer trade performance for control.
The same logic holds for the model provider or ISV on the other side. Confidential inference lets it ship weights to any customer's cloud, keep keys and license terms under its own control, and sell to regulated and sovereign buyers who will never call a hosted API.
What confidential inference actually does
Confidential inference protects three things at once: the model in use, the data in processing, and the keys that unlock both. It does it with hardware, and it proves it with cryptography, not paperwork.
The flow, start to finish:
- Pick the cloud. The model vendor or enterprise chooses their neocloud, co-lo or on-prem site with confidential CPUs and GPUs.
- Encrypt and measure. The model is encrypted before it leaves the owner, packaged with a confidential VM image, and fingerprinted so any change is detectable. The owner sets policy: region, term, instance count, what the model may connect to.
- Launch in a TEE. The infrastructure provider boots the image on confidential GPUs. It runs the hardware; it never sees inside.
- Verify, then release the key. The CPU and GPU each produce signed evidence of their firmware, mode and measurements. An independent verifier checks it against policy. Only then does the owner's key manager release the decryption key, straight into the TEE.
- Serve inference, blind. Weights decrypt only inside the TEE. Prompts and outputs stay encrypted to everyone outside it, including the operator. The TEE keeps everything sealed yet fully functional.
- Heartbeat: continue or revoke. The workload re-attests on a schedule. When drift or unauthorized change in firmware, code, policy, location or license terms is detected, the key is pulled, and workloads stop.
No one in that chain has to trust the operator. That is the point.
The hardware is ready. Here is what runs where.
Three GPU generations now support confidential mode, and each one moved the trust boundary outward: from one GPU, to one 8-GPU board, to an entire rack, and multi-nodes. Every combination needs a matching CPU TEE (Intel TDX or AMD SEV-SNP on x86, Arm CCA on Vera), and the CPU and GPU must both be in confidential mode to work.
Sources: NVIDIA Confidential Computing; NVIDIA Rubin platform blog; GMI Cloud on Confidential Bare Metal; AMD SEV; AMD MI355X brief; Helping Secure GPUs That Advance AI; overhead ranges: QSCompute.
What this tells a buyer:
- Blackwell HGX is today's sweet spot. Encrypted NVLink is the step change: a large model can finally span 8 GPUs without plaintext on the interconnect, at close to native speed. The privacy tax is close to gone.
- Hopper still works, with limits. It suits single-GPU models. Multi-GPU Hopper cannot pool memory confidentially, so large models either shrink or stay out.
- Rack scale is here, but the trust surface explodes. One NVL72 rack means verifying dozens of CPUs, 72 GPUs, the switches and their firmware against reference values that the operator controls. Across racks, InfiniBand and Ethernet sit outside NVIDIA's protected domain, so cross-rack traffic needs its own attestation and encryption.
- No two platforms attest the same way. TDX, SEV-SNP, Arm CCA and NVIDIA GPU evidence each have their own formats, services and reference values. A neocloud running three generations runs three trust stacks.
The hard part is trust, not silicon
A confidential GPU proves nothing on its own. It emits evidence. Someone has to decide whether that evidence is good enough to hand over a model worth hundreds of millions of dollars. That decision is where most confidential inference projects stall.
Each of these is solvable. Solving all of them, across TDX, SEV-SNP, Arm CCA and three GPU generations, is a real engineering program. Most neoclouds and model vendors don't want to build it. They want to ship models.
How Anjuna makes it work: prove, don't assert
Anjuna has spent years running unmodified, production workloads inside CPU and GPU TEEs for Fortune 25 financial institutions and the US Department of Defense. Anjuna Confidential Inference packages that experience into a neutral trust layer between the model owner and the infrastructure.
- The neutral verifier. Anjuna sits between model owner and operator, so the claim "the operator can't see your weights" is checked by someone with no stake in the answer.
- Encrypt, measure, package. The model is encrypted and measured before it ships. The owner's policy travels with it.
- Attest, then release. Anjuna verifies CPU and GPU evidence together across Intel TDX, AMD SEV-SNP and NVIDIA confidential GPUs, then releases the key from the owner's own KMS or HSM, straight into the TEE.
- Enforcement at every heartbeat. License term, instance count, region and usage are checked continuously. Breach a term and the key is revoked.
- Govern what the model touches. Attested configurations control which RAG stores, tools and clients a model can connect to. Paired with Anjuna Overwatch, the same TEE boundary protects the governance layer for the agents that call the model.
- One console, every fleet. Build, attest, deploy and monitor across neoclouds, hyperscalers and on-prem from one place, with full visibility across GPU fleets.
What each party gets:
The window is now
Attackers move in hours. Patching takes weeks. The newest GPUs now protect models in use at close to native speed. The only missing piece is the trust that someone other than the operator can verify.
If you run models as a neocloud and customer acquisition is limited due to concern over weight theft or tampering, or your customers are demanding higher levels of security and privacy, get in touch. If you build models, ship them anywhere without giving them away: talk to us about a confidential inference strategy. The time is now.
Contact us at anjuna.io/demo
Try free for 30 days on AWS, Azure or Google Cloud, and experience the power of intrinsic cloud security.
Start Free



