Cohere Adds Confidential Computing to Model Vault

Cohere has introduced a limited beta called Model Vault Encrypted, a single-tenant hosting service designed to secure data during active processing. While traditional encryption protects data in transit and at rest, this new mode secures prompts, responses, model weights, and intermediate activations while they are in memory. The system isolates workloads inside hardware-backed trusted execution environments that span both CPUs and GPUs.

To achieve this, the service utilizes CPUs featuring Intel TDX or AMD SEV-SNP alongside confidential-computing-capable NVIDIA GPUs. Security Protocol and Data Model sessions authenticate and encrypt the connection between the CPU and GPU. Before any sensitive data leaves a client's environment, a local Oblivious HTTP proxy performs remote attestation. This process validates the hardware against manufacturer-backed trust chains, checks that debug features are disabled, and verifies firmware measurements.

For developers and enterprise practitioners, this setup removes cloud providers, hypervisor administrators, and Cohere operators from the trusted computing base. Integration is designed to be seamless, as existing Cohere SDK applications can route API calls through the proxy without modifying their core application code. Cohere offers this proxy as either a standalone deployment or a Python package.

However, the beta is currently restricted and gated through Cohere's sales team. It supports only a subset of models and requires specialized hardware. While it prevents infrastructure administrators from inspecting memory, it does not guard against metadata analysis, such as request timing and payload size, nor does it replace application-layer controls for prompt injection. The service is primarily aimed at highly regulated sectors like finance, healthcare, and the public sector.