Securing LLM Inference in The Cloud
Enterprises are eager to put generative AI to work, but most are blocked by a single question: Wouldn’t it be great if we could use our sensitive data with GPU compute managed in the cloud to run the model we want to use?
From customer records to health data and proprietary documents, organizations need a secure way to process data during inference. For many enterprise AI teams, isolating sensitive AI workloads has become the default approach to preserving privacy. However, this not only limits data utility and scalability, but drives up operational costs in exchange for tighter data control.
That’s why we’re thrilled to announce that Protopia AI’s Stained Glass Transform (SGT) is now available in AWS Marketplace and Amazon SageMaker, enabling roundtrip, private inference in AWS managed services. SGT adds a drop-in privacy layer for open-weight models, enabling safe use of sensitive data in AI workflows without retraining or infrastructure changes.
Streamlined & Secure AI Deployment on AWS
AWS users can now deploy Stained Glass Transform directly within their AWS Cloud environments. The release supports two flexible deployment paths to meet enterprise needs:
1. SGT-enabled SageMaker endpoints + client-side Stained Glass Proxy
Set up an SGT-enabled endpoint in Amazon SageMaker and deploy Stained Glass Proxy within your enterprise zone of trust. This configuration transforms all sensitive inputs into unreadable, irreversible stochastic representations before reaching the Sagemaker-managed endpoint. The initial release supports SGT-enabled Llama 3.1 8B in SageMaker, with extension to Amazon Bedrock and support for larger models, including Llama 3.3 70B, coming soon.. This option preserves security by keeping raw data local while eliminating model management overhead and accelerating time-to-value.
2. Fully containerized deployment in AWS
Deploy the model inference server within the AWS environment using the provided Helm chart and deploy SGT within the enterprise root of trust. This option gives full control over the entire pipeline with the inference server running in a dedicated VPC, making it well suited for teams with high customization needs.
Both deployment options support the use of AWS credits and work seamlessly with other AWS-native services like EKS, S3, and IAM as well as AWS’s built-in procurement, billing, and governance tools. SGT integrates easily with AWS ecosystem partners like NetApp, extending data protection across hybrid architectures and making secure GenAI adoption is both scalable and practical for cloud-native teams.
Enterprise AI Adoption Hinges on Data Privacy
Secure GenAI adoption is not just a security issue; it is a business imperative with real-world cost envelopes that determine adoption of which use-cases will fall above the ROI bar and which never make it past a POC. LLMs promise breakthroughs in developer productivity, RAG, document processing, and customer service, but adoption often stalls in sectors with sensitive or proprietary data. LLMs fundamentally cannot run on encrypted data, which creates a critical security gap during inference. Traditional techniques like masking or redacting identifiers fall short, and encryption alone cannot protect information once it reaches the model, where sensitive content is exposed in plaintext, not only in memory during compute, but often on persistent storage as a result of logging, memory spills, or other implementation details
Protopia AI’s Stained Glass Transform (SGT) solves this plain-text exposure at its core by converting plain-text inputs into stochastic representations before inference, safeguarding data without sacrificing utility. SGT on AWS offers multiple flexible deployment paths designed for speed, simplicity, and enterprise scale without rearchitecting your stack or compromising performance.
Enterprise AWS customers can realize immediate benefits:
- Lower total cost of inference by running secure workloads on infrastructure shared among different data owners concurrently, driving higher volumes of inference. No need for isolated GPUs for every data silo.
- Accelerate time-to-impact by using sensitive data safely in RAG and document workflows, without losing valuable information to token filtering or format-specific preprocessing.
- Unblock production use cases previously stalled due to regulatory considerations, customer privacy, and data leakage.
- Future-proof AI investments with a flexible privacy layer compatible with open-weight models and AWS-managed endpoints.
By joining AWS’s extensive partner ecosystem, Protopia AI strengthens its commitment to setting a new standard for AI that never exposes data in plaintext.
Getting Started is Easy
Protopia Stained Glass solution is available through AWS Marketplace:
- SGT-enabled Llama 3.1 8B model endpoint on Amazon SageMaker.
- Stained Glass Proxy via AWS Marketplace for client-side deployment. Proxy transforms data before it leaves your environment, and routes the transformed data to your SageMaker endpoint.
- Stained Glass Transform via AWS Marketplace to run the whole pipeline in your VPC.
With just a few clicks, you can use AWS credits toward secure, production-grade deployments. No custom integration or model retraining required.
