top of page
Search

Acharya: The Zero-Trust Operating System For Secure Scaling Of Enterprise AI

10 minutes ago
2 min read

When deploying autonomous AI agents within your enterprise, you’ll often face what we call the Enterprise Agent Paradox. There are huge productivity benefits associated with AI but often come with major risks including data residency issues, high token costs on the cloud, prompt injection, and increased compliance requirements.


 To overcome this, we’ve created Acharya – the Zero-Trust Operating System for Autonomous Enterprise Agents. This is a Kubernetes native inline gateway solution which works between your enterprise and Large Language Models (LLM). The main purpose of the system is to make sure we have the tools to securely scale AI within highly regulated industries including banking, financial services, healthcare, insurance etc.



Acharya provides a Zero Trust solution for your data:


Uncompromised Data Privacy & DLP

If you’re managing sensitive customer data, sending raw PII data to frontier models will violate your data residency requirements and break zero trust. Our solution provides a clever method to swap data:

Dynamic PII Masking – We leverage Machine Learning and Regex to identify PII data. This data is then masked using structured tokens, whilst still providing privacy.

Data Reconstruction – Once the External LLM has processed our sanitised prompt, the original values are reconstructed in memory. The internal virtual agent then receives this unmasked data.

Multimodal Data – Our DLP solution goes further than text only. We use OCR to analyze attachments in order to detect sensitive data.


Efficient Resource Allocation

When running at scale, unpredictable token costs and unused on-premise GPUs can cause you to lose out on your ROI. Our system allows us to utilise 85% of target GPU usage.


Local Processing – We ensure requests are routed to local GPU’s to ensure we keep our costs down and keep data internal. We manage frameworks such as vLLM on H100 or A100 chips.

Smart Cloud Usage – We utilize KV-cache monitoring and Prometheus metrics to route any spare capacity to frontier models (such as OpenAI, Anthropic, Vertex, Azure) only when at peak and with masked data.

Asynchronous Calls – This ensures we make the most of our GPU’s at night by prioritizing requests based on type.



Enterprise Agent Management

To successfully move your autonomous agents into production, we need to provide some additional management. Our solution acts as a security layer.


Cost Management – Each agent has its own Virtual Key. This ensures we have a maximum cost per agent with RPM and TPM limits.


Semantic Prompt Firewall – This feature ensures we know what topics our prompts are discussing, and will alert us if they go off topic. We also have a prompt injection block.


Audit Trail – All decisions made, PII masked, and model interactions are logged within a routing trace ledger. We ensure complete compliance by keeping a record of everything.


By providing this functionality within an air gap solution, we add minimal latency (in seconds) and no code changes for developers. This means you can ensure complete compliance with stringent regulations (such as FREE-AI from RBI). This provides all the infrastructure needed to ensure AI becomes a secure asset within your organization.


 
 
 

Comments


bottom of page