The Architecture of AI Multi-Tenancy: Building Secure Backend Infrastructure
Introduction
When scaling a Software-as-a-Service (SaaS) business, the backend infrastructure is the invisible skeleton holding up the entire operation. As AI continues to permeate every aspect of enterprise software, a new challenge has emerged: How do you build a multi-tenant environment that securely isolates data while running local LLM inference?
At Toi 'n' Moi®, we've transitioned from handling typical e-commerce workloads to hosting heavy AI services for Legal Tech (Citizen Advocate) and Hollywood Scripting (Cinescript). This is the architecture we use to handle it.
1. The Multi-Tenancy Challenge in AI
Traditional multi-tenancy involves isolating rows in a Postgres database or separating buckets in S3. However, when you add Large Language Models (LLMs) to the mix, you face "context bleeding"—the risk of one tenant's proprietary data accidentally influencing the output generated for another tenant.
To solve this, our backend infrastructure at app.toinmoi.com employs:
- Strict JWT Isolation: Every internal request carries a cryptographic tenant ID.
- Stateless Inference Nodes: The LLMs hold zero memory between API calls, dumping context immediately after generation.
2. Moving Away from OpenAI: Why Local Inference Wins
B2B clients, especially in healthcare (Rota Management) and law (Citizen Advocate), have strict limits on where personal data may be processed. Sending PII (Personally Identifiable Information) or confidential case law to an external API is, for many of our users, not something their own obligations permit.
The Solution: On-Box AI
Hosting models on our own GPU instances removes the third-party API from the data path. That is a meaningful reduction in exposure, not a guarantee of privacy on its own — access controls, logging, backups, OS hardening and model configuration all still apply. Our infrastructure utilizes highly optimized models designed for specific tasks:
- Drafting Scripts: Cinescript uses models fine-tuned to understand formatting and creative structure.
- Parsing Case Law: Citizen Advocate uses Retrieval-Augmented Generation (RAG) strictly bounded to UK legislation databases.
3. Stripe Integration for Advanced Usage Metering
You can't just bill flat-rate for AI. Token generation is expensive. Our backend integrates directly with Stripe to meter usage seamlessly.
- The user requests a generation.
- The Toi 'n' Moi® API gateway calculates token expenditure.
- Stripe increments the usage via the
v1/subscription_items/usage_recordsendpoint in real-time.
Conclusion
Building a multi-tenant AI back-end isn't just about spinning up a database and a server; it's about redefining security patterns for token-based workloads. This exact architecture is what powers the entire Toi 'n' Moi® ecosystem today.
CTA: Looking to leverage secure AI infrastructure for your next big idea? Partner with the Toi 'n' Moi® technical team. Contact us today.
Related Articles
The 48-Hour Week in Care: What a Rota Has to Be Able to Show
The 48-hour limit is an average, the opt-out has to be in writing, and the records duty behind it is separate from the limit itself. Most rotas can show the hours and not the thing the regulations actually ask about.
InsightsWhy Local LLM Inference is the Future of Enterprise Data Privacy
When ChatGPT launched, the enterprise world sprinted to integrate the OpenAI API. It was fast, easy, and powerful. But a year later, Chief Information...
InsightsWhy an Independent Security & AI Audit Beats a Checklist
A checklist records the boxes you ticked; an audit tests what is actually true. Why an independent review against OWASP, CIS and NIST beats a checklist — and a premature pen test.