Jordan: You should not self-host an LLM for your enterprise clients.
Jordan: I know. That sounds wrong. An enterprise prospect sends over a security questionnaire, and somewhere on page three there's a line about on-premises deployment or data sovereignty or "no third-party processing of customer data" — and your brain immediately jumps to, okay, I need to run this model myself. Local inference. My hardware, my network, my problem.
Jordan: And then you spend three weeks setting up an inference server you don't know how to maintain, burning GPU hours that eat your entire margin on the project, troubleshooting CUDA driver conflicts at one AM — all because you read "on-prem" in a procurement doc and assumed that meant you had to become a DevOps team.
Jordan: You don't. And I can prove it — because every major provider now publishes the exact controls that procurement is actually asking for. Region pinning. Zero data retention. Per-tenant key isolation. Auditable logging. A kill switch the client can see.
Jordan: The question was never "self-hosted LLM versus managed." The question was always "can you prove where the data goes and who controls it?" And the answer, for ninety-plus percent of enterprise asks, is yes — without hosting a single model yourself.
Jordan: Right now — mid-year review season — enterprise buyers are tightening their AI data-handling requirements. And if you're a solo operator fielding those security questionnaires without a defensible architecture, you are losing deals you could close this month. Not because your work isn't good enough. Because you can't prove it's safe enough — fast enough. I'm Jordan. This is Headcount Zero. Today I'm walking you through the four controls that replace self-hosting for almost every enterprise ask: BYO API keys per tenant, region-pinned endpoints, strict structured logging, and a one-click kill switch your client can see in their portal. We'll hit the exact provider settings across Azure, OpenAI, Vertex AI, and Anthropic, and by the end you'll have an architecture you can document and ship to procurement the same day.
Jordan: So here's what actually happens when an enterprise prospect says "on-prem." I'll tell you exactly how this played out for me last fall. I had a fintech client — mid-size, about two hundred employees, real compliance team. They sent over a twelve-page security questionnaire. And buried in section four, there's this clause: "All AI model inference must occur within the continental United States. No customer data may be used for model training. The vendor must demonstrate the ability to immediately cease all AI processing upon request."
Jordan: And I read that and my first instinct — my gut reaction — was, okay, I need to self-host. I need to spin up a VM, download an open-weight model, figure out quantization, set up an API layer, handle scaling, monitoring, patching... the whole stack. I actually started pricing GPU instances that afternoon.
Jordan: Then I stopped. Because I re-read the clause. They didn't say "self-host." They said three specific things. Process in the US. Don't train on our data. Give us a kill switch. That's it. Those are the actual requirements. And every single one of them is a checkbox you can satisfy with managed API settings that already exist.
Jordan: The problem is that "on-prem" has become shorthand. Procurement teams use it as a catch-all for "we need control and visibility." But what they're really asking for — almost every time — is proof of four things. Where does the data go? How long is it stored? Who holds the keys? And can we shut it off?
Jordan: Let's take those one at a time.
Jordan: First — per-tenant BYO API keys. This is the foundation. Instead of routing all your clients' requests through your own API key, each tenant gets their own. Their key, their provider account, their billing, their audit trail. You store the encrypted key in a secrets manager — AWS Secrets Manager, Google Secret Manager, even a KMS-encrypted field in Airtable if you're keeping it simple — and your automation decrypts it at runtime, makes the call, and never persists the key in plaintext.
Jordan: Why does this matter so much? Because it answers the "who controls the data" question instantly. The client owns the provider relationship. They can see their own usage. They can rotate or revoke the key without calling you. And if they ever want to leave — or if something goes wrong — they cut the key and your access is gone. That's not a theoretical safety net. That's a real one.
Jordan: Rapid7 — the security company — published a pattern for exactly this. Multi-tenant admin keys with centralized lifecycle management. Rotate across all tenants from one place, reduce key sprawl, scope permissions per tenant. It's the same architecture, just applied to LLM inference instead of security tooling.
Jordan: And here's the part that changes your sales conversation. When you tell a prospect "you hold the keys, not me" — that sentence does more work than a ten-page security document. It flips the trust model. You're not asking them to trust your infrastructure. You're asking them to trust theirs.
Jordan: Second control — region-pinned endpoints. This is where the provider landscape has gotten dramatically better in the last twelve months, and most solos haven't caught up.
Jordan: Azure Direct Models — that's Microsoft's current name for the OpenAI-family models running inside Azure Foundry — process prompts and completions within the customer-specified geography. Not "usually." Not "best effort." Within the geography. And critically, Azure Direct Models do not interact with OpenAI's own services at all. Your client's data stays in Microsoft's Azure environment. Full stop. If your client needs EU processing, you deploy in West Europe. Done.
Jordan: OpenAI's own API now supports project-level data residency. You pin a project to US, EU, Australia, Canada, Japan, or India, and you use the regional domain prefix — so for EU that's eu dot api dot openai dot com. Requests are processed and stored at rest in that region.
Jordan: Anthropic added a per-request parameter called inference underscore geo. You set it to "us" on the API call, and the response comes back with a usage field that confirms where inference actually ran. That's not just a setting — it's a receipt. You can log it and hand it to an auditor.
Jordan: And Vertex AI on Google Cloud — data at rest stays in the location you selected. ML processing runs in the specific region where you make the request. You set the location parameter — us-central-one, europe-west-four, whatever the client needs — and that's where it runs.
Jordan: So when procurement asks "where does our data go?" — you don't say "we think it stays in the US." You say "here's the region setting, here's the provider documentation, and here's the response header that proves it."
Jordan: Third — and this is the one that trips people up — retention and abuse monitoring controls. Because "they don't train on your data" is not the same as "they don't store your data." Those are two different questions, and you need to answer both.
Jordan: Every major provider has stopped using API data for model training by default. OpenAI made that change back in March twenty twenty-three. But — and this is the critical distinction — abuse monitoring logs can still retain your inputs and outputs for up to thirty days. That's the default on OpenAI. Anthropic's default is also thirty days for commercial API orgs.
Jordan: So if your client's security questionnaire says "zero retention," the default API settings don't satisfy that. You need to go one step further.
Jordan: OpenAI offers two tiers. Modified Abuse Monitoring — MAM — reduces what's stored. Zero Data Retention — ZDR — excludes your content from abuse monitoring logs entirely. You apply for it at the org or project level. And here's the detail that matters: not every endpoint is ZDR-eligible. Chat completions, embeddings, audio, realtime — those qualify. But Assistants, Threads, and Vector Stores retain application state by design. If your workflow uses the Assistants API, ZDR does not cover it. You need to know that before you fill out the questionnaire.
Jordan: Anthropic's ZDR covers the Messages API and token counting. Console and Workbench are not eligible. So if you're prototyping in the Anthropic console and then telling procurement you have zero retention — that's only true for your production API calls, not your dev environment.
Jordan: Vertex AI is a two-step process. First, disable project-level in-memory caching — it has a twenty-four-hour TTL by default. Second, get an abuse monitoring exception. But watch out for Grounding with Google Search or Maps — those features store prompts and context for thirty days and you cannot disable it. If your workflow uses grounded search, that path is not ZDR-eligible. Carve it out or don't use it for that client.
Jordan: Azure is the cleanest here. The models are stateless. Prompts and responses are not persisted. If you get approved to disable abuse monitoring, a ContentLogging capability flag flips to false — and you can verify that in the Azure Portal or through the CLI. Screenshot it. Put it in the evidence pack.
Jordan: The pattern across all four providers is the same. Default settings are close but not quite enough for strict security reviews. You need to apply for the enhanced controls, verify which endpoints qualify, and document the exceptions. Roughly twenty minutes of configuration per provider, and then you have citable proof.
Jordan: Fourth — structured logging and the kill switch. This is where you turn invisible compliance work into something a client can actually see.
Jordan: Every LLM call in your stack should write a structured log entry. Not "it succeeded." A real entry. Timestamp. A correlation ID — a UUID version four generated at the trigger, propagated through every step. Tenant ID. Provider. Model. Region. The provider's own request ID. Token counts. Latency in milliseconds. Billing in cents. And then — instead of logging the raw input and output, which would defeat the purpose — you log a SHA-256 hash of the redacted text. Plus a field for what redactions were applied.
Jordan: That's OWASP's recommendation, by the way — correlation IDs and structured security-relevant logging with careful redaction of secrets and PII. It's not something I invented. It's established best practice.
Jordan: You set a retention window that matches the client's requirement — thirty days, sixty, ninety, whatever the contract says — and you auto-export immutable daily logs to a client-owned storage bucket. S3, Google Cloud Storage, Azure Blob — their bucket, their object lock. They can pull the logs anytime without asking you.
Jordan: And then the kill switch. This is the piece that makes procurement people visibly relax. You gate every model call behind a boolean feature flag — "AI Integration Enabled." LaunchDarkly documents this exact pattern. One toggle. When it's off, all model calls short-circuit instantly. No queuing, no graceful degradation, no "we'll process the remaining batch." Off means off.
Jordan: And — okay, I'll be honest, this is the part I underestimated. I thought the kill switch was a nice-to-have. A checkbox for the security review. But when I built it into a Softr portal for that fintech client — a visible red button, right there in their dashboard, next to the status chips showing provider, region, ZDR status, last key rotation date — the procurement lead actually said, "This is the first vendor who's given us a way to shut it down ourselves." They signed the contract that week.
Jordan: That's the client proof problem solved in a different way. You're not just telling them it's safe. You're handing them the controls.
Jordan: Now — I said ninety-plus percent of enterprise asks. Not a hundred. And I want to be honest about the exceptions, because pretending they don't exist would undermine everything I just told you.
Jordan: There are cases where self-hosting is the right call. Three specific ones.
Jordan: First — a strict air-gap requirement. Not "we prefer on-prem." A contractual clause that says no data leaves our network, period. Defense contractors. Certain healthcare environments. Intelligence work. If the data literally cannot touch the public internet, managed APIs are off the table regardless of their controls.
Jordan: Second — the geography you need isn't supported. OpenAI covers six regions right now. Anthropic pins to US or global. If your client requires processing in a country none of the providers serve, you're stuck.
Jordan: Third — the workload depends on a feature with unavoidable storage. If you need OpenAI's Assistants API with persistent threads, or Vertex's grounded search with Maps data, and the client requires zero retention on those specific features — the provider can't give you that today. You either redesign the workflow to avoid those features, or you self-host that narrow path.
Jordan: But notice what I said. That narrow path. You don't self-host your entire stack because one feature in one workflow can't meet ZDR. You self-host the exception and keep everything else on managed endpoints where you get reliability, automatic scaling, and model updates without touching a GPU driver.
Jordan: The decision box is simple. Can the provider's published controls satisfy the specific clause in the contract? If yes, document it and move on. If no, identify exactly which feature or geography is the blocker, scope the self-hosted piece to that and only that, and keep the rest managed.
Jordan: So let me bring this back to where we started. You get a security questionnaire. Page three says something about on-prem or data sovereignty. And instead of spending three weeks becoming an infrastructure team you never wanted to be — you open your architecture doc. BYO keys per tenant, stored encrypted, revocable by the client. Region-pinned endpoints with provider documentation and response headers to prove it. Zero data retention applied and verified, with every exception carved out and documented. Structured logs with correlation IDs exporting to the client's own bucket. And a red button in their portal that shuts everything down instantly.
Jordan: That's not a workaround. That's a better answer than self-hosting. Because self-hosting gives the client a server they can't see inside. This gives them controls they can actually use.
Jordan: We put the exact checklist I send to procurement into a one-pager — it's on the Resources page. Copy it, fill in your provider settings and screenshots, and ship it to procurement the same day you get the ask. It covers every control we talked about today plus a decision box for when self-hosting actually is justified.
Jordan: Here's what I want you to do this week. Pick one client — the one most likely to send you a security questionnaire next — and set up their BYO key. Just the key. Encrypted storage, scoped to their provider project, revocable without a code change. That takes roughly fifteen minutes. And once that key exists, the rest of this architecture has a foundation to build on.
Jordan: I'm Jordan. This is Headcount Zero. Proven builds you can ship solo. I'll see you Wednesday.