There is a specific moment in every enterprise AI conversation when the room goes quiet. Someone from security asks: "So where does the data actually go?" And the answer β "it gets sent to a model provider's API" β is the moment the project often dies.
It dies for a good reason. The workflows worth automating are the ones that touch your most sensitive data: pricing and margins, patient records, unreleased designs, customer PII, deal terms. That is not a coincidence. The data that is valuable enough to automate around is the same data you are least willing to hand to a third party. The CISO saying no is not being difficult. They are doing their job.
The real question is not "cloud or not." It is "where is the AI allowed to think?"
Teams frame this as a binary β cloud good, on-prem old-fashioned β and lose the plot. The useful question is narrower: for this specific workflow, given this specific data, where is the processing allowed to happen? That is a topology decision, and it has three honest answers, not one.
- Cloud (managed): fastest to stand up, lowest operational burden, best when the data is low-sensitivity or already lives in that cloud. A public marketing FAQ bot does not need an air gap.
- VPC / private cloud: the model and the orchestration run inside your own cloud boundary. Data does not traverse the public internet to a shared endpoint. This is the right default for most regulated enterprises β you get modern models without your data leaving your perimeter.
- On-prem / self-hosted: the whole stack runs on infrastructure you control, sometimes fully air-gapped. Slower and more expensive to operate, and absolutely correct for defense work, certain health data, and IP you would sue to protect.
Sensitivity is a property of the data, not the department
The instinct is to classify by team β "finance is sensitive, marketing is not." That is too coarse. A single workflow often mixes tiers: a quote contains a public product name (low), a negotiated margin (high), and a customer contact (regulated). Route the whole thing to the strictest tier and you over-engineer everything. Route it to the loosest and you leak.
The better unit of analysis is the field, not the workflow. Mask or tokenize the sensitive fields before anything leaves your boundary; let the model reason over the rest. PII masking is not a compliance checkbox β it is the mechanism that lets a high-value workflow run on a lower-cost, faster deployment without exposing the parts that actually matter.
Why "just send it to the API" is more expensive than it looks
The convenient path has a cost that does not show up until later. The average cost of a data breach reached 4.88 million dollars in 2024, and the breaches that hurt most are the ones involving exactly the data you would feed an AI workflow β customer records, IP, financials. When you weigh a managed API against a VPC deployment, the comparison is not "cheap versus expensive." It is "a slightly higher operating cost now" versus "a tail risk measured in millions and headlines."
Framed that way, the CISO's no is not friction. It is unpriced risk management, and the fix is to give them a topology where the answer can be yes.
Deployment flexibility is not a feature. It is the precondition for touching real data.
Here is the design principle we build around: the same workflow β same connectors, same guardrails, same approval logic β should be deployable to cloud, VPC, or on-prem without a rewrite. Because the sensitivity of the data, and therefore the required topology, is a business input that changes by workflow and by customer. A platform that forces one deployment model forces you to either over-secure the trivial or under-secure the critical.
Decouple where the AI thinks from what the AI does, and the CISO stops being the person who kills projects and becomes the person who tells you which topology each one needs. That is the conversation that gets your most valuable data β the data you were most afraid to touch β safely into production.
References
- IBM Security, Cost of a Data Breach Report 2024, IBM Corporation, 2024.
Want to map which of your workflows belong on-prem, in a VPC, or in the cloud? Book a 30-minute technical call.