Skip to content
New AI Custom Form BuilderDescribe the intake form. It ships branded and end-to-end encrypted.AI Custom Form Builder is here See it build
Security

Your Customers' Sensitive PII Shouldn't Be Training Someone Else's AI

Brian Holyfield · 7 min read
Your Customers' Sensitive PII Shouldn't Be Training Someone Else's AI

When most people talk about Sovereign AI today, they mean running a local large language model on your own hardware — keeping AI computation off the cloud and away from the hyperscalers. It solves a very real concern and valid set of data privacy and security problems.

There’s also a version of this problem that gets almost no attention: What happens to the sensitive data your customers give you, once it flows through the dozen-odd SaaS platforms your business runs on?

The answer, increasingly, is that it ends up training someone else’s AI. And your customers have no idea it’s happening.

Sovereign AI isn’t just about which model you run. It’s about who gets to learn from your customers’ most sensitive information.

The Quiet Data Grab Hiding in Your SaaS Stack

Every major SaaS vendor is building AI. That’s not a secret. What is less visible to the average person whom the data belongs to is where the training data comes from.

Salesforce, for example, has been explicit that its Agentforce and Einstein AI features improve from the activity and data flowing through its platform. This is especially relevant with the major new release of Salesforce Headless 360. As “everything on Salesforce is now an API, MCP tool, or CLI command, and agents can use all of it” even more customer data will flow through Salesforce AI.

Many vendors have similar provisions buried in their terms of service — broad rights to use customer data to improve their products. This is not a theoretical risk or fine-print edge case. It is standard industry practice. Atlassian and Slack have both drawn scrutiny for defaulting organizations into AI training programs, quietly assuming consent unless someone actively opts out. Most organizations never did — not because they agreed, but because nobody noticed the checkbox had already been ticked on their behalf. The customers whose data was used had no idea any of this was happening.

In practice, that data often includes the content of records, tickets, documents, and files that your customers gave to you in confidence. All of this was best summarized in a series of prescient posts on X by user @kepano way back in 2023:

Kepano X 1

Kepano X 2

Think about what customer data moves through your support desk, your CRM, or your onboarding workflows on any given day:

Now think about every vendor platform that touches your customer support and customer success workflow: your CRM, your helpdesk, your live chat tool, your AI chatbot, your onboarding platform, your document management system. Each one has its own AI ambitions. Each one has its own data retention policies. Each one has terms of service that grant them rights you probably haven’t read carefully. That sensitive file your customer sent you didn’t just land in one system — it passed through an entire ecosystem of vendors, and every single one of them has something to gain from the patterns inside it.

Your customer submitted a passport scan to you. By the time it’s sitting in a ticket, it may have already touched your helpdesk, your CRM, your AI bot, and your cloud storage provider — each with their own AI training agenda.

Your customer sent that passport scan to your company — not to Salesforce. Not to your ticketing vendor. Not to whatever AI startup your helpdesk platform recently acquired.

The same is true for a HAR file a support agent asked a customer to export for troubleshooting — that file can contain live session tokens, authentication cookies, and full request/response headers. It was handed over to fix a bug, not to enrich a vendor’s training dataset. Yet if those files flow through those platforms unprotected, that’s exactly where they end up: inside data pipelines your customers never consented to, feeding models you’ll never audit.

Redefining Sovereign AI: It’s About Your Data, Not Just Your Model

The traditional framing of Sovereign AI focuses on model ownership and deployment location. Run your own LLM. Keep inference on-premise. Don’t send prompts to the big AI companies.

That matters. But it misses the upstream problem.

A more complete definition of Sovereign AI looks like this: Your organization processes sensitive customer data using AI — and at no point in that pipeline does any third-party vendor gain access to it, train on it, or benefit from it.

That means two things working together:

Most organizations are working hard on the first part. Almost none have addressed the second.

Where SendSafely Fits In

SendSafely is end-to-end encrypted file exchange. Files are encrypted on the sender’s device before they ever leave it. No one in the middle — not your support platform, not your CRM, not SendSafely itself — can read the contents.

Critically, SendSafely works inside the tools your teams already use. We have deep, production-ready integrations with Salesforce, Zendesk, Jira, and Freshdesk — as well as the AI-powered chat platforms increasingly sitting at the front of customer support workflows, including Intercom Fin and Ada. Wherever your customers are interacting with your team, SendSafely can be the secure layer that ensures the files they share never enter those platforms in readable form. There’s no workflow disruption, no separate portal to manage, and no asking customers to do something unfamiliar. The encryption happens invisibly, before the file ever leaves their device.

Also importantly for this conversation: the encrypted files sent/received through SendSafely can be routed to your own Amazon S3 cloud storage rather than stored inside your vendor platforms. A sensitive document your customer submits through a SendSafely-powered Zendesk ticket or Salesforce case never actually lives in Zendesk or Salesforce. It lives in your storage, encrypted, under your control.

That means:

1. Vendor AI can’t train on what it can’t see.

If the file isn’t in Salesforce, Salesforce’s AI doesn’t have access to it. The data never entered the pipeline.

2. Your sovereign AI can still access it.

When your AI workflow legitimately needs to process that document — for fraud detection, automated underwriting, compliance review — you can grant your own controlled AI environment access to the encrypted file. You decide when, and under what conditions. You’re not locked out of AI benefits. You’re just keeping that benefit inside your own walls.

3. Your customers’ expectations are honored.

When someone submits a passport scan to verify their identity, they expect that document to be used for that purpose — not to improve some vendor’s OCR model. SendSafely lets you make that promise and keep it.

4. Breach impact shrinks.

Data that doesn’t exist inside a platform can’t be exfiltrated from it. This includes all the SaaS platforms you knowingly utilize as part of your workflow - but also their AI providers along with all of their Subprocessers - which are often buried deep in their Data Processing Agreements.

Questions Worth Asking Your Vendors

The next time you’re reviewing a SaaS contract or an AI feature rollout, it’s worth asking:

Most vendors will have answers. Not all of those answers will be reassuring.

The Bottom Line

Sovereign AI means more than running your own model. It means controlling the entire data path — from the moment a customer sends you something sensitive to the moment your AI makes use of it. SendSafely closes the gap that most organizations haven’t thought to look for: the unencrypted files sitting inside vendor platforms, quietly feeding AI systems your customers never agreed to.

Your customers trusted you with their data. Not your Support or CRM vendor. Not their AI team. You.

Contact us at sales@sendsafely.com to learn more or schedule a live demo.


SendSafely: Integrated File Transfer for the Apps you Love

If you are looking for a secure way to send or receive files with anyone, or simply need a better way to transfer large files, our platform might be right for you.

Request a Demo

Frequently asked questions

What is Sovereign AI?

Sovereign AI usually means running AI models in an environment you control, such as a local or private deployment. SendSafely proposes a broader definition: processing sensitive customer data with AI without any third-party vendor gaining access to it, training on it, or benefiting from it.

Can SaaS vendors use my customers' data to train their AI?

Many vendors reserve that right. Many SaaS terms of service grant broad rights to use customer data to improve products, and Atlassian and Slack drew scrutiny for defaulting organizations into AI training programs unless they opted out. Check each vendor’s terms directly.

How does SendSafely keep customer files out of vendor AI training?

SendSafely encrypts files on the sender’s device, so your helpdesk or CRM can’t read them. Encrypted files can also be routed to your own Amazon S3 storage, so a document submitted through a SendSafely-powered Zendesk ticket or Salesforce case never lives in Zendesk or Salesforce. See helpdesk security.

Can my own AI still process files protected by SendSafely?

Yes. When a workflow legitimately needs to process a document, for example for fraud detection, automated underwriting, or compliance review, you can grant your own controlled AI environment access to the encrypted file. You decide when and under what conditions that access happens.

What should I ask SaaS vendors about AI training on customer data?

Ask whether your data and uploaded files touch their AI training pipeline, whether data access can expire and data be deleted automatically, whether files can be stored in your own cloud storage, what rights they claim to aggregated or de-identified data, and what access their model providers have to your inputs and outputs.

Written by Brian Holyfield