Private AI: when your files should never leave the office

For most businesses, sending text to a reputable AI company under a business contract is a reasonable trade. For some firms it isn’t. Their files are covered by professional rules, health privacy law or promises written into client contracts, and “we use a well-known vendor” is not a good enough answer.

Private AI means running the AI on a server you own, in your office or your private cloud. Documents are read, summarized and searched there, and nothing is sent to an outside AI company. Here is when that makes sense, when it doesn’t, and what you give up.

The cloud is fine for a lot of work

We build most systems on the business APIs from Anthropic, OpenAI and xAI. Under their commercial terms, your data isn’t used to train their models by default. Some providers offer stricter options on top. Anthropic, for example, offers zero data retention and a business associate agreement for HIPAA-configured use of its API (Anthropic).

If your AI answers questions about opening hours, service areas and booking, none of that is secret. A cloud model is usually the better choice there. The largest models are more capable, there is no hardware to look after, and the cost follows how much you use it.

Why some firms still say no

Privacy is the concern small business owners raise most about AI. In the Goldman Sachs 10,000 Small Businesses survey of 1,256 owners in early 2026, 50% named data privacy and security as a challenge, the most common answer (Goldman Sachs). For some professions, that concern comes with rules attached.

Law firms

ABA Formal Opinion 512 says lawyers using generative AI must protect client confidentiality. That includes getting informed client consent before putting client information into AI tools that learn from what they’re given (ABA Formal Opinion 512). Texas issued similar guidance for its lawyers in Opinion 705.

Healthcare

Under HIPAA, every outside company that handles patient information for you is a business associate and needs a signed agreement (HHS). Texas law also defines who must protect health information more broadly than HIPAA does (Tex. Health and Safety Code ch. 181). Every cloud service you add is another agreement to sign and another company to trust.

Accounting and tax

Federal law restricts using or disclosing tax return information without the client’s consent, and the IRS expects paid preparers to keep a written information security plan (IRS Publication 4557). Tax preparers also fall under the FTC Safeguards Rule, which requires them to oversee their service providers (FTC).

Finance and client promises

Many financial advisers and finance teams have promised clients, in writing, that records stay in house. A private server makes that promise easier to keep and easier to explain.

There is also a newer question about what a vendor could do with your data. In a California case over Google’s contact-center AI, a court let the claims proceed based on Google’s ability to use call data, not proof that it did (Goodwin). That case was about phone calls, but the idea travels. When no outside company holds your files, that question doesn’t come up.

These are also the sectors already using AI most. Census Bureau data puts finance and insurance at 33.9% current AI use, among the highest of any sector (Census Bureau). A summary of late-2025 Census data puts professional, scientific and technical services, which includes law and accounting firms, at 33.0% (University of Missouri Extension).

What private AI is, in practice

A private AI server runs an open model, meaning one whose weights are published so anyone can run it on their own hardware. Llama from Meta, Mistral, and Qwen from Alibaba are three well-known families. Each comes in several sizes, from small models that run on one workstation to large ones that need far more hardware.

We install the server in your office or your private cloud, limit access to your network, and connect the folders and systems you choose. It summarizes contracts, case files, records and statements. It searches across them, compares versions, and drafts first versions of letters and memos for a person to finish. Each answer shows the document it came from.

The honest trade-offs

It is less capable than the biggest cloud models

The models that fit on an office server are smaller than the largest models the cloud providers run. Whether a given open model is good enough for your work is something to test, not assume.

Small models can slip on details. In an independent benchmark of AI phone systems, a setup built on a small Qwen model garbled a confirmation code in 7 of 7 attempts (Codebridge). That was a phone test, not document work. The lesson still holds: test any model on your real documents before your team relies on it.

You pay up front instead of per use

Cloud AI is billed by usage. A private server is hardware you buy, plus upkeep. Heavy batch jobs, like reading a box of scanned files or tagging years of records, don’t run up a per-use bill. Light, occasional use may cost less in the cloud. Which way it falls depends on how much your team will use it.

Someone has to look after it

A server needs updates, monitoring, backups and model upgrades. We maintain it every month, and you own the hardware and the data. That also means a power outage or a failed drive in your office is your event, not a cloud provider’s.

Private is not the same as secure

Keeping files in the building removes one risk. It doesn’t remove the others. The server still needs access controls so each person sees only what they already have access to, a locked-down network, and a written record of who can do what.

Who it suits

• Law firms with client files, discovery and contracts that can’t go to outside companies.
• Clinics and healthcare practices working with patient records.
• Accounting and tax firms handling returns and financial statements.
• Financial advisers and finance teams with confidentiality promises in their client agreements.
• Any business that has told clients, in writing, that their records stay in house.

It suits a business less when the work involves nothing sensitive. You can also mix the two. Use a cloud AI receptionist with the right agreements in place for the phones, and keep the document work on a private server.

Questions to answer before you decide

1. Which documents would the AI need to read, and which rules or contracts cover them?
2. Could a cloud provider do this under a signed agreement, or is that ruled out?
3. How many people will use it, and how often?
4. Who in your office is the contact when the server needs attention?
5. Which real tasks will you test it on before anyone relies on it?

Next step

See what a private AI server does and how we set it up on our private AI page. Or start the chat and tell us what your team needs to read.