Can a law firm use AI without sending client files to the cloud?

Most lawyers we talk to see the potential in AI. Summarizing depositions, reviewing contracts, searching years of matter files in seconds: the time savings are real. What stops them is a reasonable question. Where do our client files go when we use it?

With many popular AI tools, the answer is “to someone else’s servers.” For a law firm, that’s worth taking seriously.

Why public AI tools give firms pause

When you paste a document into a public AI tool, it’s sent to the provider’s servers to be processed. Depending on the product and your settings, it may be stored, reviewed by the provider’s staff or used to improve future models. Business versions usually come with stronger privacy terms, but the documents still leave your network.

That raises questions about confidentiality and privilege that clients, insurers and regulators are asking more often. The American Bar Association’s Formal Opinion 512, issued in 2024, says lawyers who use generative AI need to understand how a tool handles client information and take reasonable steps to protect it. Many corporate clients now include AI rules in their outside counsel guidelines as well.

There’s also the accuracy problem. In 2023, a federal judge in New York sanctioned lawyers who filed a brief citing cases an AI tool had made up. Any AI used for legal work needs a way to catch that kind of mistake before it reaches a filing.

The alternative: AI that runs inside your firm

A private language model, often called a local LLM, is an AI model that runs on hardware your firm controls: a server in your office, or a private cloud environment set up only for you.

Here’s what that means in practice:

Documents stay put. Files are processed on your own equipment. Nothing is sent to an outside AI provider.
No training on your files. The model doesn’t learn from your documents and pass that knowledge on to anyone else.
Your access rules still apply. The system can be set up so a lawyer only gets answers drawn from matters they’re already allowed to see.
A full record. Every question, and every document the AI used to answer it, can be logged. That helps if a client or regulator ever asks.

Open AI models have improved quickly in the last two years. For most of the everyday work firms want AI for, like summarizing, searching, first drafts and pulling out key terms, a well-chosen private model does the job well.

What a firm can do with it

Ask questions across matter files. “Find every agreement in this matter with an indemnification cap under $1 million” returns the documents, with the relevant passages linked to the source.
Summarize long documents. Deposition transcripts, discovery productions and lengthy contracts, summarized in minutes instead of hours.
Speed up intake. Draft intake summaries and flag possible conflicts for a person to review.
Draft first versions. Routine letters and standard clauses, based on your firm’s own templates and past work.

Catching AI mistakes before they matter

Private AI solves the confidentiality problem, but accuracy still matters. Two safeguards make a big difference:

Answers with sources. The system shows exactly which document and passage each answer came from, so a lawyer can check it in seconds.
A second model that checks the first. We often set up what we call a fusion adversarial system. One model drafts an answer, and a second model’s job is to challenge it, flagging any claim the documents don’t support. Made-up facts and citations get caught before anyone relies on them.

Neither replaces a lawyer’s review. Together, they make that review faster and more reliable.

What to weigh before you start

Cost. A private setup means buying or renting capable hardware, plus setup and ongoing upkeep. Whether that beats paying per seat for outside AI tools depends on your firm’s size and how heavily you’ll use it.
Start narrow. Pick one high-value task, such as searching closed matters or summarizing depositions. Prove it works, then expand.
Security around it. A private AI system is only as safe as the network it runs on. It should be tested like any other system that holds client data.
Your people. Lawyers need a short, clear policy on what the tool is for and how to check its work.

How we help

We build private AI systems that let law firms work with privileged documents safely, and we test the security around them.

See our work for the legal industry, learn about our custom AI systems, or talk to us about your firm.