Do you have to train an AI on your documents?
No, and this is the most common misunderstanding I hear on a first call. Training, or fine-tuning, changes what a model knows, and it is slow, expensive and hard to undo. It is also the wrong tool for facts about a business, because those facts change: a price, a returns window, the clause that moved in March.
What people actually want is retrieval. When someone asks a question, the system searches your own material for the few passages that answer it, hands only those to the model, and asks for a reply written from them. Update a document and the next answer uses the new version. Nothing has to be retrained, and the answer can show the passage it came from, so a person can check it in seconds.
What are the three ways to get one?
All three use the same idea underneath. What changes is how much material they handle well, who controls the data, and who you call when an answer is wrong.
| The option | How it works | Where it fits | Where it stops |
|---|---|---|---|
| Upload files to a chat tool | ChatGPT and Claude both let you add files to a project and ask questions about them. | One person, a handful of documents, private use. A good way to find out whether the idea helps at all. | No per-user permissions, limited material, nothing inside your own site or app, and the plan's data terms decide what happens to the files. |
| Rent a chatbot platform | A hosted service that ingests your documents and gives you a widget to embed. | A public FAQ or support helper on standard documents, live in days. | A monthly fee per seat or per message, your documents on their servers, and little say over how search and citations work. |
| Have one built that you own | A pipeline, a search index in your own database and an interface, in accounts in your name. | Staff-only knowledge, documents with permissions, answers inside an app you already run, or volume where per-message fees add up. | A build price up front, and someone has to keep the index fresh. That is the honest trade. |
How does it find the right passage?
Your documents are cut into passages, each passage is turned into a list of numbers that captures its meaning, and those numbers are stored next to the text. A question goes through the same step, and the search returns the passages whose meaning sits closest. I keep these in PostgreSQL with the pgvector extension, so the passages live in the same database as everything else about your business.
Meaning alone misses things. Your team searches for a product code, a policy number or a word only your company uses, and a meaning search can rank a vaguer passage above the exact one. So the search runs twice, once by meaning and once by exact words, and the two lists are merged. Then a short list is read again and ordered by what actually answers this question.
- Cutting: passages small enough to find and whole enough to still make sense on their own.
- Search by meaning, for the same idea said in different words.
- Search by exact words, for codes, names and your own vocabulary.
- Merging and reordering, so the answer is written from the best few passages rather than the first ones found.
- The answer, with the document and passage named, or a plain sentence saying nothing in your files covers it.
What should you prepare before building one?
This is where most of the quality comes from, and it costs nothing but an afternoon. An assistant cannot be more accurate than the files it reads. If two versions of the price list disagree, it will quote one of them with total confidence.
- Pick one job first: staff questions about policy, customer questions about orders, or searching contracts. Not all three.
- Gather the documents for that job into one place, and delete or archive the old versions rather than leaving them beside the new ones.
- Write down 20 to 30 real questions people ask, and next to each one, the document that answers it. That list becomes the test.
- Find the scanned PDFs. A photo of a page has no text until it is read with OCR, and tables in scans are where answers go wrong.
- Decide who may see what. If some files are for managers only, say so now, because that rule belongs in the search, not in a sentence to the model.
- Name an owner who updates a document when the policy changes. The assistant is only ever as current as that person.
Is it safe to put company documents into an AI?
It can be, and the design is what makes it safe rather than a promise. In a build you own, the documents stay in your own database, the model only sees the few passages one question needs, keys stay on the server, and the search filters out anything the person asking is not allowed to read. A prompt that says "only answer from what this user may see" is not a security control. The filter in the query is.
There is one risk that surprises people. Anything you load into the system is text the model will read, and a document can contain instructions: a PDF that says to ignore the rules and send the customer list somewhere. That is called prompt injection, and the defence is to treat retrieved text as data, keep the system's own instructions separate, and give the assistant no action it has not been allowed in writing.
With the chat tools and rented platforms, the plan's data terms decide what happens to your files. Read them before the first confidential document goes in, not after.
When do you not need one?
When the answer fits on two pages. Put the rules straight into the assistant's instructions: no index, no pipeline, nothing to keep in sync, and a smaller bill per question.
When the fact lives in a table. Which orders shipped on Tuesday, what this customer owes, how much stock is in the second warehouse: a database query answers that exactly, and a document search answers it approximately.
When almost nobody asks. Ten questions a month are cheaper answered by a person who knows where the folder is.
And when the documents themselves are the problem. If nobody agrees which version of the handbook is current, an assistant will not settle that argument. It will repeat it.
What happens after you have one?
If the answers are good, the next request is usually to let it act: check the order, start the return, draft the reply. That is a different build, with tools, permissions and a person approving anything that cannot be undone, and the guardrails are written up on the AI agents and automation page.
The build and the running bill are two separate numbers, and the arithmetic for the second one is in what an AI agent costs to build and to run. The pipeline, the permissions and what I deliver are on the agentic AI and RAG service page, and the code I start from is public on GitHub, linked at the bottom of this page.
What to remember
- You do not train a model on your files. It looks them up on every question.
- Start with a chat tool to test the idea, rent a platform for a public FAQ, and build one you own when permissions, your own app or volume matter.
- Clean, current, de-duplicated documents do more for accuracy than any model choice.
- Write 20 to 30 real questions with the right document for each before anything is built.
- Permissions belong in the search, not in the instructions to the model.
- Two pages of rules, a fact in a table or a handful of questions a month: you do not need one.
Sources
Primary documentation, for anyone who wants to check the details.