In this guide 6 sections
Most company assistants need RAG, which retrieves passages from your manuals, circulars and rate lists at the moment someone asks, so every answer reflects the current file. Fine tuning suits style, not facts. Use two tests: how often the content changes and whether every answer must cite its source document, such as a circular.
What each approach does
RAG stands for retrieval augmented generation. When someone asks a question, the system searches your documents, pulls the best matching passages and hands them to the model with the question. The model writes from those passages. That is the whole trick.
Fine tuning is different. You prepare hundreds or thousands of example question and answer pairs, run a training job that adjusts the model further on them, and keep the result. The knowledge ends up inside the model’s weights, not in a document store you can open, check or correct file by file.
One reads your files when asked. The other absorbs patterns in training.
Test one: how often does the content change?
A rate list updates every month. A circular replaces last quarter’s. A product manual gets a new revision when a part changes, and with RAG you simply replace the file so the next answer reflects it within minutes.
With fine tuning, a change means preparing new examples and running training again, and until then the model repeats the old rate with full confidence. The error is hard to spot. Fluent text hides it.
Content that rarely changes, such as a house style for letters or a fixed set of ticket labels, is where training makes sense.
- Monthly rate lists favour RAG
- Revised manuals favour RAG
- Stable tone and format can suit fine tuning
- Fixed classification labels can suit fine tuning
Test two: must every answer cite a document?
If a technician asks for the torque value on a flange, you want the page and the revision number beside the answer. RAG shows the passages it used along with the file name, the page and the revision, so the reader can confirm the torque value in seconds.
A fine tuned model has no single source to point to, since its knowledge was blended from thousands of examples. It produces text that sounds right. Warranty terms make that risky.
Where citations matter, choose retrieval. A warranty clerk quoting clause 4.2 of the dealer agreement to a customer wants that clause on screen, with the page number, before replying. Staff trust answers they can verify.
Where fine tuning still earns a place
Training helps with behaviour rather than knowledge. A model that must always answer in a fixed JSON layout, classify tickets into your twenty labels, or write in your support team’s voice can improve from examples.
The two can be combined. Retrieval supplies the facts and a lightly tuned model shapes the reply, though this costs more to build and maintain than either approach alone. Add it only after RAG falls short.
Most small firms never reach that point. Their effort is better spent cleaning the document folders, naming every file by revision, removing superseded circulars and agreeing who owns each manual.
A practical way to decide
Collect twenty real questions your staff ask, with the document that holds each answer, then build the retrieval version first since it is quicker to correct. Score the passage, then the answer.
Check access too. Retrieval can respect folder permissions, because the search filters by role before any passage reaches the model, so a sales user never sees HR files. A trained model has no such boundary. Whoever asks gets everything it learned.
Begin this week by listing your twenty questions, each paired with its source file, the owner of that file and its latest revision.
Assistant approach scorecard
A single-page scorecard has a row per document set, such as manuals, circulars and rate lists, and columns for change frequency, citation required yes or no and access. The last columns record the number of files, the verdict of RAG or fine tuning, and a test question with its expected source page and revision.
Open a blank worksheet to printQuestions owners ask
Is RAG cheaper than fine tuning?
Usually, because setup is quicker and an update means replacing one file instead of retraining a model on fresh examples. Costs depend on document volume, how often files change and how many people use it, so ask for figures against your own folder sizes.
Can fine tuning teach a model our documents?
It can absorb patterns, but it stays unreliable for exact facts such as rates, part numbers or clause references that a customer will check. It cannot cite the source page.
Can we use both together?
Yes, with retrieval supplying current facts and a tuned model shaping the tone or the format of each reply to match your support team. Try RAG alone first. Tune only if testing shows a gap.
How does RAG respect who may see what?
The search filters by folder or role before any passage reaches the model, so each user sees only what their permissions already allow. A salesperson never retrieves HR files.