Enterprise RAG gives language models direct access to the organisation's information assets. By retrieving precise fragments and forcing the AI to reason only over them, this mechanism acts as the operational memory any artificial intelligence strategy needs, and stands as the more efficient option against fine-tuning in corporate settings.

In short

• RAG (retrieval-augmented generation) solves the context problem; fine-tuning solves the style and format problem. They are complementary, not rivals.
• The quality of an enterprise RAG depends more on chunking, permissions and evaluation than on the model you choose.
• Without permission inheritance from the source system, a RAG becomes an information leak.
• A bank of 80–150 real questions with known answers is the deliverable that decides whether the project survives.
• Realistic timeline for an enterprise RAG in production with three sources: 4 to 6 weeks.
• RAG is layer 2 of the 5-layer architecture, not the whole project.

What is enterprise RAG and how does it differ from a search engine?

A search engine returns documents. An enterprise RAG returns a written answer, built on specific fragments of those documents and with a reference for each one.

The flow has four moments: the question is turned into a query, the most relevant fragments are retrieved from the knowledge base, they are handed to the model as mandatory context, and the model writes the answer citing where each statement comes from.

The practical difference is that RAG can combine five different documents into a single answer and say "this does not appear in the documentation" when it finds no support. A search engine never says that.

When does RAG make sense and when does fine-tuning?

Criterion	Enterprise RAG	Fine-tuning
Problem it solves	Lack of up-to-date knowledge	Lack of specific style, format or tone
Data that changes daily	Ideal	Unworkable
Source traceability	Native	None
Per-user permissions	Applicable at retrieval	Not applicable
Cost of updating	Re-ingest the document	Full retraining
Typical timeline	4–6 weeks	Weeks or months depending on volume
Main risk	Poor retrieval	Memorising sensitive data in the weights

Practical rule: if the question is "what does our documentation say?", it is RAG. If the question is "how do we write a proposal?", it may be fine-tuning. In more than 90% of the business cases that reach NEXTAI, the real need is RAG and the initial request was fine-tuning.

How do you implement an enterprise RAG step by step?

1. Bound the use case. One process, one type of question, one group of users. Not "all the company's documentation".
2. Inventory and clean up the sources. What exists, who maintains it, what is duplicated and what has been superseded. Detail in what data your AI needs.
3. Build the evaluation bank before the system. 80 to 150 real questions with the correct answer written by someone from the business.
4. Define the permission policy. Which role can see what. Permissions are inherited from the source system and applied at retrieval, not at writing.
5. Ingest with structure. Preserve titles, sections, tables and metadata: client, date, version, validity, area.
6. Chunk with hierarchy. Variable-size fragments that respect clauses, sections and table rows, with moderate overlap and the section title carried into each fragment.
7. Choose and compute embeddings. A multilingual model that performs well in your working languages; the version is pinned and documented, because changing it forces a full recompute.
8. Hybrid retrieval. Semantic search plus keyword search, with reranking of the candidates afterwards. Semantic search alone fails on references, product codes and case numbers.
9. Writing with mandatory citation. The model answers only from what was retrieved and cites the source; if there is not enough support, it says it does not know.
10. Measure, tune and open in phases. Run the evaluation bank, fix the failures layer by layer, and open to a pilot group before the whole company.

What mistakes sink an enterprise RAG project?

Mistake	Symptom it produces	Correction
Fixed-size chunking	Truncated answers, broken tables	Chunking by document structure
Ingesting old versions unmarked	The AI cites a superseded procedure	Validity metadata and filtering by default
Not inheriting permissions	A sales rep sees payroll	Permissions applied at retrieval
Semantic search only	Fails on codes and exact references	Hybrid retrieval with reranking
No evaluation bank	Nobody knows whether a change helps or hurts	80–150 questions with known answers
Retrieving too many fragments	High cost and confusing answers	Reranking and a context limit
Opening it to the whole company on day one	Irreversible loss of trust	Pilot group and phased opening

From our experience on NEXTAI projects, between 60% and 75% of the failures a user perceives as "the model is making things up" are fixed in the retrieval layer, without touching the model.

How do you evaluate whether a RAG works?

With three separate metrics, because each points to a different cause:

1. Retrieval precision. Was the correct fragment among those retrieved? If this metric fails, the problem is ingestion or chunking.
2. Answer faithfulness. Is everything asserted supported by what was retrieved? If this fails, the problem is the instructions or the model.
3. Business usefulness. Would the person who owns the process have answered the same way? Only someone from the business can validate this.

Recommended opening threshold before releasing to the whole organisation: retrieval above 90% and faithfulness above 95% on the evaluation bank. Below that, you carry on in pilot mode.

How does it look in a real company?

Realistic case: an agricultural machinery manufacturer in southern Europe. 110 employees, €24M turnover, 900 technical manuals and 15 years of after-sales service reports.

Problem: the technical service took between 25 and 50 minutes to locate the correct procedure for a fault, cross-referencing the manual, the machine version and previous reports. Two veteran technicians held the useful knowledge.

Implementation: three sources (manuals, service reports and the spare parts catalogue), chunking by manual section, mandatory model and version metadata, hybrid retrieval (spare part codes did not work with semantic search alone) and a bank of 120 questions written by the technicians themselves.

Result after phase 2 of the NEXT-5 Roadmap, from our experience on NEXTAI projects with equivalent profiles: locating the procedure went from 25–50 minutes to 3–6 minutes, retrieval precision of 92% on the bank, and one less visible but more valuable consequence: new technicians stopped depending on the two veterans.

How NEXTAI does it

NEXTAI implements enterprise RAG as the Memory layer of the Enterprise Brain, never as a standalone tool. We start with the evaluation bank and the permission policy: they are the two deliverables that decide whether the system can be opened to the organisation.

We document the embedding model, the chunking strategy and the validity rules, so your team can maintain it without us. When retrieval clears the agreed threshold, and only then, we connect Superagents on top. Permissions and traceability are designed with the AI governance framework from week one.

Frequently asked questions

How much does it cost to implement an enterprise RAG? A RAG bounded to one process and three sources sits in the indicative market range of €3,000 to €15,000 to implement, plus the recurring cost of infrastructure, embeddings and queries. Extending it to several areas with fine-grained permissions and a knowledge graph moves it into the €15,000 to €50,000 range.

Do I need my own model or will a commercial one do? In most cases a commercial model will do, because the knowledge is in the retrieval, not in the model. An open model on your own infrastructure is justified when the data is particularly sensitive or when query volume makes the cost per call dominate the budget.

How do I stop the system leaking information between departments? By applying permissions at retrieval, inherited from the source system, and not as an instruction to the model. Asking a model "not to reveal" something it has in fact retrieved is not access control. It is also worth logging every query with the user, the fragments retrieved and the answer.

How long until something is in production? Between 4 and 6 weeks for a bounded case with three sources, including evaluation and permissions. Projects that promise two weeks usually skip evaluation and permission inheritance, which is exactly where the problems show up in month three.

RAG sharply reduces hallucinations, though it does not eradicate them. The key is to force source citation and set confidence thresholds that reveal when the information is insufficient. Combining that rigour with human oversight is the only way to turn experimental systems into reliable decision tools. If you want to assess how ready your data is, our free Digital Audit identifies which sources are ready to be integrated.