What Is RAG? How AI Actually Reads Your Company Documents

RAG makes an AI search your documents first and answer second. It stands for Retrieval-Augmented Generation. The model never "learns" your company documents — it is handed the relevant passages to read the moment a question arrives, and answers from what it just read. This is what Thai businesses actually mean by "we want AI that knows our company documents": not fine-tuning, no retraining, no GPU of your own.

How RAG works — the four steps behind every answer

The first step happens once at setup. The other three happen fresh on every question:

1. 1.Index (once) — split documents into chunks, turn each into a vector with an embedding model, store them in a vector database.
2. 2.Retrieve — the question is embedded the same way, and the system pulls the k nearest chunks (usually 4-8).
3. 3.Augment — those chunks go into the prompt with an instruction: answer only from this material; if it isn't here, say you don't know.
4. 4.Generate — the model answers from what it just read, and can point to the file and page it came from.

Why RAG beats fine-tuning for this job

Fine-tuning teaches a model how to speak, not what is true. Bake your price list into the weights and when prices change next week there is no fix short of a full retraining run.

The advantage Thai teams overlook: PDPA

If the documents contain personal data — customer records, phone numbers, delivery addresses, HR files — Thailand's Personal Data Protection Act gives the data subject a right to request erasure. With RAG you delete the row and you are done, provably. With fine-tuning that data has been fused into the weights and there is no per-record delete; the only remedy is retraining without it. That is the legal reason RAG is the correct default for internal documents at almost any Thai organisation.

What RAG actually costs a Thai SME

A worked hypothetical, not a client invoice. Assume 2,000 pages of internal documents (SOPs, price lists, warranty policy, staff handbook) at roughly 1,200 tokens per page once tokenised as Thai — about 2.4M tokens:

• •Initial index — text-embedding-3-small at $0.02 per 1M tokens → 2.4M × $0.02 = $0.048 (~฿1.70), once (Gemini Embedding is $0.15 per 1M with a free tier). A monthly re-index of 5% of the corpus costs under ten satang.
• •3,000 answers a month — ~3,900 input tokens per question (6 chunks + system prompt + question) and ~300 output. On Gemini 2.5 Flash-Lite ($0.10/$0.40 per 1M) that is $1.53 (~฿55) a month; on GPT-5.6 Luna ($0.20/$1.20), $3.42 (~฿123).
• •Vector database — pgvector on the Postgres you already run costs ฿0. The bills that shock people come from managed vector DBs priced on index size, not from model calls.

Every model cost combined lands under ฿150 a month. That is the point: the cost of RAG is not tokens. It is the person-days spent finding the right chunking strategy, building an evaluation set, and making re-indexing automatic. Anyone quoting a RAG project purely on API spend has not shipped one to production.

Why Thai text is harder to chunk than English

Nearly every RAG tool is designed around English assumptions, and those break on Thai immediately.

• •Thai has no spaces between words. A chunker that splits on whitespace and counts "500 words" sees a whole Thai paragraph as one blob, then falls back to a character limit — which usually cuts mid-word.
• •Thai has no sentence-ending full stop. Regex splitters looking for . ? ! find zero boundaries, so the whole file becomes one chunk and gets truncated at the limit.
• •Thai costs more tokens for the same meaning. Mainstream tokenizers are trained mostly on English and shred Thai script into small pieces, so a 500-token Thai chunk carries less information than a 500-token English one. Count with the real tokenizer instead of trusting English intuition.
• •Embeddings know Thai less well. Multilingual models have seen far less Thai, so recall on Thai queries is naturally lower — worse when customers misspell or run words together.

How to make Thai retrieval more accurate

All four are fixable, and cheaply, if you know about them up front:

• •Chunk on document structure, not character count. Thai business documents are almost always numbered (ข้อ 1, ข้อ 2, บทที่ 3) — split there. And keep table headers with their rows: a separated header is the number-one cause of a bot quoting the wrong price.
• •Segment Thai words first. Run PyThaiNLP (the newmm engine) to insert word boundaries before chunking — the same output feeds BM25.
• •Hybrid search plus a Thai-English alias table. Vector search alone misses SKUs; run BM25 alongside it, and link what customers say ("เครื่องซักผ้าฝาหน้า") to what the document says (Front Load Washer WM-2200).

Where RAG breaks — and it almost always breaks silently

When RAG fails it does not raise an error. It answers wrongly with exactly the same confidence.

1. 1.Chunking splits the answer in half. The warranty condition ends chunk 1, the exception starts chunk 2, one is retrieved, and the customer gets half the truth.
2. 2.Retrieval misses. The correct chunk ranks 7th and you set k=5. The model never saw the right answer and politely invents one.
3. 3.The index goes stale. Sales uploads a new price list but the pipeline never re-indexes. Re-index on file change, not when someone remembers.
4. 4.Documents contradict each other. Four versions of a price list in one folder, two retrieved, the model picks one. RAG does not clean up messy documents — it answers from them faster.
5. 5.RAG does not make a model competent in a domain it does not understand. Feed the Thai Revenue Code to a model that does not grasp Thai tax and it still misreads it. Retrieval supplies text, not understanding.

When not to build RAG (an opinionated take)

More than half of the failed RAG projects I see in Thailand failed because they should never have started. Four cases where you should stop:

• •The whole corpus fits in context. An 80-page staff handbook drops into the prompt comfortably on current models, and prompt caching makes it cheap. A vector pipeline for 80 pages is engineering theatre.
• •There are only a few answers and they never change. Thirty questions like "what time do you open?" belong in a lookup table or a LINE OA rich menu. Right 100% of the time beats right 92% of the time. (See the LINE OA chatbot guide.)
• •The documents still contradict each other. If nobody knows which price list is authoritative, index nothing yet. Cleaning documents is the job nobody wants and the one that decides the outcome.
• •The question requires aggregation. "What was total revenue last quarter?" — RAG retrieves top-k chunks; it cannot sum 4,000 rows. That is a SQL job, not retrieval.

If you are going to build it, start here

This order is designed to fail in the right place, before you have spent money:

1. 1.Collect 50 real questions from customer chats, each labelled with the file containing the correct answer. This set is the yardstick for the project.
2. 2.Clean the documents. Delete old versions and give each file an owner allowed to edit it.
3. 3.Start with the simplest thing that works — chunk on headings, pgvector on your existing Postgres. Do not buy a managed vector DB yet.
4. 4.Measure recall@k before connecting the LLM, then add hybrid search and a reranker once measurement shows where you are missing.