Building a RAG Model

| November 11, 2025 | in

It feels like everyone wants to sprinkle a little AI on every software project right now. Sometimes for good reason, and sometimes just because they want a little AI. But AI, much like a seasoning for our food, might not be an improvement. We need to know what tool to use and when.

One of those tools is a RAG Model. RAG stands for Retrieval-Augmented Generation. Basically, some fancy words to say we are telling an LLM to source its answers from a known set of answers.

RAG models have a few advantages:

We can provide custom knowledge to an LLM. We could give an LLM access to specific data. Data such as our customer list or our customer support FAQ.
We can force the LLM to cite its answers, reducing hallucination.
To make changes to our data source, we only have to update our vector database. We don’t have to retrain an entire model.

Building a RAG model and querying it is a straightforward process. Probably the biggest issue is figuring out which tools you want to use to complete the task.

Working with a RAG model requires two steps:

Building the embedding vector database.
Querying and output generation.

Flowchart showing RAG pipeline: JSON files and user query processed into FAISS vector store, LLM, and response.

This might seem odd, in that we really don’t use the LLM until we are ready to generate the answer, but it actually works pretty well.

Below are the steps for creating a vector database.

Sequence diagram of RAG setup: system loads JSON, converts to text, embeds, saves vectors to FAISS database.

Once you have a vector database, you can query the vector database, then ask an LLM to generate an answer.

Sequence diagram of query flow: user question embedded, similarity search retrieves docs, LLM generates answer.

Once you have this up and running, you have a RAG that you can query and ask questions of. You can ask questions, such as when did “Bob Smith” join our gym, and get real answers.