Building RAG with Open AI

Step 1: Verify Docker Installation

First, let’s verify that Docker is properly installed and open in your system. Open your terminal (Command Prompt on Windows) and run:

docker --version

You should see the Docker version information if it's installed correctly. If not, you might want to revist the prerequisites and the Docker Basics section.

Step 2: Clone the LLM App Templates Repository

Next, clone the llm-app repository from GitHub. This repository contains all the files you’ll need.

git clone https://github.com/pathwaycom/llm-app.git

If you get an error because you have previously cloned an older version of the llm-app repository, ensure you're in the correct repository directory and update it using:

git pull

This will update your local repository with the latest changes from the remote repository.

Step 3: Navigate to the relevant project directory

Change to the directory where the example is located.

cd examples/pipelines/demo-question-answering

Step 4: Create a .env File and put your Open API key

Create a file named .env and add your Open AI API key using the Bash command below.

echo "OPENAI_API_KEY=<your_openai_api_key>" > .

Alternatively, you can open your preferred text editor (e.g., Notepad), create a new file and add just the following line:

OPENAI_API_KEY=<your_openai_api_key>

Then, save the file as environment (.env) in the demo-question-answering folder. It will do the same thing.

Step 5: Build the Docker Image

Now, let’s build the Docker image. This step might take a few minutes depending on your machine. Ensure you have enough space (approximately 8 GB).

docker build -t rag .

The -t rag part is tagging the Docker image with the name ‘rag’. Whereas the . at the end specifies the build context directory, which is the current directory. This tells Docker to look for the Dockerfile in the current directory and include any files/subdirectories in the build context.

Step 6: Run the Docker Container

Run the Docker container, mounting (described below) the data folder, and exposing port 8000.

For Windows:

docker run -v "%cd%/data:/app/data" -p 8000:8000 rag

For Linux/Mac:

docker run -v "$(pwd)/data:/app/data" -p 8000:8000 --env-file .env rag

Note: You will see the logs for parsing & embedding documents in the Docker image logs. Give it a few minutes to finish up on embeddings. You will see 0 entries (x minibatch(es)) have been... message. If there are no more updates, this means the app is ready for use!

Handling Port Conflicts: If port 8000 is already in use and you see an error related to it, you can specify a different port. For example, if you want to use port 8080 instead, modify the command as follows:

For Windows:

docker run -v "${PWD}/data:/app/data" -p 8000:8000 rag

For Linux/Mac:

docker run -v "$(pwd)/data:/app/data" -p 8080:8000 --env-file .env rag

This will map port 8080 on your local machine to port 8000 in the Docker container. Just remember to update the port in the next step as well.

Open up another terminal window and follow the next steps.

Step 7: Check the List of Files

You will see the logs for parsing & embedding documents in the Docker image logs. Give it a few minutes to finish up on embeddings, you will see 0 entries (x minibatch(es)) have been… message. If there are no more updates, this means the app is ready for use! Now let’s see the files from which we’ll retrieve information for our LLMs.To test it, let's query to get the list of available inputs and associated metadata using the curl command:

curl -X 'POST'   'http://localhost:8000/v1/pw_list_documents'   -H 'accept: */*'   -H 'Content-Type: application/json'

This will return the list of files e.g. if you start with the data folder provided in the demo, the answer will be as follows:

[{"created_at": null, "modified_at": 1718810417, "owner": "root", "path":"data/IdeanomicsInc_20160330_10-K_EX-10.26_9512211_EX-10.26_Content License Agreement.pdf", "seen_at": 1718902304}]

Step 8: Last Step – Run the RAG Service

You can now run the RAG service. Start by asking a simple question. For example:

curl -X 'POST' \
  'http://0.0.0.0:8000/v1/pw_ai_answer' \
  -H 'accept: */*' \
  -H 'Content-Type: application/json' \
  -d '{
  "prompt": "What is the start date of the contract?"
}'

It should return the following answer:

December 21, 2015

---

But how do you tweak this for your use-case? Let's see that by understanding the contents of the repo which just used.