Step 4: Generate

In the last section, we learned how retrieve raw chunks using the retrieval API, but now it's time to put it all together using a prompt and an LLM to generate our final content. Remember, the primary advantage of building a RAG application is to generate content with data from sources that the LLM has not been trained on. This can yield outstanding results for a wide variety of applications. We do this by telling the language model to use our chunks to help generate an answer.

To see how it all works, we'll create a very basic command line application that uses data that we've already provided to Ragie in the previous steps of this tutorial.

Let's create a quick command line RAG application

Let's walk through an example that uses Ragie to pull chunks and OpenAI as the LLM to generate content.

First, let's set the project up.

mkdir generate-example
cd generate-example

Create a package.json

{
  "name": "generate-example",
  "version": "1.0.0",
  "main": "index.mjs",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": ""
}

Install the openai client:

npm install openai

Create an index.mjs file with the following content (NOTE: use the .mjs file extension)

import OpenAI from "openai";

const ragieApiKey = "<YOUR RAGIE API KEY>";
const openAiApiKey = "<YOUR OPENAI API KEY>";

const query = "What does Chamath think about Davos?";

const response = await fetch("https://api.ragie.ai/retrievals", {
  method: "POST",
  headers: {
    "Content-Type": "application/json",
    Authorization: "Bearer " + ragieApiKey,
  },
  body: JSON.stringify({ query, filters: { scope: "tutorial" }}),
});

if (!response.ok) {
  console.error(
    `Failed to retrieve data from Ragie API: ${response.status} ${response.statusText}`
  );
  process.exit(1);
}

const data = await response.json();
const chunkText = data.scored_chunks.map((chunk) => chunk.text);
const systemPrompt = `These are very important to follow:

You are "Ragie AI", a professional but friendly AI chatbot working as an assitant to the user.

Your current task is to help the user based on all of the information available to you shown below.
Answer informally, directly, and concisely without a heading or greeting but include everything relevant.
Use richtext Markdown when appropriate including bold, italic, paragraphs, and lists when helpful.
If using LaTeX, use double $$ as delimiter instead of single $. Use $$...$$ instead of parentheses.
Organize information into multiple sections or points when appropriate.
Don't include raw item IDs or other raw fields from the source.
Don't use XML or other markup unless requested by the user.

Here is all of the information available to answer the user:
===
${chunkText}
===

If the user asked for a search and there are no results, make sure to let the user know that you couldn't find anything,
and what they might be able to do to find the information they need.

END SYSTEM INSTRUCTIONS`;

const openai = new OpenAI({ apiKey: openAiApiKey });
const chatCompletion = await openai.chat.completions.create({
  messages: [
    { role: "system", content: systemPrompt },
    { role: "user", content: query },
  ],
  model: "gpt-4o",
});

console.log(chatCompletion.choices[0].message.content);

This example will answer the question "What does Chamath think about Davos?" using Ragie to return chunks found from the All-In Podcast files that we uploaded earlier.

Run the example:

node index.mjs

If everything is set up correctly, you should see an answer that looks similar to the following:

Chamath is quite critical of Davos. Here are his thoughts:

- **Irrelevance to Builders:** He believes that nobody building anything significant cares about Davos or even knows when it happens. It's not important for those who are genuinely contributing to innovation and development.
- **Status Symbol:** He views Davos as a place for those who care about status, went to prestigious schools, and want to feel like they're part of an exclusive club.
- **Membership Organization:** Chamath describes Davos as a membership organization where people pay substantial amounts based on the number of participants they bring.
- **Appeal to Elites:** He feels that Davos appeals to insecure overachiever elites who value the status signal it represents rather than substantive contributions or discussions.
- **Worthless Discussions:** According to Chamath, the discussions at Davos are not noteworthy as the speakers don't say anything significant or different from what they would say elsewhere.
- **Criticism of Elitism:** He criticizes Davos for being more about elitism, where people pay to feel important rather than fostering genuine, impactful dialogues.
- **Compares to Other Conferences:** He compares it to other conferences, mentioning that the same trivial activities happen at Davos as they do in places like CES in Vegas, though with more security guards.

Chamath believes that such gatherings have lost their original purity and have become revenue-driven efforts that mainly signal status to a specific group of elites.

Congrats! We just completed a RAG application! 🙌

If this was your first time trying RAG, congratulations! If you're an experienced vet, then you probably understand how much time Ragie saved you by handling the data ingest pipeline and giving you some tools for easy retrieval. Either way, nice job.

Let's break down what happened here

Here's the RAG breakdown:

1. We (R)ETRIEVED results from Ragie. We used a filter so we only returned data from the tutorial scope. In our retrieval step, we used the Ragie defaults which includes reranking so we only get the best chunks helping us avoid hallucinations during generation.
2. We (A)UGMENTED our prompt with the chunks from Ragie. Our system prompt included a lot of context setting which is common for RAG applications. We also injected raw chunks of text returned from Ragie into the prompt, which is what the LLMs use to generate content for data that they have not been trained on.
3. Finally, we (G)ENERATED an answer using an API call to OpenAI for a chat completion. OpenAI used our prompt injected with our chunks from Ragie to generate a meaningful answer to our original question: "What does Chamath think of Davos?"