TDM 20100: Project 3 — Fall 2026

  • Introduction to Retrieval-Augmented Generation (RAG)

Motivation: Large Language Models (LLMs) are powerful tools, but they are limited to information that’s within their training data and may generate incorrect responses. Retrival-Augmented Generation (RAG) improves the accuracy of LLMs by allowing them to retrieve and use information from external documents when generating answers.

Context: In this project, students will build a RAG pipeline using Ollama, LangChain, and Milvus. Students will convert document text into embeddings, store those embeddings in a vector database, retrieve relevant document chunks in response to questions, and evaluate how context influences generate answers.

Scope: RAG, Ollama, LangChain, Milvus, document chunking

Learning Objectives

  • Understand the purpose and workflow of a Retrieval-Augmented Generation system.

  • Create embeddings from document text and store them in a vector database.

  • Evaluate how document retrieval affects the accuracy and relevance of generated answers.

  • Analyze similarity scores on specific document chunks to understand selections made during retrieval.

Dataset

In Question 2, we will use a PDF document as our "dataset" to send to the LLM. A relevant PDF file that you could try is:

During week 2, in your Jupyter Lab session, please be sure to use The Data Mine Jupyter Lab notebook, with 4 cores per session.

Questions

Question 1 (2 pts)

Please read the overview about RAG here:

the-examples-book.com/projects/fall2026/20100/rag

and then run the 3 lines from this overview:

print("Using Milvus DB at:", URI)

print("Using the collection:", collection_name)

print(os.environ["OLLAMA_HOST"])
Deliverables

  • Print the three lines above, from the RAG setup documentation.

Question 2 (2 pts)

Now read the LangChain documentation given here:

the-examples-book.com/projects/fall2026/20100/rag2

Store at least 3 questions into the questions variable. Show the output from llm.invoke.

Deliverables

  • Store at least 3 questions into the questions variable. Show the output from llm.invoke.

Question 3 (2 pts)

Now read the documentation on Converting and Splitting Text given here:

the-examples-book.com/projects/fall2026/20100/rag3

Show the chunks of page content from The Data Mine example document:

datamine.purdue.edu/posters/TDM-Report_2024-4.pdf

Deliverables

  • Show the chunks of page content from The Data Mine example document

Question 4 (2 pts)

Now go through the document here, on Embeddings and Building a Database of Vectors:

the-examples-book.com/projects/fall2026/20100/rag4

culminating in asking questions about The Data Mine example document.

Print the results from asking two questions about The Data Mine example document.

Deliverables

  • Print the results from asking two questions about The Data Mine example document.

Question 5 (2 pts)

Now try the framework in this project on a document of your own choosing!

Print the results from asking two questions about the document that you chose.

Deliverables

  • Print the results from asking two questions about the document that you chose.

  • Based on your queries, does the LLM represent your document somewhat accurately?

Submitting your Work

Please make sure that you added comments for each question, which explain your thinking about your method of solving each question. Please also make sure that your work is your own work, and that any outside sources (people, internet pages, generative AI, etc.) are cited properly in the project template.

Prior to submitting your work, you need to put your work into the project template, and re-run all of the code in Jupyter Lab and make sure that the results of running that code is visible in your template. Please check the detailed instructions on how to ensure that your submission is formatted correctly. To download your completed project, you can right-click on the file in the file explorer and click 'download'.

Items to submit

  • firstname_lastname_project3.ipynb

It is necessary to document your work, with comments about each solution. All of your work needs to be your own work, with citations to any source that you used. Please make sure that your work is your own work, and that any outside sources (people, internet pages, generative AI, etc.) are cited properly in the project template.

You must double check your .ipynb after submitting it in gradescope. A very common mistake is to assume that your .ipynb file has been rendered properly and contains your code, markdown, and code output even though it may not.

Please take the time to double check your work. See here for instructions on how to double check this.

You will not receive full credit if your .ipynb file does not contain all of the information you expect it to, or if it does not render properly in Gradescope. Please ask a TA if you need help with this.