TDM 20100: Project 2 — Fall 2026

  • Introduction to RAG and Text Embeddings

Motivation: In Project 2, we learn about embeddings, and also how to use a RAG pipeline.

Context: LLMs are not used in isolation. We start to see how to use LLMs as part of a pipeline.

Scope: Anvil, Jupyter Lab, embeddings, RAG

Make sure to read about, and use the template found here, and the important information about project submissions here.

During week 2, in your Jupyter Lab session, please be sure to use The Data Mine Jupyter Labl notebook, with 4 cores per session.

Questions

Question 1 (2 pts)

As a warmup, return to the llama-3.2 model that you used in Project 1, and identify some flaws with the model, because it was created a couple of years ago. For instance:

Identify at least 2 queries for which the response is lacking, in terms of lacking knowledge about recent events.

Identify at least 2 queries for which the response has incomplete information, for instance, about lists of facts.

Deliverables

  • Identify at least 2 queries for which the response is lacking, in terms of lacking knowledge about recent events.

  • Identify at least 2 queries for which the response has incomplete information, for instance, about lists of facts.

  • Be sure to document your work from Question 1 (and all questions! always! we will start dropping the note to document your work, because you always need to document your work).

Question 2 (2 pts)

Read this discussion about text embeddings:

the-examples-book.com/projects/fall2026/20100/textembeddings

Create mxbai-embed-large-2 as discussed.

Use /anvil/projects/tdm/bin/ollama list in your notebook to show that mxbai-embed-large-2 was created.

Then compare the similarity between the first ten values of the vectors of each of "The sky is blue", "The grass is green", and "I like cats".

Explain (in 1 to 2 sentences) what you notice from comparing the similarity of these partial vectors. (We say "partial" because the vectors are much longer than ten values, but we are just starting with the first ten values.)

Deliverables

  • Explain (in 1 to 2 sentences) what you notice from comparing the similarity of these partial vectors built from the first 10 values.

Question 3 (2 pts)

Read more about cosine similarity here:

the-examples-book.com/projects/fall2026/20100/cosinesimilarity

Now embed the phrases "The sky is blue", "The grass is green", and "I like cats" as their full numerical vectors.

Use cosine similarity to compare the entire vector of each phrase against the others.

Deliverables

  • Use cosine similarity to compare the entire vector of each phrase against the others.

  • Explain (in 1 to 2 sentences) what you notice from comparing the similarity of these vectors

Question 4 (2 pts)

Find two phrases of your own, which have very strong cosine similarity.

Find two phrases of your own, which have very different cosine similarity.

Deliverables

  • Find two phrases of your own, which have very strong cosine similarity.

  • Find two phrases of your own, which have very different cosine similarity.

Question 5 (2 pts)

Download gemma3 and then use create an all-new model starts with gemma3 and only uses 2 threads. Call this new model gemma3-2.

Revisit question 1, using this gemma3-2 model that you built:

Do your queries from question 5 still lack knowledge about recent events?

Do your queries about incomplete information (for instance, about lists of facts) still have the same gaps?

Do you see any similarities and differences with this new model?

Deliverables

  • Do your queries from question 5 still lack knowledge about recent events?

  • Do your queries about incomplete information (for instance, about lists of facts) still have the same gaps?

  • Do you see any similarities and differences with this new model?

Submitting your Work

Please make sure that you added comments for each question, which explain your thinking about your method of solving each question. Please also make sure that your work is your own work, and that any outside sources (people, internet pages, generative AI, etc.) are cited properly in the project template.

Prior to submitting your work, you need to put your work into the project template, and re-run all of the code in Jupyter Lab and make sure that the results of running that code is visible in your template. Please check the detailed instructions on how to ensure that your submission is formatted correctly. To download your completed project, you can right-click on the file in the file explorer and click 'download'.

Items to submit

  • firstname_lastname_project2.ipynb

It is necessary to document your work, with comments about each solution. All of your work needs to be your own work, with citations to any source that you used. Please make sure that your work is your own work, and that any outside sources (people, internet pages, generative AI, etc.) are cited properly in the project template.

You must double check your .ipynb after submitting it in gradescope. A very common mistake is to assume that your .ipynb file has been rendered properly and contains your code, markdown, and code output even though it may not.

Please take the time to double check your work. See here for instructions on how to double check this.

You will not receive full credit if your .ipynb file does not contain all of the information you expect it to, or if it does not render properly in Gradescope. Please ask a TA if you need help with this.