TDM 19000: Week 1 Questions — Summer 2026

Large Language Models

questions to be posted soon

Project Objectives

Motivation: We want to understand how large language models work. We download and try some variants of language models using Ollama which has many models in its Library.

Context: Working with llama3.2, which is a 2 GB model, we test several model behaviors, check timings, and learn about similarity scores.

Scope: This is only an introduction to large language models, aimed at students with no background in the data sciences.

Learning Objectives

Introduction to large language models, behaviors, timings, and similarity scores.

Question 1

1.1 Start the Ollama server and show the process status of the model. (At the outset, it should only show NAME ID SIZE…​

1.2 In 2 to 3 sentences, explain what Ollama is, and what its main functions are. You may want to read more about Ollama on the internet. Please list any websites that you read here, or explain how you used generative AI to answer this question. (We want you to do this for all questions, i.e., if you use any web-based resources or any generative AI or help from other people, please indicate this in your submissions.)

1.3 In 1 to 2 sentences, explain why we need to use a symbolic link to the $SCRATCH diretory in our setup.

Question 2

2.1 Download the llama3.2 model to your scratch directory.

2.2 Create a model using a custom modelfile to specify the number of threads to use.

2.3 Explain (in 2 to 3 sentences) the purpose of using the mymodel file.

2.4 Add comments to your mymodel file.

Question 3

3.1 Use /anvil/projects/tdm/bin/ollama list in your notebook to display the created models.

3.2 Compare the outputs (written response, response time) of sending a prompt to the model using 1 thread, 2 threads, and 4 threads. Please report the times for all three models.

3.3 Explain (in 1 to 2 sentences) the differences that you noticed between the models and their responses.

Question 4

4.1 Use '/anvil/projects/tdm/bin/ollama list' in your notebook to list the current models.

4.2 Try creating at least two models with new system behavior definitions.

4.3 Explain how (in 2 to 3 sentences) your new models behave compared to the default?

Question 5

Identify some flaws in the llama-3.2 model, namely:

5.1 Identify at least 2 queries for which the response is lacking, in terms of lacking knowledge about recent events.

5.2 Identify at least 2 queries for which the response has incomplete information, for instance, about lists of facts.

Question 6

6.1 Use /anvil/projects/tdm/bin/ollama list in your notebook to show that mxbai-embed-large-2 was created.

6.2 Compare the similarity between the first ten values of the vectors of each of "The sky is blue", "The grass is green", and "I like cats".

6.3 Explain (in 1 to 2 sentences) what you notice from comparing the similarity of these partial vectors.

Question 7

7.1 Embed the phrases "The sky is blue", "The grass is green", and "I like cats" into numerical vectors

7.2 Use cosine similarity to compare the entire vector of each phrase against the others

7.3 Explain (in 1 to 2 sentences) what you notice from comparing the similarity of these vectors

Question 8

8.1 Find two phrases of your own, which have very strong cosine similarity.

8.2 Find two phrases of your own, which have very different cosine similarity.

Question 9

What happens if you try to download and use llama4? (It will NOT work, but it would be helpful for you to experience the error!)

Question 10

Download gemma3 and then use mymodel that starts with gemma3 and only uses 2 threads. Call this new model gemma3-2.

Revisit question 5, using this gemma3-2 model that you built:

10.1 Do your queries from question 5 still lack knowledge about recent events?

10.2 Do your queries about incomplete information (for instance, about lists of facts) still have the same gaps?

Submitting your Work

Once you have completed the questions, save your Jupyter notebook. You can then download the notebook and submit it to Gradescope.

Items to submit

  • firstname_lastname_week1.ipynb

You must double check your .ipynb after submitting it in gradescope. A very common mistake is to assume that your .ipynb file has been rendered properly and contains your code, markdown, and code output even though it may not. Please take the time to double check your work. See here for instructions on how to double check this.

You will not receive full credit if your .ipynb file does not contain all of the information you expect it to, or if it does not render properly in Gradescope. Please ask a TA if you need help with this.