TDM 19000: Embeddings
Embeddings in LLMs
Embeddings in LLMs are numerical vector representations of text, images, or audio generated by the model. A sentence of text - such as "The sky is blue" - will get converted into text that represents its meaning. These vectors allow computers to search and compare text based on meaning rather than written matches.
Mxbai-embed-large is a highly rated open-source embedding model made by Mixedbread AI. It was designed for high-performance Retrieval-Augmented Generation (RAG) - the process of improving LLM accuracy by using data from external sources when generating a response.
This model can be put in SCRATCH using /anvil/projects/tdm/bin/ollama pull mxbai-embed-large.
Take another modelfile and create a version of mxbai-embed-large that uses just 2 threads.
%%bash
cat > ~/mymodel << HERE
FROM mxbai-embed-large
PARAMETER num_thread 2
HERE|
Do not forget to use this |
In your notebook, run the following to test embedding a phrase.
/anvil/projects/tdm/bin/ollama run mxbai-embed-large-2 "The sky is blue"This should output a long vector of numerical values that represent the phrase you entered.
Take the first ten values from the vectors of three phrases:
# phase: "The sky is blue"
the_sky_is_blue = [-0.021142907,0.009792235,-0.00078518514,-0.041012164,-0.028142547,0.007936959,-0.0028563524,0.04603205,0.07597107,0.043744832]
# phase: "The grass is green"
the_grass_is_green = [-0.050176393,0.011092771,-0.005391531,0.011824846,-0.0063454183,0.052363742,0.033608414,0.010098344,0.04681839,0.04693919]
# phrase: "I like cats"
cats = [-0.015537318,0.03649514,0.026832601,-0.031201454,-0.021033717,-0.04197758,0.008024173,0.042692337,0.0215113,0.024097823]We would assume that two of these vectors would be more similar to each other than with the third. However, 'the_sky_is_blue', 'the_grass_is_green', and 'cats' contain just ten out of many values in the numerical vectors of their original phrases.
Use:
import numpy as np
similarity = np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))
print("Similarity score:", similarity)….to compare the similarity between each pair of the three vectors.
Cosine Similarity in LLMs
We made comparisons on the first ten numerical values to appear in the embedded vectors of some phrases. These vectors actually contain a lot more than ten values, and each is important for defining what has been embedded.
We could copy and paste the entire vector that outputs from running '/anvil/projects/tdm/bin/ollama run mxbai-embed-large-2 "The sky is blue"', but this is not very nice to do; it is long and messy, and some values could easily get left out.
Instead, we will use subprocess to run the mxbai-embed-large-2 model on each phrase, then we’ll save the vectors into a list that can easily be used for comparison.
import subprocess
import numpy as np
import ast
# run the Ollama mxbai-embed-large-2 model on "The sky is blue"
# capture the output as a string
result = subprocess.run(
['/anvil/projects/tdm/bin/ollama', 'run', 'mxbai-embed-large-2', "The sky is blue"],
capture_output=True,
text=True
)
# get the raw text
embedding_text = result.stdout.strip()
# convert the string representation of list into an actual Python list
vector_list = ast.literal_eval(embedding_text)
# convert list of vectors to NumPy array
vector1 = np.array(vector_list)
print("Vector shape:", vector1.shape)
print("First 5 values:", vector1[:5])|
Do this for each of the three phrases:
|
With the vectors of the phrases, you can now use cosine similarity to give a score from -1 to 1 on how similar each phrase is to the others.
# calculate cosine similarity
similarity = np.dot(first_vec, second_vec) / (np.linalg.norm(first_vec) * np.linalg.norm(second_vec))
print("Similarity score:", similarity)|
Cosine similarity ranges from -1 to 1:
|