Home/Agentic AI/Vector Databases/Embeddings Fundamentals

Vector Databases for Memory

Master how AI agents use vector databases to store, search, and retrieve embeddings for semantic memory

Your Progress

0 / 5 completed
1
2
3
4
5

Understanding Embeddings

An embedding is a dense numerical representation of text, images, or other data in a high-dimensional space. Words or sentences with similar meanings have similar embeddings.

For example: "cat" and "kitten" produce embeddings that are close together in vector space, while "cat" and "computer" are far apart.

Interactive: Text → Embedding Converter

Select different texts to see how they become numerical vectors. Similar concepts cluster together!

Input Text:
"The cat sat on the mat"
Embedding Vector (3D visualization):
[0.80, 0.30, 0.10]
Simplified 3D representation
Key Insight: Animal-related texts (cat, dog) have similar vectors in first two dimensions. Tech texts (Python, ML) cluster in the third dimension. Real models use hundreds of dimensions!

📊 Key Properties of Embeddings

📏High-Dimensional

Typically 384-1536 dimensions (OpenAI: 1536, Cohere: 768)

More dimensions = more nuance and semantic richness captured

🎯Dense Vectors

Every dimension contains meaningful information (not sparse)

Unlike one-hot encoding, all values contribute to meaning

🧲Semantic Proximity

Similar meanings → close vectors in space

Distance/similarity metrics quantify semantic relatedness

🔒Model-Specific

Embeddings from different models are incompatible

Must use the same model for encoding queries and data

🤖 Popular Embedding Models

OpenAI text-embedding-3-small/large

API

High quality, 1536 dimensions, excellent for general-purpose semantic search

Use case: RAG, semantic memory, production systems

sentence-transformers/all-MiniLM-L6-v2

Open Source

384 dimensions, fast inference, runs locally without API costs

Use case: Local development, prototypes, cost-sensitive apps

Cohere embed-english-v3.0

API

1024 dimensions, optimized for retrieval and clustering tasks

Use case: E-commerce search, document clustering

💻 Creating Embeddings (Code Example)

# Using OpenAI (Python)
from openai import OpenAI
client = OpenAI()

text = "The cat sat on the mat"
response = client.embeddings.create(
    model="text-embedding-3-small",
    input=text
)

embedding = response.data[0].embedding
# embedding is a list of 1536 floats
print(f"Dimensions: {len(embedding)}")
print(f"First 5 values: {embedding[:5]}")
# Using sentence-transformers (Local)
from sentence_transformers import SentenceTransformer

model = SentenceTransformer('all-MiniLM-L6-v2')
text = "The cat sat on the mat"
embedding = model.encode(text)

# embedding is a numpy array of 384 floats
print(f"Dimensions: {embedding.shape}")
print(f"First 5 values: {embedding[:5]}")
← Prev