Overview
The Embedding Generator transform produces vector embeddings from text columns using a connected AI/LLM provider. Embeddings convert human-readable text into dense numeric vectors that capture semantic meaning, enabling similarity search, clustering, deduplication, and ML feature engineering. Available on Professional plans and above.When to Use
- Semantic search: Store embeddings in a vector database for similarity queries
- Deduplication: Find semantically similar records even with different wording
- Clustering: Group similar text records (support tickets, product descriptions)
- Recommendation: Find similar products, articles, or customers
- Classification: Use embeddings as ML input features
Configuration
Supported Models
The model list depends on your connected AI provider. Connect an AI provider first, then select the model in the node configuration.
How It Works
Each text value is sent to the configured embedding API. The response vector is stored in the output column as a numeric array.Example
Input
Output
Products 1 and 2 have similar embedding vectors (both are headphones) while product 3 is distant.
Pipeline Patterns
Semantic Search Pipeline
Deduplication Pipeline
ML Feature Engineering
Cost and Performance
Tips
- Store embeddings efficiently: Use a column type that supports arrays or JSON; for vector databases, use their native vector type
- Chunk long text: If text exceeds the model’s token limit, split into overlapping chunks and embed each separately
- Normalize vectors: Some similarity search engines expect unit-normalized vectors — check your destination’s requirements
- Version your model: Embedding vectors from different models are not comparable — document which model generated each embedding
Related
AI Provider Connections
Connect OpenAI, Google AI, or Anthropic
LLM Transform
Generate text with prompts using LLM models
Advanced Nodes
All advanced transform nodes
PII Detection
Protect sensitive data before sending to external APIs