Haystack (End-to-end RAG Framework)

Core Philosophy:

Haystack (by deepset) is an open-source framework for building composable AI pipelines. Unlike LangChain which prioritizes flexibility, Haystack prioritizes reliability and type safety. Every component has clearly defined inputs and outputs, and pipelines are validated at construction time - not at runtime.

Think of it as the difference between a JavaScript project (flexible, anything goes) and a TypeScript project (strict, catches errors early). Haystack is the TypeScript of RAG frameworks.

Real-World Analogy - LEGO vs Jigsaw Puzzle:

• LangChain = LEGO: Flexible, can build anything, but pieces do not always fit together perfectly. Easy to prototype, harder to make production-grade.
• Haystack = Jigsaw Puzzle: Each piece has a specific shape (typed I/O). When pieces connect, they fit perfectly. Harder to start, but the final picture is always clean and reliable.

Key Features of Haystack 2.x:

• Component System: Every building block is a Component with typed inputs/outputs
• Pipeline Graph: Components are connected in a DAG (Directed Acyclic Graph)
• Validation: Pipeline validates connections at build time
• Serialization: Pipelines can be saved/loaded as YAML
• Provider Agnostic: Switch between OpenAI, Anthropic, Hugging Face with one line

Haystack vs LangChain vs LlamaIndex:

Component Architecture:

A Haystack Component is a Python class with a run() method that has typed inputs and outputs. The @component decorator registers it with the framework. This is the fundamental building block - retrievers, generators, rankers, converters are all components.

Built-in Component Categories:

• Converters: Turn raw files (PDF, HTML, DOCX) into Haystack Documents. Examples: PyPDFToDocument, HTMLToDocument, MarkdownToDocument
• PreProcessors: Split documents into chunks. DocumentCleaner removes noise, DocumentSplitter handles chunking with overlap.
• Embedders: Generate embeddings. SentenceTransformersTextEmbedder, OpenAITextEmbedder, etc.
• Retrievers: Fetch relevant documents. InMemoryEmbeddingRetriever, ElasticsearchBM25Retriever, QdrantEmbeddingRetriever.
• Rankers: Re-rank retrieved results. TransformersSimilarityRanker, CohereRanker.
• Generators: Generate text via LLMs. OpenAIGenerator, AnthropicGenerator, HuggingFaceLocalGenerator.
• Builders: Construct prompts. PromptBuilder uses Jinja2 templates for dynamic prompt construction.

Custom Components:

Writing your own component is straightforward. Decorate a class with @component, define input_type and output_type, implement run(). Haystack validates everything at pipeline construction time.

This is powerful for domain-specific logic: a PAN number validator, a Hindi-to-English translator, or a custom re-ranker trained on your data.

Pipeline Concept:

A Haystack Pipeline is a directed graph of connected components. You add components, connect their outputs to inputs, and run the pipeline. The framework handles data flow, parallel execution where possible, and error propagation.

Two Types of Pipelines:

• Indexing Pipeline: Takes raw documents, processes them, embeds them, and stores in a document store. Run once or on a schedule. Components: Converter -> Cleaner -> Splitter -> Embedder -> DocumentWriter.
• Query Pipeline: Takes a user query, retrieves relevant documents, and generates an answer. Run on every user request. Components: Embedder -> Retriever -> PromptBuilder -> Generator.

Pipeline Features:

• Branching: Route queries to different paths based on conditions (Router component)
• Loops: Implement self-correcting pipelines that retry on failure
• Serialization: Save entire pipeline as YAML, version control it, load in production
• Visualization: Generate a visual graph of your pipeline for documentation

YAML Serialization - Why It Matters:

Haystack pipelines can be fully serialized to YAML. This means:

• Version control your pipelines like code
• Deploy different pipeline configs for dev/staging/prod
• Non-developers can modify pipeline parameters
• A/B test different pipeline configurations easily

Indexing Pipeline Design:

1. PyPDFToDocument: Convert PDFs to Document objects with metadata
2. DocumentCleaner: Remove headers, footers, watermarks
3. DocumentSplitter: Split into chunks (split_by: "word", split_length: 200, split_overlap: 20)
4. SentenceTransformersDocumentEmbedder: Generate embeddings
5. DocumentWriter: Store in Qdrant/Weaviate/Elasticsearch

Query Pipeline Design:

1. SentenceTransformersTextEmbedder: Embed user query
2. QdrantEmbeddingRetriever: Retrieve top-10 chunks
3. TransformersSimilarityRanker: Re-rank to top-5
4. PromptBuilder: Combine context + question into prompt
5. OpenAIGenerator: Generate answer with GPT-4

Advanced Patterns:

• Hybrid Retrieval: Use both embedding retriever AND BM25 retriever, join results with DocumentJoiner
• Fallback: If retrieval returns no results, route to a web search component
• Multi-Query: Generate 3 query variations, retrieve for each, merge results
• Guardrails: Add a content filter component before returning to user

Document Stores Supported:

• InMemory: For development and testing
• Elasticsearch: Hybrid search, production-ready
• Qdrant: Vector-native, high performance
• Weaviate: Native hybrid, multi-tenancy
• Pinecone: Fully managed vector DB
• ChromaDB: Lightweight, open-source

Choose Haystack When:

• Production is priority: You need reliability, type safety, and clear error messages over rapid prototyping
• Pipeline complexity: You are building multi-step pipelines with branching, loops, or conditional logic
• Team collaboration: Multiple developers working on the same pipeline - typed components prevent integration bugs
• MLOps integration: YAML serialization fits naturally into CI/CD and configuration management
• Document processing: Heavy document ingestion with PDFs, HTML, and various formats

Avoid Haystack When:

• Quick prototyping: LangChain is faster for hackathons and POCs
• Agent-heavy workflows: LangGraph is better for complex stateful agents
• Tiny projects: For a simple Q&A chatbot, raw OpenAI API + a vector DB is simpler
• Ecosystem lock-in concerns: LangChain has the largest ecosystem of integrations

Common Migration Path:

Many teams prototype in LangChain, then migrate to Haystack for production. The key reason: LangChain abstractions sometimes make debugging hard, while Haystack components are transparent and predictable.

Q: What makes Haystack different from LangChain?

Q: Explain the difference between an Indexing Pipeline and a Query Pipeline in Haystack.

Q: Why is YAML pipeline serialization useful in production?