LlamaIndex (Data Agents & Query Engines)

Core Mission:

LlamaIndex (formerly GPT Index) is a data framework that helps you ingest, structure, and query your data using LLMs. While LangChain is a Swiss Army knife for all LLM applications, LlamaIndex laser-focuses on the data connection problem - getting your private data into LLMs effectively.

Think of it this way: If you are building a customer support bot over your documentation, LlamaIndex handles the "over your documentation" part brilliantly.

Real-World Analogy - JioMart Warehouse:

Imagine JioMart building an AI assistant for their warehouse:

• Data Connectors = Receiving Dock: Products arrive from different sources (PDFs, databases, APIs, Slack). LlamaIndex ingests them all.
• Indices = Organized Shelves: Products are sorted and indexed so you can find anything quickly. Vector index, keyword index, tree index.
• Query Engines = Smart Staff: When you ask "Where is the organic dal?", the engine knows which shelf to check and how to answer.
• Data Agents = Manager: Can coordinate across multiple indices, use tools, and handle complex multi-step requests.

LlamaIndex Core Components:

• Data Connectors (Readers): Ingest data from 160+ sources
• Documents & Nodes: Data representations (documents split into nodes/chunks)
• Indices: Data structures optimized for different query types
• Query Engines: Interface for asking questions over your data
• Chat Engines: Conversational interface with memory
• Data Agents: Autonomous agents that can query multiple data sources

LlamaIndex vs LangChain:

What are Indices?

An Index in LlamaIndex is a data structure built over your documents that enables efficient querying. Different index types are optimized for different query patterns - just like how a database needs different indices for different query types.

Index Types Explained:

• VectorStoreIndex: The most common. Embeds all chunks and uses vector similarity for retrieval. Best for semantic similarity queries. "What is our refund policy?"
• SummaryIndex (formerly ListIndex): Stores all nodes sequentially. Queries go through all nodes (or a summary). Best for summarization tasks. "Summarize all customer complaints."
• TreeIndex: Builds a tree of summaries bottom-up. Leaf nodes are original chunks, parent nodes are summaries of children. Best for hierarchical queries. Efficient for large document sets.
• KeywordTableIndex: Extracts keywords from each node. Routes queries to nodes matching keywords. Best for keyword-specific queries. Faster than vector search for exact lookups.
• KnowledgeGraphIndex: Builds a knowledge graph from your documents. Best for relationship queries and multi-hop reasoning.

Composable Indices - The Power Move:

LlamaIndex lets you compose indices - build an index over other indices. This is incredibly powerful:

• Index 1: VectorIndex over technical documentation
• Index 2: VectorIndex over customer support tickets
• Index 3: KeywordIndex over product catalog
• Top-Level: A router that directs queries to the right sub-index

The query engine automatically routes "How do I configure SSL?" to the technical docs index and "What is the price of Widget X?" to the product catalog.

Query Engine:

A Query Engine is the interface that takes a natural language question, retrieves relevant data from an index, and generates an answer. It is the core RAG abstraction in LlamaIndex.

Simple usage: index.as_query_engine() gives you a ready-to-use query engine. But LlamaIndex also offers powerful customization options.

Query Engine Types:

• RetrieverQueryEngine: Standard RAG - retrieve then generate. Most common.
• CitationQueryEngine: Generates answers with source citations. Essential for trust.
• SubQuestionQueryEngine: Breaks complex questions into sub-questions, queries different sources for each, then combines answers. Incredibly powerful for multi-source queries.
• RouterQueryEngine: Routes queries to different query engines based on the question type. Like a smart receptionist directing you to the right department.

Chat Engine - Conversational RAG:

A Chat Engine adds conversation memory on top of a query engine. It tracks message history and provides context-aware responses across turns.

Chat modes:

• condense_question: Rewrites user question to be standalone based on chat history. Then queries normally.
• context: Retrieves context for every message and includes chat history.
• react: Uses ReAct agent under the hood. Can decide whether to query the index or respond from memory.

SubQuestionQueryEngine Example:

User: "Compare TCS and Infosys annual revenue and employee count for 2025."

1. Sub Q1: "What is TCS annual revenue 2025?" -> TCS index
2. Sub Q2: "What is TCS employee count 2025?" -> TCS index
3. Sub Q3: "What is Infosys annual revenue 2025?" -> Infosys index
4. Sub Q4: "What is Infosys employee count 2025?" -> Infosys index
5. Final: Combines all sub-answers into a comparison table

Data Connectors (Readers):

LlamaIndex has 160+ data connectors via LlamaHub - an open-source repository of data loaders. From simple file readers to complex API integrations, if your data lives somewhere, there is probably a connector for it.

Popular Connectors:

• Files: PDF, DOCX, CSV, JSON, HTML, Markdown, EPUB
• Databases: PostgreSQL, MySQL, MongoDB, Snowflake
• Cloud: Google Drive, Notion, Confluence, Slack, Discord
• Web: Web scraper, Sitemap reader, RSS feed
• Code: GitHub repos, code files with language-aware splitting

LlamaParse - Premium Document Parsing:

LlamaParse is LlamaIndex's proprietary document parsing service. It uses AI to parse complex documents (PDFs with tables, charts, multi-column layouts) far better than traditional parsers.

• Table extraction: Handles merged cells, multi-row headers, nested tables
• Layout understanding: Multi-column, side-by-side content, footnotes
• Instruction-based: You can give parsing instructions like "Extract all financial tables as markdown"
• Multimodal: Understands images and charts within documents

Ingestion Pipeline:

LlamaIndex provides an IngestionPipeline that chains transformations:

1. Reader loads raw documents
2. NodeParser splits into chunks (SentenceSplitter, SemanticSplitter)
3. Metadata extractors add titles, summaries, keywords
4. Embeddings are generated
5. Nodes are stored in the vector store

The SemanticSplitter is noteworthy - it splits based on semantic similarity rather than fixed character count, producing more meaningful chunks.

LlamaIndex Strengths:

• Best-in-class data ingestion: 160+ connectors, LlamaParse for complex docs
• Advanced RAG built-in: SubQuestion, Router, Citation engines out of the box
• Semantic chunking: SemanticSplitter produces better chunks than fixed-size
• Index composition: Build complex multi-source query systems easily
• Evaluation: Built-in evaluation tools for retrieval and generation

LlamaIndex Weaknesses:

• Agent capabilities: Not as strong as LangGraph for complex agent workflows
• General tooling: Focused on data/RAG - not ideal for general LLM apps that do not involve data retrieval
• API stability: Like LangChain, APIs change frequently between versions

Decision Guide:

Q: How does LlamaIndex differ from LangChain in approach?

Q: What is the SubQuestionQueryEngine and when would you use it?

Q: What is LlamaParse and why is it significant for RAG applications?