How LLMs Work

Simple Definition:

An LLM is a neural network (specifically, a Transformer) trained on massive amounts of text data to predict the next word (token) in a sequence. That is it. The entire magic of ChatGPT, Claude, and Gemini boils down to incredibly sophisticated next-word prediction.

But this simple objective, when applied at enormous scale (trillions of words, billions of parameters), produces emergent abilities - reasoning, coding, translation, summarization - that the model was never explicitly taught.

Analogy - The Auto-Complete Genius:

Think of your phone's keyboard auto-complete. It predicts the next word based on what you have typed. Now imagine making that prediction engine 1 billion times more powerful - trained on every book, website, and conversation ever written. It can now complete not just words but entire essays, code, and reasoning chains.

That is essentially what an LLM is - the world's most powerful auto-complete, trained on the internet.

What Makes It "Large"?

Scale is the key: These models need thousands of GPUs training for months, costing millions of dollars. That is why only big labs can train frontier models from scratch.

Phase 1: Pre-training (Most Expensive)

The model reads trillions of tokens from the internet - books, Wikipedia, code, websites, forums. For each chunk of text, it predicts the next token, compares with the actual token, and updates its weights.

• Objective: Next-token prediction (autoregressive language modeling)
• Data: Massive web crawls (Common Crawl, C4, The Pile, etc.)
• Cost: Millions of dollars, thousands of GPUs, weeks to months
• Result: A "base model" that can complete text but is not great at following instructions

Analogy: This is like a child reading millions of books. They learn language, facts, and patterns, but they do not yet know how to have a helpful conversation.

Phase 2: Supervised Fine-Tuning (SFT)

Human annotators create thousands of high-quality instruction-response pairs. The model learns to follow instructions by training on these examples.

• Objective: Learn to follow instructions and be helpful
• Data: 10K-100K curated prompt-response pairs written by experts
• Cost: Much cheaper than pre-training, days on fewer GPUs
• Result: An "instruct model" that follows instructions but may sometimes be harmful or unhelpful

Analogy: Like taking a well-read person and teaching them customer service skills - how to understand questions and give helpful answers.

Phase 3: RLHF / DPO Alignment

Human raters compare multiple model responses and rank them. A reward model learns these preferences. The LLM is then optimized to maximize the reward model's score.

• RLHF: Reinforcement Learning from Human Feedback - uses PPO algorithm
• DPO: Direct Preference Optimization - newer, simpler alternative to RLHF
• Goal: Make the model helpful, harmless, and honest
• Result: The "chat model" you actually interact with (ChatGPT, Claude)

Analogy: Like a manager reviewing the customer service agent's performance, telling them "this response was great, this one was bad" until quality consistently improves.

Step-by-Step Generation:

When you type a prompt, here is what happens inside the LLM:

1. 1. Tokenize: Your text is split into tokens (subwords). "Artificial Intelligence" becomes ["Art", "ificial", " Intell", "igence"]
2. 2. Embed: Each token is converted to a high-dimensional vector (numbers)
3. 3. Process: Vectors pass through dozens of Transformer layers (attention + feedforward)
4. 4. Predict: The final layer outputs a probability distribution over ALL possible next tokens (50,000+ options)
5. 5. Sample: One token is selected based on the probabilities (controlled by temperature)
6. 6. Repeat: The selected token is appended to the input, and steps 2-5 repeat for the next token

Temperature - Controlling Randomness:

After the model outputs probabilities, temperature controls how tokens are selected:

• Temperature 0: Always picks the highest-probability token. Deterministic, factual, but repetitive
• Temperature 0.7: Good balance of quality and creativity. Most common setting
• Temperature 1.0: Full sampling from the distribution. More creative but may produce errors
• Temperature >1.0: Amplifies randomness. Can produce nonsensical output

Analogy: Temperature 0 is like a strict teacher who always gives the "textbook" answer. Temperature 1 is like a creative writer who takes risks. Temperature 2 is like someone having a fever dream.

Top-p and Top-k Sampling:

• Top-k: Only consider the top k most likely tokens. k=50 means choose from the 50 most probable tokens
• Top-p (nucleus): Only consider tokens until cumulative probability reaches p. p=0.9 means consider enough tokens to cover 90% probability
• Why both? They prevent the model from selecting extremely unlikely tokens while still allowing creativity

Emergent Abilities - The Magic of Scale:

As models get larger, they suddenly gain abilities that smaller models do not have. These are called emergent abilities:

• Few-shot learning: Given a few examples in the prompt, the model generalizes to new inputs
• Chain-of-thought reasoning: Can solve complex problems by thinking step-by-step
• Code generation: Writes functional code in dozens of languages
• Translation: Translates between languages it was never explicitly taught to translate
• Tool use: Learns to call APIs and use external tools from instructions alone

These abilities were not programmed. They emerged from training on enough data at enough scale.

Fundamental Limitations:

• Hallucination: Models confidently generate false information. They optimize for plausibility, not truth
• Knowledge Cutoff: Training data has a date. The model does not know about events after that date
• No Real Understanding: It is pattern matching, not comprehension. It can write about swimming without knowing what water feels like
• Context Window Limit: Cannot process infinitely long text. Information beyond the window is lost
• Poor Math: Arithmetic is unreliable for large numbers. Tokenization makes math hard for LLMs
• Brittleness: Slight prompt changes can dramatically alter output quality
• Bias: Training data contains biases that the model reproduces

Why Hallucinations Happen - A Technical Perspective:

LLMs are trained to produce plausible-sounding text, not factually correct text. During training, the model learns that "The capital of France is Paris" and "The CEO of Tesla is Elon Musk" are patterns. But it also learns patterns like "[Famous person] wrote [Book title]" - and may combine these patterns incorrectly.

The model has no internal fact-checker. It generates the most likely next token based on the input context and its training, even if that leads to factual errors.

Data Collection & Cleaning:

• Sources: Common Crawl (web pages), Wikipedia, Books, GitHub code, ArXiv papers, Reddit, StackOverflow
• Cleaning: Remove duplicates, low-quality text, toxic content, personal information
• Filtering: Quality classifiers to keep only high-quality text
• Scale: Typically 1-15 trillion tokens for a frontier model

Analogy: Like Zomato collecting reviews - you want millions of reviews, but you need to filter out spam, duplicate content, and inappropriate language.

Training Infrastructure:

• Hardware: Thousands of NVIDIA A100 or H100 GPUs (each costs $25,000-$40,000)
• Networking: InfiniBand connections between GPUs at 400 Gbps
• Duration: Weeks to months of continuous training
• Cost: GPT-4 estimated at $100 million+. LLaMA 3 405B cost ~$30 million
• Power: Training GPT-4 used estimated 50 GWh of electricity

Post-Training Pipeline:

1. SFT Data Creation: Hire hundreds of expert annotators to write instruction-response pairs
2. Reward Model Training: Annotators compare model responses, building a preference dataset
3. RLHF/DPO: Optimize the model against the reward model
4. Red-teaming: Dedicated team tries to break the model and find harmful outputs
5. Safety Evaluation: Benchmark against safety criteria before release

Scaling Laws - The Key Insight:

Research by Kaplan et al. (2020) showed that model performance improves predictably with three factors:

• More parameters: Bigger models learn better representations
• More data: More diverse training data reduces errors
• More compute: Longer training with more GPUs

These "scaling laws" are why companies keep building bigger models. The performance gains are mathematically predictable!

Q: How does an LLM generate text?

Q: What are the three phases of LLM training?

Q: Why do LLMs hallucinate?

Q: What is the difference between a base model and a chat model?

Q: What does temperature control in LLM generation?