Semantic Kernel (Microsoft AI Framework)

Simple Definition:

Semantic Kernel (SK) is Microsoft's open-source SDK that helps you integrate LLMs into your existing applications. Instead of rewriting your entire app around AI, SK lets you add AI capabilities to your existing codebase through a plugin system.

Think of it like adding a smart assistant to your existing office. The assistant (LLM) can use all your existing tools (plugins) - your calculator, your database, your email system. Semantic Kernel is the bridge that connects the assistant to your tools.

Real-World Analogy - IRCTC with AI:

Imagine adding AI to IRCTC. You do not rewrite the entire booking system. Instead:

• Existing code (Plugins): Train search API, seat availability checker, payment gateway, PNR status checker
• AI Layer (Semantic Kernel): Connects user's natural language request to the right plugin
• User says: "Book me a Rajdhani from Delhi to Mumbai for next Friday, window seat, vegetarian meal"
• SK orchestrates: Parses intent -> Calls train search -> Checks availability -> Applies preferences -> Initiates booking

SK vs LangChain - Key Differences:

1. The Kernel (Brain):

The Kernel is the central orchestrator. It connects your LLM, plugins, memory, and configuration. Think of it as the operating system for your AI application - everything runs through the kernel.

• Manages LLM connections (OpenAI, Azure OpenAI, local models)
• Loads and manages plugins
• Handles memory and context
• Provides the execution pipeline

2. Plugins (Hands):

Plugins are collections of functions that the AI can use. There are two types:

• Semantic Functions: LLM-powered functions defined by a prompt template. Example: "Summarize this text", "Translate to Hindi", "Generate email reply".
• Native Functions: Regular code functions (C#/Python/Java) that do deterministic operations. Example: database queries, API calls, calculations, file operations.

The magic is that the LLM can choose which plugin to use based on the user's request. It sees all available plugins and their descriptions, then decides the best one to call.

3. Planners (Brain + Decision Making):

Planners are the AI's ability to create a step-by-step plan using available plugins. Given a user goal, the planner figures out which plugins to call and in what order.

User Goal: "Send a birthday email to Priya with a personalized poem"

Planner creates this plan:
  Step 1: Call ContactPlugin.GetEmail("Priya") -> priya@email.com
  Step 2: Call ContactPlugin.GetBirthday("Priya") -> "March 15"
  Step 3: Call WriterPlugin.WritePoem(topic="birthday", person="Priya")
  Step 4: Call EmailPlugin.SendEmail(to=priya@email.com,
           subject="Happy Birthday Priya!", body=poem)

How They Work Together:

User Request
    |
    v
[Kernel] -- connects to --> [LLM (OpenAI/Azure)]
    |                              |
    |-- loads --> [Plugins]        |-- uses AI to -->
    |   |-- Semantic (prompts)     |   choose plugins
    |   |-- Native (code)          |   and create plans
    |                              |
    |-- uses --> [Memory]          |
    |   |-- Short-term (chat)      |
    |   |-- Long-term (vectors)    |
    |                              |
    v                              v
[Planner] creates execution plan using plugins
    |
    v
[Execute plan step by step]
    |
    v
Final Response to User

Semantic Functions (Prompt-Based):

Semantic functions are essentially prompt templates that the LLM executes. You define them with a prompt template and configuration (model, temperature, max tokens).

Plugin: WriterPlugin
Function: Summarize

Prompt Template:
  "Summarize the following text in 3 bullet points.
   Keep it concise and highlight key insights.
   
   Text: {{input}}
   
   Summary:"

Config:
  model: gpt-4
  temperature: 0.3
  max_tokens: 200

These are great for tasks that need LLM intelligence: summarization, translation, content generation, sentiment analysis, etc.

Native Functions (Code-Based):

Native functions are regular code that does deterministic operations. The LLM cannot do math reliably, but a native function can. It cannot query your database, but a native function can.

Plugin: MathPlugin
Function: CalculateEMI

// This is regular Python/C# code, not a prompt
def calculate_emi(principal, rate, tenure_months):
    monthly_rate = rate / 12 / 100
    emi = principal * monthly_rate * (1 + monthly_rate)**tenure_months
    emi = emi / ((1 + monthly_rate)**tenure_months - 1)
    return round(emi, 2)

// LLM calls this when user asks:
// "What is the EMI for a Rs 50 lakh home loan at 8.5% for 20 years?"
// Result: Rs 43,391

When to Use Which:

Why Memory Matters:

Without memory, your AI is like a person with amnesia - every conversation starts from scratch. SK provides a memory system that lets your AI remember previous conversations, user preferences, and important facts.

Types of Memory in SK:

• Chat History (Short-term): The current conversation. Automatically managed by SK. Like your working memory - what you are thinking about right now.
• Semantic Memory (Long-term): Facts and knowledge stored as vector embeddings. Like your long-term memory - things you learned and can recall. Powered by vector databases (Azure AI Search, Pinecone, Qdrant).
• Entity Memory: Structured information about specific entities (users, products, companies). Like your memory of specific people and things.

Example - Travel Agent with Memory:

Conversation 1 (January):
User: "I am vegetarian and prefer window seats"
-> Stored in Entity Memory: user_preferences

Conversation 2 (March):
User: "Book me a flight to Goa next week"

AI recalls from memory:
  - User is vegetarian -> auto-select veg meal
  - User prefers window -> auto-select window seat
  - Previous trips: User visited Goa in Dec 2024
    -> "I see you visited Goa recently! Want the same hotel?"

Memory Best Practices:

• Be selective: Do not store everything. Only store information that will be useful later.
• Privacy first: Be careful about what personal data you store. Follow GDPR/data protection rules.
• Relevance decay: Old memories may become irrelevant. Implement a strategy to age out stale data.
• Context window management: Do not inject too much memory into the prompt. Select only the most relevant memories.

Typical SK Application Architecture:

[Frontend (React/Angular)]
        |
        v
[API Layer (ASP.NET / FastAPI)]
        |
        v
[Semantic Kernel]
  |-- Kernel Config (LLM settings, API keys)
  |-- Plugins
  |     |-- OrderPlugin (native: query orders DB)
  |     |-- ProductPlugin (native: search products)
  |     |-- WriterPlugin (semantic: generate descriptions)
  |     |-- TranslatePlugin (semantic: multi-language)
  |-- Memory
  |     |-- Chat History (Redis)
  |     |-- Semantic Memory (Azure AI Search)
  |-- Planner (auto-orchestrates plugins)
        |
        v
[External Services]
  |-- Azure OpenAI (LLM)
  |-- SQL Database (products, orders)
  |-- Redis (session, cache)
  |-- Azure AI Search (vectors)

Best Use Cases for Semantic Kernel:

• Enterprise Copilots: Add AI assistant to existing enterprise software (.NET/Java). Microsoft 365 Copilot is built on SK.
• Customer Support Bots: Connect LLM to your existing support tools and knowledge base.
• Internal Tools: Build AI-powered tools that query internal databases and APIs using natural language.
• Document Processing: Summarize, extract, translate, and analyze documents at scale.

When NOT to Use Semantic Kernel:

• Simple chatbots: If you just need a chat interface, direct API calls to OpenAI are simpler.
• Python-only projects: LangChain has a larger Python ecosystem. SK's strength is C#/.NET.
• Experimental/research: LangChain and LlamaIndex have more cutting-edge experimental features.
• Non-Microsoft stack: SK integrates best with Azure and Microsoft services.

Q: What is Semantic Kernel and how does it differ from LangChain?

Q: What are semantic functions vs native functions in SK?

Q: What is the role of the Planner in Semantic Kernel?

Q: When would you choose Semantic Kernel over LangChain?