OpenAI API

What is the OpenAI API?

The OpenAI API gives developers programmatic access to OpenAI models - GPT-4o, GPT-4 Turbo, o1, o3, DALL-E 3, Whisper, TTS, and Embeddings. It is a REST API with client libraries for Python, Node.js, and more.

Think of OpenAI API as the Paytm of AI - it was the first to make powerful AI accessible to everyone through a simple interface, and now it is the default that everyone builds on.

Core API Endpoints:

• Chat Completions - The main endpoint. Send messages, get AI responses. Powers chatbots, code assistants, content generation.
• Assistants API - Managed agents with built-in memory, file handling, code execution, and tool use.
• Embeddings - Convert text to vectors for search, RAG, similarity matching.
• Images (DALL-E) - Generate and edit images from text descriptions.
• Audio (Whisper/TTS) - Speech-to-text and text-to-speech.
• Moderations - Content safety classification. Free to use!

Model Tiers (2025-26):

How Chat Completions Work

You send an array of messages with roles (system, user, assistant) and the model returns a completion. The system message defines behavior, user messages are inputs, and assistant messages are previous AI responses for conversation context.

Key Parameters That Matter:

• temperature (0-2) - Controls randomness. 0 = deterministic, 0.7 = creative, 1.5+ = very random. Use 0 for factual tasks, 0.7-1.0 for creative writing.
• max_tokens - Maximum tokens in the response. Always set this to prevent runaway costs. A typical response is 200-500 tokens.
• top_p - Nucleus sampling. Alternative to temperature. 0.1 = only top 10% probability tokens considered.
• response_format - Force JSON output. Set to { type: "json_object" } for structured responses. Prevents hallucinated format.
• stream - Get tokens as they generate. Essential for chat UIs - user sees response building in real-time.

Function Calling (Tool Use):

The most powerful feature. You define functions with JSON Schema, and the model decides when to call them and with what arguments. The model outputs a structured tool call, your code executes it, and you feed the result back.

• Use Cases - Database queries, API calls, calculations, web search, file operations
• Parallel Tool Calls - GPT-4o can request multiple tool calls in a single response
• Forced Tool Use - Set tool_choice to force the model to use a specific function

Structured Outputs:

New feature that guarantees the response matches a JSON Schema exactly. No more parsing errors or validation failures. The model is constrained during generation to only output valid JSON matching your schema.

What is the Assistants API?

The Assistants API is OpenAI managed infrastructure for building AI agents. Instead of you managing conversation history, file storage, code execution, and tool orchestration, OpenAI handles it all server-side.

Think of Chat Completions as raw ingredients - you cook everything yourself. Assistants API is like ordering a thali from Swiggy - everything is prepared, plated, and delivered ready to eat.

Key Concepts:

• Assistant - A configured AI entity with instructions, model, tools, and file access. Think of it as a specialized employee.
• Thread - A conversation session. Automatically manages message history. No need to resend all messages each time.
• Run - When you ask the assistant to respond. It processes messages, calls tools, generates responses.
• Vector Store - Upload files (PDFs, docs, code) and the assistant can search them using RAG automatically.

Built-in Tools:

• Code Interpreter - Runs Python code in a sandbox. Can analyze data, generate charts, process files. Like having a data scientist on call.
• File Search - RAG built-in. Upload documents, assistant searches them to answer questions. Vector embeddings handled automatically.
• Function Calling - Same as Chat Completions but integrated into the Run lifecycle.

Assistants vs Chat Completions:

Vision (GPT-4o)

GPT-4o natively understands images. You can send images (as URLs or base64) alongside text in the Chat Completions API. The model can describe images, read text from screenshots, analyze charts, understand diagrams, and even solve visual math problems.

• Use Cases - Receipt scanning, UI screenshot analysis, document OCR, product image analysis, medical image triage
• Pricing - Based on image resolution. Low-res (~85 tokens), high-res (~765 tokens per 512x512 tile)
• Tips - Use detail: "low" for simple images to save cost. Use detail: "high" only when fine details matter.

Audio APIs:

• Whisper (Speech-to-Text) - Industry-leading accuracy. Supports 97+ languages. Can transcribe and translate. Rs 0.5/minute approximately.
• TTS (Text-to-Speech) - Natural-sounding voices (alloy, echo, fable, onyx, nova, shimmer). Great for audio content, accessibility, voice assistants.
• Realtime API - WebSocket-based real-time speech conversation. Like talking to AI on a phone call. Low latency, interruption support.

Image Generation (DALL-E 3):

• Generate images from text descriptions
• 1024x1024, 1024x1792, 1792x1024 sizes
• High quality but no fine-grained editing control
• Good for: marketing content, concept art, illustrations

Error Handling & Resilience:

• Retry with Exponential Backoff - Rate limits (429) and server errors (500, 503) are common. Always retry with backoff: 1s, 2s, 4s, 8s.
• Timeout Settings - Set request timeout to 30-60 seconds. Streaming helps - you get tokens incrementally so timeout is less of an issue.
• Fallback Models - If GPT-4o is down/slow, fall back to GPT-4o-mini. If OpenAI is down entirely, fall back to Anthropic Claude.
• Idempotency - Same input should produce consistent results. Use temperature=0 and seed parameter for reproducibility.

Cost Optimization:

• Use the Right Model - GPT-4o-mini for simple tasks (10x cheaper). GPT-4o only when quality demands it. o1 only for complex reasoning.
• Reduce Token Usage - Shorter system prompts, summarize conversation history, use structured outputs to avoid retries.
• Caching - Cache responses for identical/similar queries. Semantic caching can match similar questions.
• Batch API - 50% discount for non-time-sensitive requests. Process thousands of queries at half price with 24-hour turnaround.

Security Considerations:

• API Key Security - Never expose keys in frontend code. Use server-side proxy. Rotate keys regularly.
• Prompt Injection - Users can manipulate the model through crafted inputs. Validate and sanitize user input. Use system prompt guardrails.
• Data Privacy - By default, OpenAI does not train on API data. But verify your data processing agreement for compliance.
• Content Filtering - Use the Moderations API (free!) to screen user input and model output for harmful content.

Q: What is function calling in the OpenAI API and how does it work?

Q: When would you use the Assistants API vs Chat Completions?

Q: How would you handle rate limits and errors in a production OpenAI integration?

Q: How do you optimize OpenAI API costs for a high-volume application?

Q: What are the security risks of using the OpenAI API and how do you mitigate them?