AWS Bedrock & Azure OpenAI

The Enterprise Problem

Large companies cannot just use OpenAI API directly. They need data governance (where is data processed?), compliance (GDPR, HIPAA, SOC2), network security (VPC, private endpoints), access control (IAM, audit logs), and existing billing (consolidated cloud bills, not another vendor).

Think of it like this: a chai tapri can buy ingredients directly from the market (direct API). But Starbucks India needs supply chain management, food safety compliance, centralized procurement, and audit trails. AWS Bedrock and Azure OpenAI are the "enterprise supply chain" for AI.

Why Not Use Direct APIs?

• Data Residency - Indian banking regulations may require data to stay in India. Direct OpenAI API sends data to US servers.
• Network Security - Enterprise data should not traverse the public internet. Need private endpoints within VPC.
• Compliance - HIPAA for healthcare, PCI for payments, RBI guidelines for banking. Cloud providers have these certifications.
• Procurement - Large orgs have existing AWS/Azure contracts. Adding another vendor (OpenAI) takes months of legal review.
• Audit Trail - Every AI request must be logged with user identity, timestamp, and data accessed. Cloud IAM provides this.

The Two Giants:

• AWS Bedrock - Multi-model platform. Access Claude, Llama, Mistral, Cohere, Stability, and Amazon models through AWS.
• Azure OpenAI - Exclusive OpenAI models (GPT-4, DALL-E, Whisper) deployed on Azure infrastructure with enterprise features.

What is AWS Bedrock?

Bedrock is AWS managed service for accessing foundation models from multiple providers through a unified API. You get Claude (Anthropic), Llama (Meta), Mistral, Cohere, Stability AI, and Amazon Titan - all through your AWS account with AWS security, billing, and governance.

Available Models on Bedrock:

Bedrock Key Features:

• Knowledge Bases - Managed RAG. Connect S3 documents, automatic chunking, vector storage, retrieval. No infrastructure to manage.
• Agents - Build AI agents with tool use, memory, and multi-step reasoning. Integrated with AWS services (Lambda, S3, DynamoDB).
• Guardrails - Content filtering, PII detection, topic blocking. Configurable per use case.
• Fine-tuning - Fine-tune supported models with your data. Data stays in your AWS account.
• Model Evaluation - Built-in evaluation tools to compare model quality on your specific tasks.

Bedrock Security:

• VPC Endpoints - AI requests never leave your VPC. No public internet.
• IAM - Fine-grained access control. Which users/roles can use which models.
• CloudTrail - Every API call logged for audit.
• KMS Encryption - Data encrypted at rest and in transit with your keys.
• No Data Training - Your data is never used to train the models.

What is Azure OpenAI?

Azure OpenAI Service gives you dedicated deployments of OpenAI models on Azure infrastructure. Same GPT-4, DALL-E, Whisper models but running on Azure with enterprise security, compliance, and networking.

Key difference from direct OpenAI: you get a dedicated model deployment in a specific Azure region. Your data is processed in that region only. No shared infrastructure with other customers.

Azure OpenAI Key Features:

• Dedicated Deployments - Create named deployments of specific model versions. Pin versions to prevent unexpected model changes.
• Provisioned Throughput - Reserve capacity (PTUs - Provisioned Throughput Units) for guaranteed performance. No rate limit surprises.
• Content Filtering - Azure provides content safety filters on top of OpenAI models. Configurable severity levels.
• Fine-tuning - Fine-tune GPT-3.5 and GPT-4 models with your data on Azure infrastructure.
• On Your Data - Built-in RAG with Azure AI Search. Connect your enterprise data sources without building RAG yourself.

Provisioned Throughput (PTU) Explained:

Normal Azure OpenAI uses pay-per-token pricing with shared capacity (rate limits apply). PTU gives you reserved capacity - guaranteed tokens per minute regardless of platform load.

• Standard (Pay-per-token) - Cheaper for low/variable usage. Subject to rate limits.
• Provisioned (PTU) - Reserved capacity. Higher cost but guaranteed throughput. No rate limits within your allocation.
• When to use PTU - Production workloads with consistent volume. Cost-effective at ~30% utilization or higher.

Head-to-Head Comparison:

Choose AWS Bedrock When:

• You want model diversity (not locked to one provider)
• You are already on AWS (most Indian startups and enterprises)
• You want Claude (not available on Azure natively)
• You need Guardrails and Knowledge Bases as managed services

Choose Azure OpenAI When:

• You specifically need GPT-4/OpenAI models with enterprise governance
• You are already on Azure (many Indian banks and large enterprises)
• You need Provisioned Throughput for guaranteed capacity
• You use Microsoft 365 / Copilot ecosystem

Indian Enterprise Context:

• Banking/BFSI - Most Indian banks use Azure. Azure OpenAI is natural choice. RBI data localization requirements met with India regions.
• Startups - Most use AWS. Bedrock with Claude is the go-to choice.
• Government - Depends on existing contracts. Both have India regions.
• Healthcare - HIPAA equivalent compliance needed. Both platforms offer it.

Pitfall 1: Ignoring Quota/Capacity Planning

Both platforms have request quotas and TPM (tokens per minute) limits that are much lower than direct API access by default. You must request quota increases well before production launch. Azure PTU needs to be provisioned in advance - you cannot get capacity instantly.

Pitfall 2: Model Version Pinning

Cloud providers update model versions. A model that works perfectly today might behave differently next month. Always pin specific model versions (e.g., gpt-4-0613, not just gpt-4) and test before upgrading.

Cost Optimization Tips:

• Right-size your deployment - Do not provision PTU capacity you do not need. Start with pay-per-token, move to PTU when usage is consistent.
• Use model tiers - Haiku/GPT-4o-mini for simple tasks, larger models only for complex ones.
• Cache responses - Implement semantic caching to avoid repeated identical calls.
• Batch where possible - Both platforms offer batch processing at lower costs.

Migration Path:

Starting with direct OpenAI/Anthropic API and want to move to enterprise platform?

• Azure OpenAI - API is almost identical to OpenAI. Change base URL and API key. Minimal code changes.
• AWS Bedrock - Uses AWS SDK (boto3). Different API format from direct Anthropic. More code changes needed. Use LiteLLM to abstract this.

Q: Why would an enterprise use AWS Bedrock instead of calling Anthropic API directly?

Q: What is the difference between AWS Bedrock and Azure OpenAI?

Q: What is Provisioned Throughput (PTU) in Azure OpenAI?

Q: How would you handle data residency requirements for AI in Indian banking?

Q: What are common mistakes when deploying AI on enterprise cloud platforms?