AutoGineer
Generated banner for Azure Aigateway
Generated logo for Azure Aigateway

Azure Aigateway

MicrosoftOfficialCreated by Microsoft

  • Skill
  • Tool integration
  • Cloud and DevOps
  • MIT
  • Low risk
Download ZIP

From the author

Configure Azure API Management as an AI Gateway for AI models, MCP tools, and agents. WHEN: semantic caching, token limit, content safety, load balancing, AI model governance, MCP rate limiting, jailbreak detection, add Azure OpenAI backend, add AI Foundry model, test AI gateway, LLM policies, configure AI backend, token metrics, AI cost control, convert API to MCP, import OpenAPI to gateway.
— the description by Microsoft

AutoGineer summary

Written by AutoGineer from the skill’s own files, not by the author

This skill provides the configuration patterns and policy references needed to transform Azure API Management (APIM) into a robust AI Gateway. It enables centralized governance for AI models, including Azure OpenAI and AI Foundry, by implementing essential features like semantic caching, token usage tracking, load balancing, and content safety filtering. The skill covers the full lifecycle from backend setup and managed identity authentication to advanced policy application and troubleshooting.

What it does

  • Connects AI models as backends within Azure API Management.
  • Implements cost control through token limits and semantic caching.
  • Enforces security policies like jailbreak detection and content safety.
  • Enables load balancing across multiple AI backend instances.
  • Provides observability by emitting token usage metrics.
  • Facilitates MCP tool protection and rate limiting.

When to use it

  • When you need to govern and secure access to AI models in production.
  • When you want to implement cost controls for LLM usage.
  • When you need to load balance requests across multiple AI regions.
  • When you need to protect AI agents with content filtering and jailbreak detection.
  • When you want to convert existing APIs into MCP-compatible tools.

Advantages

  • Centralized management of AI policies and security.
  • Significant cost savings via semantic caching.
  • Improved reliability through load balancing and retry logic.
  • Enhanced security using managed identities for backend authentication.
  • Detailed observability into token consumption and model performance.

Usage examples

“Add a new Azure OpenAI backend to my existing APIM instance.”
The agent provides the CLI commands to create the backend and the XML policy snippet to authenticate using managed identity.
“How do I implement semantic caching to save on AI costs?”
The agent explains how to apply the azure-openai-semantic-cache-lookup/store policies within the APIM inbound processing block.
“Set up load balancing for my AI models across two regions.”
The agent provides a policy pattern that uses a variable to rotate between backend URLs based on request hash.
“How can I protect my AI gateway from jailbreak attempts?”
The agent guides you to apply the llm-content-safety policy to filter harmful content and detect malicious inputs.

Install

  1. Download the skill as a ZIP:

    https://autogineer.com/agent-skills/microsoft/azure-aigateway/download

  2. Unzip it into the cross-client skills folder of your project:

    mkdir -p .agents/skills && unzip azure-aigateway.zip -d .agents/skills

Documentation by Microsoft

Azure AI Gateway

Configure Azure API Management (APIM) as an AI Gateway for governing AI models, MCP tools, and agents.

To deploy APIM, use the azure-prepare skill. See APIM deployment guide.

When to Use This Skill

CategoryTriggers
Model Governance"semantic caching", "token limits", "load balance AI", "track token usage"
Tool Governance"rate limit MCP", "protect my tools", "configure my tool", "convert API to MCP"
Agent Governance"content safety", "jailbreak detection", "filter harmful content"
Configuration"add Azure OpenAI backend", "configure my model", "add AI Foundry model"
Testing"test AI gateway", "call OpenAI through gateway"

Quick Reference

PolicyPurposeDetails
azure-openai-token-limitCost controlModel Policies
azure-openai-semantic-cache-lookup/store60-80% cost savingsModel Policies
azure-openai-emit-token-metricObservabilityModel Policies
llm-content-safetySafety & complianceAgent Policies
rate-limit-by-keyMCP/tool protectionTool Policies

Get Gateway Details

# Get gateway URL
az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv

# List backends (AI models)
az apim backend list --service-name <apim-name> --resource-group <rg> \
  --query "[].{id:name, url:url}" -o table

# Get subscription key
az apim subscription keys list \
  --service-name <apim-name> --resource-group <rg> --subscription-id <sub-id>

Test AI Endpoint

GATEWAY_URL=$(az apim show --name <apim-name> --resource-group <rg> --query "gatewayUrl" -o tsv)

curl -X POST "${GATEWAY_URL}/openai/deployments/<deployment>/chat/completions?api-version=2024-02-01" \
  -H "Content-Type: application/json" \
  -H "Ocp-Apim-Subscription-Key: <key>" \
  -d '{"messages": [{"role": "user", "content": "Hello"}], "max_tokens": 100}'

Common Tasks

Add AI Backend

See references/patterns.md for full steps.

# Discover AI resources
az cognitiveservices account list --query "[?kind=='OpenAI']" -o table

# Create backend
az apim backend create --service-name <apim> --resource-group <rg> \
  --backend-id openai-backend --protocol http --url "https://<aoai>.openai.azure.com/openai"

# Grant access (managed identity)
az role assignment create --assignee <apim-principal-id> \
  --role "Cognitive Services User" --scope <aoai-resource-id>
Apply AI Governance Policy

Recommended policy order in <inbound>:

  1. Authentication - Managed identity to backend

  2. Semantic Cache Lookup - Check cache before calling AI

  3. Token Limits - Cost control

  4. Content Safety - Filter harmful content

  5. Backend Selection - Load balancing

  6. Metrics - Token usage tracking

See references/policies.md for complete example.

Troubleshooting

IssueSolution
Token limit 429Increase tokens-per-minute or add load balancing
No cache hitsLower score-threshold to 0.7
Content false positivesIncrease category thresholds (5-6)
Backend auth 401Grant APIM "Cognitive Services User" role

See references/troubleshooting.md for details.

References

SDK Quick References

Files

Requirements and compatibility

Works with: Claude Code, Claude apps, OpenAI Codex, GitHub Copilot, Cursor, Gemini CLI, Kiro, OpenCode, Goose, Amp, Roo Code, JetBrains Junie, Factory Droid, Devin (ex-Windsurf), Mistral Vibe, OpenHands, Letta

“Works with” means the tool reads the open Agent Skills format; a skill written for one tool may use features others lack.

Needs: Needs a command-line tool

The author says: Requires Azure CLI (az) for configuration and testing

Security

What our scanner noticed· Low risk

  • Instructions that use the network

A static scan is not a guarantee. Skills are third-party content: review a skill before you let an agent use it.

Attribution and licence

Created by
Microsoft
Publisher
Microsoft (Official)
Source
microsoft/skills · .github/plugins/azure-skills/skills/azure-aigateway · commit 0696600
Licence
MIT License (redistribution allowed)
  • LICENSE (added by AutoGineer from .github/plugins/azure-skills/LICENSE)
  • ATTRIBUTION.md (written by AutoGineer)
Synced
Also distributed in
microsoft/azure-skills

AutoGineer is not affiliated with or endorsed by Microsoft or Microsoft.

Download ZIP

The ZIP holds the skill’s files, its licence files and an ATTRIBUTION.md. Download azure-aigateway.zip