AI Gateway provides enterprises with a unified AI service management platform. Its core benefits include:

• Unified Access to Multiple AI Models and Services
• Provide identity authentication and access control mechanisms
• Meet corporate security and compliance requirements
• Implement usage monitoring and budget management
• Supports intelligent load balancing and failover
• Enforcing Data Usage and Security Policies
• Supports horizontal scaling and rapid integration of new models

With AI Gateway, enterprises can manage their AI infrastructure in a more secure, efficient, and controllable manner.

AI Gateway provides unified access across models, intelligent routing, and automatic failover capabilities. Its core features include:

• Governance and Security: Achieve enterprise-level governance and security through identity authentication, access control, and policy enforcement mechanisms.
• Cost Optimization: Reduce AI usage costs through rate limiting, token quota management, and budget control mechanisms.
• Comprehensive Observability: Track usage, performance metrics, and operating costs in real time.
• Agent Workflow Support: Supports complex agent workflow orchestration and the execution of multi-step tasks.

As a unified control plane for enterprise AI infrastructure, AI Gateway helps organizations scale their AI deployments in a more secure and efficient manner.

Hongke AI Gateway provides comprehensive AI service deployment and management capabilities, designed specifically for enterprise-level scenarios. Its key features include:

• RBAC, OAuth 2.0, and API Keys: Multi-Layer Security Authentication
• Request Rate Limiting and Intelligent Load Balancing
• Automatic Failover Mechanism
• Built-in security and content governance capabilities
• Comprehensive logging, analytics, and observability
• Cloud Architecture Support
• Real-time reasoning ability

For enterprises that need to balance security, governance, performance, and cost control, Hongke offers a flexible and scalable AI infrastructure solution.

Traditional API gateways are primarily responsible for forwarding and managing general network requests. AI gateways, on the other hand, are specifically designed for large language model (LLM) scenarios and can handle many tasks that traditional API gateways cannot perform efficiently, such as:

• Token Statistics and Management
• Prompt Cache
• Model Routing
• Model Failure Transition
• AI Cost Control
• Model Governance and Security Policy Enforcement

Therefore, the AI gateway is not only a traffic entry point but also a unified governance and control center for enterprise AI applications.

AI Gateway is deployed directly within the production inference chain between the application system and the model service.

As a unified control plane, it is responsible for:

• Model Routing
• Safety Controls
• Permissions Management
• Observability Monitoring
• Cost Management
• Agent and Tool Orchestration

The entire process provides unified AI management capabilities without requiring any changes to the application logic.

Support. The Enterprise AI Gateway supports not only hosted models but also self-hosted or open-source models such as LLaMA and Mistral.

These models can be deployed in a VPC, on-premises data center, hybrid cloud, or isolated environment, while sharing the same policy controls, security governance, and observability capabilities as hosted models. Enterprises can achieve unified operations and governance without having to build separate management systems for different models.

AI Gateway provides real-time usage monitoring, token-level tracking, quota management, and budget control capabilities. Additionally, AI Gateway continuously optimizes AI spending through the following methods:

• Intelligent Model Routing
• Request Cache
• Automatic Failover
• Traffic Scheduling Optimization

By reducing unnecessary calls to high-cost models, we effectively prevent inference costs from spiraling out of control and continuously improve resource utilization.

AI Gateway enables the unified implementation of enterprise data governance strategies, including:

• Personal Identifiable Information (PII) Masking/De-identification
• Request Filtering
• Log Auditing and Access Control
• Data Processing Rules Management

When deployed in a VPC, on-premises data center, or a standalone isolated environment, all sensitive data remains within the enterprise perimeter at all times and never leaves the organization’s control. This helps enterprises ensure data security while meeting industry regulatory and compliance requirements.

AI Gateway implements team-level isolation and governance through the following mechanisms:

• Role-Based Access Control (RBAC)
• Team-Level API Key Management
• Quota Control
• Usage Tracking and Auditing

Multiple teams can securely share model resources and infrastructure while maintaining independent permissions, cost transparency, and accountability.

Playground is an interactive development environment built on top of AI Gateway. Developers can quickly test different options before deploying models to production systems:

• Large Language Models (LLMs)
• Prompt
• MCP Tools
• Inference Configuration Parameters

In Playground, users can:

• Select an integrated model from the Models page
• Adjust parameters such as Temperature and Max Tokens
• Configuring Streaming Output
• Set up generation rules such as Stop Sequences

The system will display the following in real time:

• Model Response Results
• Token Usage
• Response Delay Data

The development team can evaluate models and fine-tune parameters without writing code, significantly shortening the experimentation cycle.

Once the test results meet the requirements, the complete configuration—including the prompt, model, tools, security measures, and structured output definitions—can be saved as a reusable template and shared within the team.

Playground can also automatically generate code examples for frameworks such as the OpenAI SDK and LangChain, helping development teams quickly deploy validated solutions to production environments.

Through the Hongke AI Gateway, all model services and tools are unified into a single API layer. Development teams no longer need to manage SDKs, API endpoints, and access credentials for OpenAI, Anthropic, Amazon Bedrock, self-hosted models, and other model providers separately. Applications simply connect to a single Gateway Endpoint and use a single Gateway Key to access all model resources.

AI Gateway automatically handles model routing based on the configuration, so even if you replace a model or switch providers, you won't need to modify your application code.

This unified access capability extends beyond models to include:

• MCP (Model Context Protocol) Tool Ecosystem
• A2A (Agent-to-Agent) Protocol
• Agent Workflow

This enables unified orchestration and management of models, tools, and agents.

For developers, this means:

• A Simpler Approach to System Integration
• A More Consistent Development Experience
• A Clearer Safety Management Model

The "Supplier Key" needs to be maintained centrally in the gateway only once, and all access permissions are managed uniformly through RBAC and the policy framework. When a new model or supplier is integrated, it can be made available to the entire organization immediately through the same interface.

In AI Gateway, prompts, tools, and agent configurations are all treated as manageable assets. Development teams can define the following configurations in Playground and save them as named templates:

• System Prompt
• User Prompt
• Input Variables
• MCP Tools
• Security
• Model Parameters

Each template supports version control, allowing team members to safely iterate on and optimize configurations without overwriting each other’s changes. If a problem arises with a new version, you can quickly roll back to a previous version.

As a result, AI Gateway effectively provides enterprises with a unified platform for managing prompt and agent configurations. Once a specific configuration has been validated and is ready for deployment, it can be published as an Agent App.

Hongke AI Gateway's security mechanisms operate simultaneously at both the input and output stages, enabling defense-in-depth.

Input Phase Before a request is sent to the model, the system will automatically check for:

• Personally Identifiable Information (PII)
• Prompt Injection Attacks
• Prohibited Content
• Prohibited Topics

And implement them in accordance with corporate strategy:

• Interception
• Desensitization/Masking
• Replace
• Content Conversion

Output Phase After the model generates a response, the system performs another security review to check for:

• Harmful Content
• Biased Content
• Hallucination Output (Hallucination)
• Policy Violations
• Risk of Sensitive Data Leaks

And based on the strategy, decide:

• Return the result
• Editing Results
• Refuse to Export

AI Gateway can also be integrated with:

• OpenAI Moderation
• AWS Guardrails
• Azure Content Safety
• Azure PII Detection

It supports existing security services such as [...], while also enabling organizations to create custom rules using profiles or Python code.

Because all security policies are enforced uniformly at the gateway layer, security and compliance teams can implement organization-wide AI usage guidelines in a consistent and auditable manner, making this particularly suitable for highly regulated industries such as healthcare, finance, and insurance.

The Agent App is an agent application built on the AI Gateway that can be delivered directly to end users.

Unlike Playground, the Agent App provides a streamlined and controlled interactive interface. Business teams, testers, or internal users can directly experience how the Agent actually works without having to interact with the underlying prompts, tool configurations, or model parameters.

At the same time:

• Keep the prompt configuration locked
• Tool permissions are strictly controlled
• Security protection remains in effect
• Centralized Management of Model Routing Policies

This makes the Agent App ideal for:

• User Acceptance Testing (UAT)
• Internal Business Pilot Program
• High-Rise Model Unit
• Enterprise Copilot
• Knowledge Q&A Assistant

The product team and the platform team can both provide an open user experience while maintaining full control over configuration and governance rules.