How Lenovo & Alibaba Cloud Built a Token Platform for Scalable AI Model Operations
Lenovo Baiying partnered with Alibaba Cloud AI Gateway to create Token Plan and Token Key, enabling unified model access, fine-grained metering, traffic scheduling, and FinOps for scalable AI service delivery.
Background
Lenovo Baiying provides AI services and solutions to enterprises, lowering the barrier to AI adoption. As AI penetrates deeper into business, enterprises need stable model access and efficient management of identity, permissions, quotas, and costs.
To address this, Lenovo Baiying launched the Token Plan platform and Token Key (词元宝) — China's first integrated hardware-software token solution. Token Plan serves individual developers, teams, and enterprises with model packages, usage queries, and unified management of members, credentials, and quotas. Token Key enables multi-model unified access, fingerprint-level hardware key control, and plug-and-play simplicity, reducing token costs, improving data security, and simplifying operations.
Business Challenges
1. Fine-Grained User Management at Scale
The platform must differentiate model access permissions per user, handle token metering and deduction, and enforce rules consistently as user base grows. For example, when designing different Token Plan packages, each package maps to specific model access scopes and quota limits. On each request, the system identifies the user, checks permissions and quotas, and processes the call. Scattering this logic across applications and model adapters causes duplicate development and operational complexity.
2. Granular Observability of Model Calls
Beyond success/failure, the platform needs to collect usage fields (for billing), performance fields (e.g., TTFT — Time To First Token, critical for streaming UX), and error fields (for anomaly detection). Correlating these enables usage accounting and model scheduling optimization.
3. Upstream Model Service Rate Limits
Single model platforms impose API key quotas. Even if Token Key can accept more requests, upstream quotas become bottlenecks. The platform must distribute traffic across multiple service sources, fully utilize available quotas, and execute fallback routing when a service is limited or abnormal.
4. Protocol Heterogeneity
Different applications use different model calling protocols. The platform must bridge application-side access patterns with model-side interface capabilities so aggregated model services adapt to diverse terminals and agent scenarios.
Solution Architecture: API Gateway + AI Gateway + SLS
Lenovo Baiying combined Alibaba Cloud API Gateway (request authentication), AI Gateway (model management, usage observability, traffic scheduling), and Log Service SLS (log ingestion and data processing for metering). The architecture aggregates models like Qwen, MiniMax, DeepSeek, GLM, Kimi, and connects upward to terminals and agents.
AI Gateway: Unified Operational Framework for Model Calls
Fine-Grained Observability: Shared Data for Billing and Scheduling
AI Gateway captures usage, performance, and error metrics at the traffic ingress. SLS processes logs to support operations.
Usage fields aggregate model consumption for package accounting and metering.
Performance fields monitor service responsiveness; TTFT reflects user wait time for first generated token.
Error fields combined with above help analyze failure causes and inform routing adjustments (e.g., shift traffic when a source shows rising errors or latency).
This observability pipeline underpins the metering/billing system and accelerates feature rollout.
Model Scheduling & Fallback: Handling Upstream Quotas and Volatility
AI Gateway distributes traffic by model name or traffic ratio, allowing the platform to leverage quotas from multiple services and avoid concentration on a single source.
Model-name routing directs requests to the correct model service path.
Ratio-based allocation provides a configurable load distribution mechanism.
For rate-limiting or anomalies, AI Gateway offers Fallback with per-model backup policies. When predefined switch conditions are met, the gateway routes to a standby service, reducing upstream impact on users. Policies can reflect model capabilities, business needs, and resource plans.
Protocol Conversion: Bridging Application Demands and Model Supply
AI Gateway converts models that only support OpenAI protocol to expose Anthropic protocol, enabling applications requiring Anthropic to call those models. This eliminates per-scenario adapter development and expands the usable model pool across diverse terminals and agent tools.
Token Key: Connecting Model Services to End-User Delivery
Token Key focuses on how users obtain and configure services: plug-and-play, one-click setup, secure storage, one-click encryption, device binding, and usage transparency. For enterprise AI services, configuration is part of the product experience. Token Key links service configuration to delivery entry points, while the gateway handles identity, permissions, and usage post-request. Together they form a delivery chain from configuration to actual invocation, connecting scenarios like Baiying AI Host and Baiying Agents to backend model services so Token Plan benefits reach users concretely.
AI Gateway FinOps System
Consumer Management: Linking User Identity to Model Permissions
AI Gateway's consumer system provides unified call identity management. Creating consumers and binding relationships ties permission management to usage management. The platform can view model usage per identity, supporting quota management and billing. Service entitlements are enforced at call time, giving operations a clear management object.
Token & Credit Management: Grounding Package Entitlements in Usage Data
Leveraging AI Gateway's observability and FinOps features, the platform configures Credit Quota per consumer to support diverse Token Plan packages. The system builds a Token-to-Credit conversion and quota deduction layer on top of observable usage, enabling upper-layer package quota management. Operations can organize package benefits around business rules, linking user call consumption to quota enforcement. For ops teams, package management rests on real usage data; for users, transparent usage and billing improve trust.
Results & Future Outlook
The API Gateway + AI Gateway + SLS combination established a service management foundation for Token Plan, enabling rapid launch and ongoing operations around a unified call entry. User management, model supply, and usage data are now correlated, providing reusable governance for expanding access scenarios or adjusting service strategies.
Business teams align package design with model permissions and quota config.
Technical teams gain a unified control plane for model onboarding, traffic distribution, and anomaly handling.
Operations teams use call logs and usage data for cost accounting and runtime analysis.
The combined architecture is designed for million-user scale with precise metering/billing, reserving infrastructure headroom. As scenarios grow, the platform can add model sources, optimize traffic policies based on quality and usage, and refine Token Plan offerings — all supported by unified identity, observability, and scheduling mechanisms.
Conclusion
The Lenovo Baiying Token Plan & Token Key case demonstrates a core requirement for model service platforms: model capabilities become sustainably deliverable only through clear permissions, reliable metering, and stable call chains. Lenovo Baiying orchestrated terminals, agents, and Token Plan product delivery for enterprise scenarios; Alibaba Cloud AI Gateway provided model call governance, traffic scheduling, and observability. Together they built a platform that turns model services into scalable, operable products.
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.
