Integrating Paddle Serving with Kong Security Gateway for AI Model Deployment

As the AI wave continues, many enterprises are experiencing the benefits of deep learning in digital transformation and daily operations. Deploying deep‑learning services using a server‑client (serving) architecture has become the most common way to meet the needs of non‑AI developers.

Microservice architectures, while solving many monolithic problems, introduce new security challenges: multiple protocols (HTTP, AMQP, gRPC), evolving service boundaries, dynamic host/port changes, and lack of HTTPS support.

The new security gateway feature of Paddle Serving can address all these issues in a single solution.

A security gateway typically consists of a router and a processing unit that together provide protocol, link, and application‑level protection. Unlike generic gateways, it does not perform protocol conversion but controls inbound/outbound traffic and protects internal networks.

To illustrate the solution, the open‑source API gateway framework Kong (with its UI management tool Konga) is used. Kong, built on OpenResty (Nginx + Lua), offers high availability and extensibility. Konga provides a graphical interface for managing Kong configurations, multiple nodes, users, and database integration (MySQL, PostgreSQL, MongoDB).

Paddle Serving, Baidu’s inference framework for enterprise‑grade model serving, tightly integrates with the Paddle training ecosystem and supports one‑click deployment, model management, online loading, A/B testing, high‑concurrency client‑server communication, and multiple client languages (C++, Python, Java).

By combining Paddle Serving with Kong and Konga, a complete secure serving gateway is built. The deployment steps are demonstrated using Docker‑Compose:

docker-compose -f tools/auth/auth-serving-docker.yaml up -d

After launching the containers, the following services are visible (excerpt):

Docker ID: 97c5af96b29e, Image: pantsel/konga:next, Port: 0.0.0.0:8005→1337/tcp

Docker ID: bf98bad4a6f6, Image: registry.baidubce.com/serving_gateway/kong:paddle, Ports: 0.0.0.0:8000→8000/tcp, 127.0.0.1:8001→8001/tcp, 0.0.0.0:8443→8443/tcp, 127.0.0.1:8444→8444/tcp

Docker ID: 750be31a8b7f, Image: registry.baidubce.com/serving_dev/serving-runtime:cpu-py36, Port: 0.0.0.0:9393→9393/tcp

Access the Kong admin console at https://$IP_ADDR:8001 , register the Paddle Serving service, and create a route mapping /serving-uci to the serving container.

Finally, a prediction request can be sent through the secured gateway:

curl -H "Content-Type:application/json" -H "X-INSTANCE-ID:kong_ins" -H "apikey:hP6v25BQVS5CcS1nqKpxdrFkUxze9JWD" -X POST -d '{"feed":[{"x":[0.0137,-0.1136,0.2553,-0.0692,0.0582,-0.0727,-0.1583,-0.0584,0.6283,0.4919,0.1856,0.0795,-0.0332]}],"fetch":["price"]}' https://127.0.0.1:8443/serving-uci/uci/prediction -k

This request demonstrates key security features: HTTPS encryption, path mapping that hides the original service endpoint, and header‑based authentication (X‑INSTANCE‑ID and API key).

The whole workflow can be reproduced locally with Docker, and a Kubernetes‑based industrial deployment is also available in the project's GitHub repository.

For further details, refer to the documentation at https://github.com/PaddlePaddle/Serving/blob/v0.6.0/doc/SERVING_AUTH_DOCKER.md#k8s部署 . A live demo session is scheduled for August 17, 19:00‑20:00, with registration via the QR code below.