How We Cut 70% Costs and 90% Ops Work by Moving from Kubernetes to Serverless

Why the migration was needed

Our order‑center experienced extreme peak‑valley traffic, long‑running pods, high CPU idle time, and costly Kafka, monitoring, and mesh components. HPA scaling introduced several seconds of delay, forcing manual pre‑warming and extensive SRE effort on node, pod, and Kafka health.

When Kubernetes is still a good fit

Long‑running, always‑on services

Latency‑sensitive services with stable traffic

Workloads that need sidecars, service mesh, or custom networking

Infrastructure‑type workloads such as self‑hosted middleware, AI inference, or batch platforms

When Serverless shines

Highly variable traffic with large idle periods

Short‑duration, event‑driven requests

Heavy use of asynchronous messaging

Teams spending more time on platform ops than business code

Cost driven by actual requests rather than reserved capacity

Architecture before and after

Original K8s stack

30+ Spring Boot microservices

Kubernetes + HPA + Ingress

Kafka as event bus

Nacos for config & service discovery

Prometheus, Grafana, ELK for observability

Istio for service mesh

Target Serverless stack

API Gateway as unified entry point

Lambda functions for each business step (Order, Inventory, Payment, Notify, etc.)

DynamoDB for the order state table

EventBridge for domain event routing

SQS for reliable async buffering

Step Functions for long‑running transaction orchestration

Key design principles

Keep the synchronous API to three actions: validate & auth, idempotency check, write initial state & publish OrderCreated event.

Write the main order record first, then emit the event to avoid “ghost orders”.

All side‑effects must be explicitly idempotent because retries are normal in Serverless.

Long workflows are modeled as state machines, not scattered if/else code.

API design

POST /orders
Idempotency-Key: 20260808-u1001-8f1d
Content-Type: application/json

{
  "userId": "u1001",
  "items": [{"skuId": "sku-1", "quantity": 2, "price": 19900}],
  "currency": "CNY",
  "totalAmount": 39800,
  "clientRequestId": "req-90811"
}

Response (202 Accepted):

{
  "orderId": "ord_01J6X3XQZ4YJ5R7A4Z4M0Q7E1F",
  "status": "PENDING",
  "traceId": "1-6895d8f2-9a0dce2b215d4f7d85f1a210"
}

Production‑grade OrderFunction

// lambda/order-create/src/handler.ts
import { APIGatewayProxyEventV2, APIGatewayProxyStructuredResultV2 } from 'aws-lambda';
import { randomUUID } from 'node:crypto';
import { z } from 'zod';
import { OrderRepository } from './repository/order-repository';
import { IdempotencyRepository } from './repository/idempotency-repository';
import { EventPublisher } from './service/event-publisher';
import { Logger } from './support/logger';

const orderSchema = z.object({
  userId: z.string().min(1),
  currency: z.string().length(3),
  totalAmount: z.number().int().positive(),
  clientRequestId: z.string().min(1),
  items: z.array(z.object({
    skuId: z.string().min(1),
    quantity: z.number().int().positive(),
    price: z.number().int().positive()
  })).min(1)
});

const orderRepository = new OrderRepository();
const idempotencyRepository = new IdempotencyRepository();
const eventPublisher = new EventPublisher();
const logger = new Logger('order-create');

export async function handler(event: APIGatewayProxyEventV2): Promise<APIGatewayProxyStructuredResultV2> {
  const traceId = event.requestContext?.requestId ?? randomUUID();
  const idempotencyKey = event.headers['idempotency-key'] ?? event.headers['Idempotency-Key'];
  if (!idempotencyKey) {
    return json(400, { code: 'IDEMPOTENCY_KEY_REQUIRED', message: 'Missing Idempotency-Key', traceId });
  }
  let payload;
  try {
    payload = orderSchema.parse(JSON.parse(event.body ?? '{}'));
  } catch (error) {
    logger.warn('invalid_request', { traceId, error });
    return json(400, { code: 'INVALID_REQUEST', message: 'Invalid request payload', traceId });
  }
  const existing = await idempotencyRepository.findSuccessResult(idempotencyKey);
  if (existing) {
    logger.info('idempotent_hit', { traceId, idempotencyKey, orderId: existing.orderId });
    return json(202, existing);
  }
  const orderId = `ord_${randomUUID().replace(/-/g, '')}`;
  const now = Date.now();
  const order = {
    orderId,
    userId: payload.userId,
    currency: payload.currency,
    totalAmount: payload.totalAmount,
    items: payload.items,
    status: 'PENDING',
    paymentStatus: 'INIT',
    inventoryStatus: 'INIT',
    clientRequestId: payload.clientRequestId,
    idempotencyKey,
    createdAt: now,
    updatedAt: now
  };
  try {
    await orderRepository.create(order);
    await eventPublisher.publish({
      source: 'com.acme.order',
      detailType: 'OrderCreated',
      detail: { orderId, userId: payload.userId, totalAmount: payload.totalAmount, items: payload.items, traceId }
    });
    const response = { orderId, status: 'PENDING', traceId };
    await idempotencyRepository.saveSuccessResult(idempotencyKey, response, now + 24 * 3600 * 1000);
    logger.info('order_created', { traceId, orderId, userId: payload.userId, totalAmount: payload.totalAmount });
    return json(202, response);
  } catch (error) {
    logger.error('order_create_failed', { traceId, orderId, idempotencyKey, error });
    return json(500, { code: 'INTERNAL_ERROR', message: 'Create order failed', traceId });
  }
}

function json(statusCode: number, body: unknown) {
  return { statusCode, headers: { 'content-type': 'application/json' }, body: JSON.stringify(body) };
}

Supporting repositories

// OrderRepository – DynamoDB write with conditional PK existence
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, PutCommand } from '@aws-sdk/lib-dynamodb';
const client = DynamoDBDocumentClient.from(new DynamoDBClient({}));
export class OrderRepository {
  async create(order: Record<string, unknown>) {
    await client.send(new PutCommand({
      TableName: process.env.ORDERS_TABLE!,
      Item: { pk: `ORDER#${order.orderId}`, sk: 'META', ...order },
      ConditionExpression: 'attribute_not_exists(pk)'
    }));
  }
}

// IdempotencyRepository – stores successful responses for replay
import { DynamoDBClient } from '@aws-sdk/client-dynamodb';
import { DynamoDBDocumentClient, GetCommand, PutCommand } from '@aws-sdk/lib-dynamodb';
const client = DynamoDBDocumentClient.from(new DynamoDBClient({}));
type SuccessResult = { orderId: string; status: string; traceId: string };
export class IdempotencyRepository {
  async findSuccessResult(idempotencyKey: string): Promise<SuccessResult | null> {
    const result = await client.send(new GetCommand({
      TableName: process.env.IDEMPOTENCY_TABLE!,
      Key: { pk: `IDEMP#${idempotencyKey}` }
    }));
    return (result.Item?.response as SuccessResult) ?? null;
  }
  async saveSuccessResult(idempotencyKey: string, response: SuccessResult, expireAtMs: number) {
    await client.send(new PutCommand({
      TableName: process.env.IDEMPOTENCY_TABLE!,
      Item: { pk: `IDEMP#${idempotencyKey}`, response, ttl: Math.floor(expireAtMs / 1000) }
    }));
  }
}

// EventPublisher – puts events onto EventBridge
import { EventBridgeClient, PutEventsCommand } from '@aws-sdk/client-eventbridge';
const eventBridgeClient = new EventBridgeClient({});
export class EventPublisher {
  async publish(event: { source: string; detailType: string; detail: Record<string, unknown> }) {
    await eventBridgeClient.send(new PutEventsCommand({
      Entries: [{ EventBusName: process.env.EVENT_BUS_NAME!, Source: event.source, DetailType: event.detailType, Detail: JSON.stringify(event.detail) }]
    }));
  }
}

// Logger – structured JSON logs
export class Logger {
  constructor(private readonly service: string) {}
  info(message: string, context: Record<string, unknown>) { console.log(JSON.stringify({ level: 'INFO', service: this.service, message, ...context })); }
  warn(message: string, context: Record<string, unknown>) { console.warn(JSON.stringify({ level: 'WARN', service: this.service, message, ...context })); }
  error(message: string, context: Record<string, unknown>) { console.error(JSON.stringify({ level: 'ERROR', service: this.service, message, ...context })); }
}

Inventory function responsibilities (pseudocode)

async function handleOrderCreated(event) {
  if (await inventoryDedupRepo.exists(event.orderId)) return;
  const success = await inventoryGateway.reserve(event.orderId, event.items);
  if (success) {
    await orderStatusRepo.markInventoryReserved(event.orderId);
    await eventBus.publish('InventoryReserved', { orderId: event.orderId, traceId: event.traceId });
  } else {
    await orderStatusRepo.markCancelled(event.orderId, 'OUT_OF_STOCK');
    await eventBus.publish('OrderCancelled', { orderId: event.orderId, reason: 'OUT_OF_STOCK' });
  }
  await inventoryDedupRepo.save(event.orderId);
}

Payment callback responsibilities (pseudocode)

async function handlePaymentCallback(event) {
  // verify signature, dedup by channel transaction ID
  const order = await orderRepo.get(event.orderId);
  if (!order || order.status !== 'PAYING') return; // ignore if not in a payable state
  // idempotent update
  await orderRepo.updateStatus(event.orderId, 'PAID');
  await eventBus.publish('PaymentSucceeded', { orderId: event.orderId, traceId: event.traceId });
}

Infrastructure‑as‑Code (CDK)

import * as cdk from 'aws-cdk-lib';
import * as apigatewayv2 from 'aws-cdk-lib/aws-apigatewayv2';
import * as integrations from 'aws-cdk-lib/aws-apigatewayv2-integrations';
import * as dynamodb from 'aws-cdk-lib/aws-dynamodb';
import * as events from 'aws-cdk-lib/aws-events';
import * as lambda from 'aws-cdk-lib/aws-lambda-nodejs';
import * as sqs from 'aws-cdk-lib/aws-sqs';

export class OrderStack extends cdk.Stack {
  constructor(scope: cdk.App, id: string, props?: cdk.StackProps) {
    super(scope, id, props);
    const ordersTable = new dynamodb.Table(this, 'OrdersTable', {
      partitionKey: { name: 'pk', type: dynamodb.AttributeType.STRING },
      sortKey: { name: 'sk', type: dynamodb.AttributeType.STRING },
      billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
      timeToLiveAttribute: 'ttl'
    });
    const idempotencyTable = new dynamodb.Table(this, 'IdempotencyTable', {
      partitionKey: { name: 'pk', type: dynamodb.AttributeType.STRING },
      billingMode: dynamodb.BillingMode.PAY_PER_REQUEST,
      timeToLiveAttribute: 'ttl'
    });
    const eventBus = new events.EventBus(this, 'OrderEventBus');
    const orderCreateFn = new lambda.NodejsFunction(this, 'OrderCreateFunction', {
      entry: 'lambda/order-create/src/handler.ts',
      runtime: cdk.aws_lambda.Runtime.NODEJS_20_X,
      timeout: cdk.Duration.seconds(10),
      memorySize: 1024,
      environment: {
        ORDERS_TABLE: ordersTable.tableName,
        IDEMPOTENCY_TABLE: idempotencyTable.tableName,
        EVENT_BUS_NAME: eventBus.eventBusName
      }
    });
    ordersTable.grantReadWriteData(orderCreateFn);
    idempotencyTable.grantReadWriteData(orderCreateFn);
    eventBus.grantPutEventsTo(orderCreateFn);
    const httpApi = new apigatewayv2.HttpApi(this, 'OrderHttpApi');
    httpApi.addRoutes({
      path: '/orders',
      methods: [apigatewayv2.HttpMethod.POST],
      integration: new integrations.HttpLambdaIntegration('OrderCreateIntegration', orderCreateFn)
    });
    new cdk.CfnOutput(this, 'HttpApiUrl', { value: httpApi.apiEndpoint });
  }
}

Distributed transaction with Step Functions

{
  "Comment": "Order timeout and compensation workflow",
  "StartAt": "WaitForPayment",
  "States": {
    "WaitForPayment": { "Type": "Wait", "Seconds": 900, "Next": "CheckOrderStatus" },
    "CheckOrderStatus": { "Type": "Task", "Resource": "arn:aws:lambda:region:account:function:OrderStatusCheckFunction", "Next": "IsPaid" },
    "IsPaid": {
      "Type": "Choice",
      "Choices": [{ "Variable": "$.status", "StringEquals": "PAID", "Next": "Done" }],
      "Default": "CancelOrder"
    },
    "CancelOrder": { "Type": "Task", "Resource": "arn:aws:lambda:region:account:function:OrderCancelFunction", "Next": "ReleaseInventory" },
    "ReleaseInventory": { "Type": "Task", "Resource": "arn:aws:lambda:region:account:function:ReleaseInventoryFunction", "Next": "Done" },
    "Done": { "Type": "Succeed" }
  }
}

High‑concurrency considerations

Cold‑start impact is mitigated by keeping function packages small, choosing a lightweight runtime, and using provisioned concurrency for the order entry Lambda.

Concurrency quotas must be verified before big‑sale events; plan for account‑level limits, API Gateway throttling, and downstream DynamoDB / SQS capacity.

Database connections are avoided in hot paths; state is stored in DynamoDB and only occasional relational queries go through a managed connection pool or proxy.

Observability shift

Instead of watching pod counts and CPU, we monitor business‑centric metrics such as CreateOrder.SuccessRate, latency percentiles, inventory reserve failure rate, payment duplicate rate, order timeout cancellations, Lambda throttles, and SQS queue age. All functions emit structured JSON logs containing traceId, orderId, userId, eventType, functionName and errorCode. X‑Ray (or equivalent) is enabled to answer where a request stalls, which downstream step slowed the flow, and whether a failure occurred in the sync entry or an async stage.

Security and governance

Least‑privilege IAM policies per function (order‑create can only write orders, idempotency table, and put events).

Strict input validation with zod and signature verification for external callbacks.

Separate environments (dev, staging, prod) with isolated EventBuses, SQS queues, KMS keys, and alerting pipelines.

Migration phases

Move side‑car, async, and compensation paths (notifications, audit, timeout handling) to Serverless first to establish event contracts and monitoring.

Switch the order entry API to API Gateway + Lambda while keeping downstream services temporarily.

Gradually replace synchronous inventory, payment, and notification calls with event‑driven Lambda functions.

Decommission the old K8s services, Kafka topics, and related alerts once traffic is fully on Serverless.

Results and boundaries

By eliminating always‑on nodes, Kafka clusters, and extensive monitoring stacks, we reduced infrastructure spend by ~70% and cut operational toil by ~90%. The approach is unsuitable for ultra‑low‑latency matching engines, long‑running batch jobs, or workloads that rely heavily on relational transactions and have consistently high utilization.