How AI Model Inference Optimization Boosted Address Standardization Speed by 4×

Overview

Deep learning inference performance is critical for services such as address standardization. Optimizing inference can reduce response time, lower cost, and allow more complex models without degrading latency.

Inference Optimization Methodology

Natural language processing tasks such as RNN and BERT face performance challenges on x86 CPUs. The proposed solution combines high‑performance operators, model quantization, and AI compiler optimizations to accelerate inference.

Key Techniques

Model compression: quantization, sparsity, pruning.

High‑performance operators tailored to the model graph.

AI compiler optimizations: graph fusion, operator fusion, code generation.

Address Standardization Service

The service addresses non‑standard address data across many industries. Alibaba DAMO‑Lab provides an address purification service that standardizes address inputs, builds a unified address library, and offers high‑performance search, vector recall, and re‑ranking models.

Blade Optimization Platform

PAI‑Blade offers a unified interface for all the above optimizations, integrating high‑performance operators, Intel Custom Backend, and BladeDISC compiler to deliver end‑to‑end inference acceleration.

High‑Performance Operators on Intel Xeon

Optimizations for LSTM on Intel Xeon leverage AVX‑512 instructions, operator fusion, and cache‑aware scheduling. Input batching is performed with pack_padded_sequence() to handle variable‑length sequences efficiently.

Custom Backend Features

The Intel Custom Backend introduces a primitive cache to reuse compiled primitives, graph fusion to eliminate intermediate tensors, and memory optimizations that reduce runtime overhead.

Performance Evaluation

Two representative address‑search models were evaluated on an Alibaba ECS g7.large instance equipped with an Intel Xeon Platinum 8369B CPU.

ESIM (LSTM‑based) – LSTM‑A latency improved from 0.199 ms to 0.066 ms (+3.02×) and overall end‑to‑end latency dropped from 6.3 ms to 3.4 ms (+1.85×) while maintaining accuracy.

BERT – The 4‑layer INT8‑quantized model reduced latency from 37.0 ms to 9.0 ms (+4.11×). Macro F1 score increased from 77.24 to 78.85, demonstrating that quantization and compiler optimizations can improve both speed and accuracy.