Inside Huawei’s Open‑Source AscendNPU IR: Architecture, Triton Integration, and New Optimizations

On August 1, the Meet AI Compiler technical salon (session 9) gathered speakers from leading AI companies and research institutes to discuss AI compiler language expression, operator computation, inference execution, and scenario deployment. Huawei AscendNPU IR architect Hai Lijuan presented "AscendNPU IR: Open Compiler Base, Multi‑Language Support for Ascend".

Hai Lijuan systematically introduced the overall technical architecture and design philosophy of the AscendNPU IR component of the BiSheng compiler. She explained new features targeting Ascend 950, the advantages of recent technical iterations, and the ecosystem construction, dissecting the core capabilities and technical benefits of the open‑source base.

Why is Triton programming efficient and easy to use? Triton adopts Python‑like syntax and provides block‑level Tile programming, shielding developers from low‑level memory, instruction, and pipeline details, thereby lowering the barrier for operator development and improving productivity.

The Tile programming model lets developers focus on data partitioning and Tensor‑op abstraction. Compared with C++‑style heterogeneous languages, Tile abstracts kernel‑level data scheduling, while the compiler automatically handles intra‑kernel memory merging and shared‑memory management.

Both the Triton dialect and the official Triton GPU compiler are built on MLIR, enabling multi‑level IR abstraction for deep hardware optimization.

AscendNPU IR is an MLIR‑based Tile‑level abstraction for Ascend hardware, serving low‑level compilation optimization. It maps down to LLVM IR and finally compiles to Ascend binaries. Front‑end languages such as Triton can plug into AscendNPU IR to achieve hardware adaptation.

Key technical features of AscendNPU IR include a bottom‑up abstraction of Ascend hardware resources (memory, data‑movement units, etc.) and a Tile‑level OP abstraction that provides cross‑architecture support from A2/A3 to the latest Ascend 950 SIMD/SIMT architectures. Built on MLIR’s openness and extensibility, AscendNPU IR can accommodate more front‑end languages.

The compilation architecture consists of two layers. The hardware‑agnostic HFusion layer implements multi‑dimensional fusion optimizations based on community Linalg and extended OPs, handling data‑type and OP normalization, as well as auto‑scheduling. The hardware‑related HIVM layer performs high‑level hardware abstraction, including kernel mapping, on‑chip memory mapping, and processing‑unit mapping.

Kernel mapping compilation covers Cube and Vector kernels. The compiler automatically inserts data communication, synchronization, and workspace memory management, while also performing high‑order CV pipeline parallel optimizations.

On‑chip memory mapping translates logical Tensor data to physical memory, handling Cube‑specific matrix format derivation and allocation automatically.

Processing‑unit mapping ensures pipeline synchronization, vectorization, and tensorization to exploit high‑performance hardware instructions.

From Ascend A2/A3 to Ascend 950, the hardware‑related layer expanded from Mem‑based SIMD to support Reg‑based SIMD and SIMT. The new CV pipeline enables tighter coupling between Cube and Vector kernels, greatly benefiting Attention‑type operators.

Key optimizations include:

Vector Reg‑based SIMD: Tensor operations are vectorized onto fixed‑width registers, forming a Load‑Compute‑Store loop.

Register‑level fusion: Multiple Triton ops are merged into a single loop to reduce load‑store traffic, guided by search algorithms and cost models.

AVE dialect abstraction: Ascend‑specific Vector dialect hides hardware mask differences, enabling mask analysis, inference, and instruction merging.

Ascend 950 adds SIMT units, allowing Vector kernels to execute in pure SIMD, pure SIMT, or mixed modes, supporting both dense SIMD computation and sparse SIMT‑accelerated memory access.

Before compilation enters Vector SIMD or SIMT, a fusion analysis pass identifies code suitable for SIMT, extracts it, compiles SIMD and SIMT separately, then merges the results, modeling both paths.

CV pipeline optimization (e.g., FlashAttention) splits work across Cube and Vector kernels, using MultiBuffer to enable parallel execution and reduce serialization.

CV 1:2 splitting divides Tensor data between two Vector kernels and one Cube kernel, performing global dimension analysis to select the optimal split axis, inserting split ops from the store side upward, and falling back to 1:1 when necessary.

The open‑source AscendNPU IR project is now hosted in the Ascend community, with a bi‑weekly SIG meeting schedule. Developers can follow the community calendar for upcoming topics and minutes.

The community also offers open‑source internship projects and tasks, providing developers with opportunities to contribute. For participants without Ascend hardware, the free HiDevLab platform (https://hidevlab.huawei.com/home) supplies 100 hours of compute time after registration.

Overall, the presentation provided a detailed walkthrough of AscendNPU IR’s architecture, its integration with Triton, and the latest hardware‑aware optimizations that enable efficient AI operator compilation on Ascend 950.