How Didi’s Open‑Source DELTA Platform Accelerates NLP and Speech Model Development

Overview

DELTA is an open‑source deep‑learning training platform released by Didi at ACL 2019. It is built on TensorFlow and provides a unified framework for both natural‑language processing (NLP) and speech tasks.

Supported Tasks and Models

Text classification

Named‑entity recognition (NER)

Natural language inference (NLI)

Question answering (QA)

Sequence‑to‑sequence text generation

Automatic speech recognition (ASR)

Speaker verification

Speech emotion recognition

Architecture and Workflow

Users provide raw training data and a YAML/JSON configuration file that specifies:

Data preprocessing steps (tokenization, feature extraction, audio preprocessing)

Task type and corresponding model class

Hyper‑parameters such as learning rate, batch size, optimizer, and number of epochs

Output directory for the trained checkpoint

The DELTA pipeline then automatically:

Loads and preprocesses the data according to the configuration.

Selects the appropriate model architecture (e.g., Transformer, CNN‑RNN, BiLSTM) for the chosen task.

Runs distributed training on CPU/GPU or multi‑node clusters using TensorFlow’s Estimator or Keras APIs.

Saves a unified model artifact that includes the graph, weights, and a standardized inference interface.

Extensibility

Developers can extend DELTA by adding new model classes or custom preprocessing modules. The framework’s modular design allows composition of dozens of complex models on top of the core pipeline, enabling rapid reproduction of benchmark results from research papers and straightforward integration of novel algorithms.

Benchmarks and Deployment

DELTA ships with reference implementations and benchmark scripts for each supported task, providing baseline performance metrics (e.g., accuracy for classification, word error rate for ASR). Users can run the benchmark with a command such as: python run_benchmark.py --task=text_classification After training, the built‑in deployment tool converts the checkpoint into a serving package (SavedModel) that can be loaded by TensorFlow Serving or exported to other runtimes, facilitating a seamless transition from research to production.

Repository

Source code, documentation, and example configurations are hosted at:

https://github.com/didi/delta