Alibaba Cloud Developer

Jun 5, 2017 · Artificial Intelligence

Alibaba’s Distributed Training Boosts Neural Machine Translation Speed

Since its 2013 debut, Neural Machine Translation (NMT) has approached human quality, but training costs are high; Alibaba’s team developed a distributed NMT system in 2017, employing data‑parallel, model‑average, BMUF, Downpour SGD, and Ring‑allReduce techniques to cut training time from over 20 days to a few days while maintaining translation quality.