How Alibaba Cloud’s PAI Breakthroughs Are Shaping AI at EMNLP 2023

Paper Summaries

BeautifulPrompt: Automatic Prompt Engineering for Text-to-Image Synthesis

Text‑to‑image generation requires detailed prompts; non‑experts often struggle, leading to wasted resources. BeautifulPrompt uses a large language model with a three‑stage training pipeline to automatically generate high‑quality prompts, improving both objective model scores and human evaluations.

Domain‑Specific Knowledge‑Enhanced Language Model

Knowledge‑enhanced PLMs (KEPLM) inject facts from large knowledge graphs, but open‑domain methods fail in vertical domains due to sparse graph coverage. This work proposes a unified framework that leverages a hyperbolic knowledge‑aware aggregator and a multi‑level knowledge‑aware augmenter to capture dense local graph structures, yielding superior performance on financial and medical downstream tasks.

CogTree: Cognitive Tree Reasoning with Small Language Models

Large language models excel at many NLP tasks but struggle with complex logical or mathematical reasoning and incur high inference costs. CogTree introduces a dual‑system (intuition and reflection) that iteratively generates and verifies reasoning hypotheses using a 7B model, significantly boosting accuracy on Entailment Bank and GSM8K benchmarks.

X-STA: Cross-Lingual Machine Reading Comprehension via Knowledge Transfer

Cross‑lingual MRC suffers from scarce annotated data and translation‑induced answer misalignment. X-STA employs gradient‑based knowledge sharing, attention‑guided teaching, and multi‑level alignment to transfer knowledge from source to target languages, achieving noticeable accuracy gains on three multilingual MRC datasets.

These research outcomes have been integrated into various PAI modules, such as the BeautifulPrompt plugin for Stable Diffusion WebUI and the PAI‑QuickStart suite that supports over 20 large language models, enabling customers to deploy and fine‑tune models with minimal effort.