How Adversarial Attacks Threaten AI: Real-World Cases & Alibaba’s Defense

AI Security

We know AI will dominate the internet era, but wherever code exists, vulnerabilities follow. AI security concerns can be divided into internal model flaws and external adversarial attacks.

Adversarial Example Technology

Adversarial examples were introduced by Christian Szegedy et al., showing that tiny, human‑imperceptible perturbations can cause deep neural networks to output high‑confidence wrong predictions.

Attacks can be non‑targeted (any misclassification) or targeted (forcing a specific wrong class). Generating them involves maximizing or minimizing the model’s loss within a bounded perturbation, which is an optimization problem.

Application Scenarios

Adversarial research has expanded from image classification to object detection and beyond.

Face Recognition

Face‑recognition systems are widely used, yet studies (Bose & Aarabi) show that adding imperceptible noise can evade detection or force the system to recognize a chosen wrong identity.

Adversarial Captchas

Traditional image and SMS captchas are being broken by deep‑learning models. Alibaba Security’s Turing Lab creates adversarial captchas by adding perturbations that remain user‑friendly but dramatically reduce automated solving rates.

Competition and Community

The IJCAI‑19 Alibaba AI Adversarial Algorithm Competition invites teams worldwide to develop attacks and defenses for image‑classification models, with over 1,000 teams registered.

Winners will present at the AI for Business Security workshop at IJCAI‑2019 in Macau.