How MML Simplifies Mobile AI Deployment: Architecture, Tools, and Code Walkthrough

Background

Deploying AI directly on mobile devices (edge AI) provides real‑time inference, reduced bandwidth, and better privacy, but requires selecting an inference engine, converting models, handling model delivery, preprocessing, post‑processing, resource management, upgrades, and encryption. A unified abstraction reduces integration effort across different backends.

MML Architecture

MML (Mobile Machine Learning) sits between the low‑level inference engine (Backend) and the application layer. It abstracts backends such as Paddle‑Lite, Paddle‑Mobile, Baidu’s BML, and Apple’s CoreML, exposing a single API to the business code.

MML Core

The Core layer abstracts the APIs of supported inference engines, allowing developers to program against a unified interface regardless of the underlying backend. It defines a standard inference pipeline: initialization, preprocessing, inference, post‑processing, and resource release. MML provides Java, Objective‑C, and C++ interfaces to minimize performance loss and enable code sharing across platforms.

Key Utilities

Model Decryption : Integrated Baidu‑developed encryption/decryption to securely load encrypted models in memory, with optional delegation to the backend.

Pre/Post‑Processing Toolkit : ARM‑assembly and GPU‑accelerated replacements for common OpenCV operations, handling data‑type conversion, resizing, and layout adjustments.

Task Scheduling : Dynamic runtime profiling decides whether a model runs on CPU or GPU and can perform hybrid scheduling where some operators execute on CPU and others on GPU.

Prediction Workflow with MML

1. Initialization

MMLMachineService *predictor = new mml_framework::MMLMachineService();
MMLConfig config;
config.precision = MMLConfig::FP32; // prediction precision
config.modelUrl = modelDir;      // model path
config.machine_type = MMLConfig::MachineType::PaddleLite; // Backend type
MMLConfig::PaddleLiteConfig paddle_config; // Backend‑specific config
config.machine_config.paddle_lite_config = paddle_config;
predictor->load(config);

2. Pre‑Processing

std::unique_ptr<const MMLData> input = predictor->getInputData(0);
input->mmlTensor->Resize({1, 3, 224, 224});
auto *data = input->mmlTensor->mutable_data<float>();

3. Inference

predictor->predict();

4. Post‑Processing

std::unique_ptr<const MMLData> output = predictor->getOutputData(0);
auto shape = output->mmlTensor->shape();
auto *data = output->mmlTensor->data<float>();
// Apply MML post‑processing utilities as needed

5. Resource Release

delete predictor;

MML Kit

MML Kit bundles ready‑to‑use SDKs for common AI capabilities such as super‑resolution and gesture recognition. These SDKs encapsulate model handling and inference details, allowing developers to integrate advanced AI features without dealing with model conversion or backend specifics.

Future Outlook

The framework will continue to enhance the Core toolset, expand the Kit with more AI capabilities, and be open‑sourced on GitHub, providing a stable, secure, and easy‑to‑use solution for deploying AI on mobile devices.