How Hybrid AI Engine Simplifies On‑Device Image Recognition for Mobile Apps

Introduction

Artificial intelligence, deep learning and machine learning are now widely used in mobile apps for tasks such as background removal, real‑time dressing, automatic caption generation, speech‑to‑text conversion and AI customer service.

Why On‑Device AI?

Deploying machine‑learning results to the phone enables these features directly on the client side.

Three Traditional Solutions

Solution 1 : Capture and compress the image on the client, upload it to cloud storage, use a server‑side AI model for object recognition and return the result. This works on all devices but consumes bandwidth, cloud compute and raises privacy concerns.

Solution 2 : Train a model for the client and run it with an on‑device AI framework such as TensorFlow Lite or Paddle Paddle. It avoids network traffic but increases CPU/GPU power consumption, enlarges the app package and requires developers to be both AI and front‑end engineers.

Solution 3 : Call the phone’s built‑in AI capability (e.g., Huawei HiAI) which can schedule NPU resources for faster inference. It offers the best performance but is limited to a few device models and ROM versions.

Each approach has drawbacks.

Hybrid AI Engine

Sohu News created a hybrid dual‑engine AI client framework that abstracts common AI functions as independent, extensible modules. It combines TensorFlow Lite and system‑level AI (HiAI) and selects the optimal mode (speed‑first, result‑first, performance‑first) at runtime.

The framework provides a unified AIHelperFactory API, so application developers do not need to know whether the capability is cloud‑based, on‑device, or system‑level.

Code Example – Simple Image‑to‑Text

AIHelperFactory aiHelperFactory = AIHelperFactory.getInstance(context);
aiHelperFactory.init(AIHelperFactory.AI_TOOL_RECOGNIZER);
...
if (AIHelperFactory.getInstance(context).isReady(AIHelperFactory.AI_TOOL_RECOGNIZER)) {
    discription = AIHelperFactory.getInstance(context).getRecognizer().recognize(recognizeBitmap);
}
...
AIHelperFactory.getInstance(context).release(AIHelperFactory.AI_TOOL_RECOGNIZER);

Compared with a raw TensorFlow Lite implementation, the hybrid API is much shorter.

Hybrid Engine Workflow

public static AIHelperFactory getInstance(Context context) {
    if (sInstance != null) return sInstance;
    sInstance = new AIHelperFactory(context);
    return sInstance;
}
public void init(final int... tools) {
    if (HiAIUtils.isHuaweiAIAppInstalled(mContext)) {
        VisionBase.init(mContext, new ConnectionCallback() { ... });
    } else {
        for (int tool : tools) { createTools(tool); }
    }
}

The engine first checks for system‑level AI; if unavailable it falls back to TensorFlow Lite, downloading models on‑demand and updating them as needed.

Future Outlook

The hybrid engine currently supports image recognition, face detection, document detection, speech input and more, with a core SDK of about 100 KB and optional modules downloadable at runtime. The project is in beta and will be open‑sourced after the 1.0 release.