on-device inference

DaTaobao Tech

Oct 14, 2024 · Artificial Intelligence

MNN Stable Diffusion: On‑Device Deployment and Performance Optimizations

The article presents Alibaba’s open‑source MNN inference engine, demonstrating how quantization, operator fusion (including fused multi‑head attention, GroupNorm/SplitGeLU, Winograd convolutions), optimized GEMM and memory‑paging enable on‑device Stable Diffusion with 1‑second‑per‑step performance on Snapdragon 8 Gen3 and Apple M3 GPUs, and outlines future speed‑up directions.

AIMNNPerformance Optimization

0 likes · 11 min read

MNN Stable Diffusion: On‑Device Deployment and Performance Optimizations

Sohu Tech Products

Mar 6, 2024 · Mobile Development

On‑Device Deployment of Large Language Models Using Sohu’s Hybrid AI Engine and GPT‑2

The article outlines how Sohu’s Hybrid AI Engine enables on‑device deployment of a distilled GPT‑2 model by converting it to TensorFlow Lite, detailing the setup, customization with Keras, inference workflow, and core SDK calls, and argues that this approach offers fast, private, and cost‑effective AI for mobile devices despite typical LLM constraints.

GPT-2Hybrid AIKeras

0 likes · 9 min read

On‑Device Deployment of Large Language Models Using Sohu’s Hybrid AI Engine and GPT‑2

DataFunTalk

Feb 10, 2022 · Artificial Intelligence

Evolution of Re‑ranking Techniques in Kuaishou Short‑Video Recommendation System

This article details the technical evolution of Kuaishou's short‑video recommendation pipeline, focusing on sequence re‑ranking, multi‑content mixing, and on‑device re‑ranking, and explains how transformer‑based models, generator‑evaluator frameworks, and reinforcement‑learning strategies are employed to maximize overall sequence value, user engagement, and revenue.

Kuaishoumulti-content mixingon-device inference

0 likes · 15 min read

Evolution of Re‑ranking Techniques in Kuaishou Short‑Video Recommendation System

Sohu Tech Products

Jan 20, 2021 · Mobile Development

Hybrid AI Engine: Integrating On‑Device Image Recognition with TensorFlow Lite and HiAI

This article introduces three traditional approaches for deploying machine‑learning models on mobile devices, analyzes their drawbacks, and presents a hybrid AI engine that combines TensorFlow Lite and system‑level HiAI to provide a unified, lightweight, and developer‑friendly on‑device image‑recognition solution, including code examples.

Android DevelopmentHybrid AI EngineTensorFlow Lite

0 likes · 12 min read

Hybrid AI Engine: Integrating On‑Device Image Recognition with TensorFlow Lite and HiAI