HarmonyOS On-Device AI Super-Resolution: Send Small Images, Get HD Results Offline

Introduction

HarmonyOS 7 (API 26) introduces a system-level AI image processing capability in Core Vision Kit called Image Super-Resolution (ImageSR). It can intelligently upscale low-resolution, blurry, or detail-poor images by 4x while enhancing clarity. Developers do not need to train custom models; they simply convert images to PixelMap and hand them to the system analyzer for on-device super-resolution reconstruction. The entire process runs offline — no data leaves the device.

Why Use On-Device Image Super-Resolution

Projects involving images often face an "impossible" trade-off among clarity, cost, and user experience. Three typical approaches and their drawbacks:

Send HD images : Clear with rich details, but consumes bandwidth and storage, loads slowly, and provides poor weak-network experience.

Send compressed images : Saves bandwidth and loads fast, but becomes blurry when enlarged, hurting user experience and conversion rates.

Custom AI super-resolution : Allows customization, but incurs high algorithm team costs, power consumption, heat, and difficult multi-device adaptation.

The solution: cloud sends only low-resolution thumbnails; the terminal uses the system-level AI capability to perform 4x HD reconstruction on-device via NPU. Transmission phase reduces bandwidth and storage costs; display phase completes reconstruction in hundreds of milliseconds, fully offline.

AI Super-Resolution Is Not Just "Upscaling"

Traditional Upscaling vs. AI Super-Resolution

Traditional upscaling interpolates missing pixels by averaging surrounding pixels — pixel count increases but information does not, resulting in blur. For example, a 100×100 image scaled to 800×800 looks very blurry.

AI super-resolution uses a system-provided model trained on millions of clear images and their blurred counterparts. The model predicts and reconstructs plausible details rather than averaging. Key distinction: interpolation spreads existing information thin; super-resolution adds new information.

Core Tasks of AI Super-Resolution

Edge enhancement : Sharpen object contours, text boundaries, and lines.

Texture completion : Reconstruct surface textures of products, architectural lines, photo details based on context.

Noise reduction : Suppress blocky artifacts and blur noise common in compressed images.

Underlying Optimizations

HarmonyOS employs NPU quantization, XPU heterogeneous computing, and multi-threaded pipelines. The system assigns appropriate computations to suitable hardware units, making on-device processing faster and more power-efficient.

Important Caveat

Super-resolution is essentially "guessing" — it may guess wrong. For example, a blurry line of small text on a product image might become a clear but incorrect line of text after super-resolution. Therefore, this capability is suitable for improving visual perception, not for evidentiary use. It is not recommended for scenarios heavily reliant on true details such as social photos or ID photos.

Core API Specifications and Constraints

Core Interfaces

ImageSRAnalyzer.create()

: Creates an image super-resolution analyzer instance. ImageSRAnalyzer.process(request): Executes one super-resolution pass, returns ISPResponse. ISPResponse.pixelMap: Retrieves the super-resolved image (pixels scaled 4x synchronously). ImageSRAnalyzer.destroy(): Releases the analyzer resources.

Reuse guidance : ImageSRAnalyzer can be reused within a page; no need to recreate per image. Typically call create() on page appearance and destroy() on exit or when super-resolution is no longer needed.

Input and Output

process()

accepts a visionBase.Request. The image must be wrapped as visionBase.ImageData and placed in Request.inputData:

const imageData: visionBase.ImageData = {
  pixelMap: inputPixelMap
};
const request: visionBase.Request = {
  inputData: imageData
};

process()

is asynchronous. On completion it returns imageSuperResolution.ISPResponse; its pixelMap field is the super-resolution result, ready for ArkUI Image component display. The interface provides no scale factor or quality parameters; business logic should rely on the returned PixelMap 's actual dimensions and visual effect.

Complete Development Workflow

Flow Overview

Permission Configuration

First call to ImageSRAnalyzer.create() triggers an AI model download, requiring ohos.permission.INTERNET. Selecting images from the gallery also needs ohos.permission.READ_IMAGEVIDEO. Declare in module.json5:

{
  "module": {
    "name": "entry",
    "type": "entry",
    "deviceTypes": ["phone", "tablet", "2in1"],
    "abilities": [
      {
        "name": "EntryAbility",
        "srcEntry": "./ets/entryability/EntryAbility.ets",
        "launchType": "singleton"
      }
    ],
    "requestPermissions": [
      {
        "name": "ohos.permission.INTERNET",
        "reason": "$string:reason_internet",
        "usedScene": {
          "abilities": ["EntryAbility"],
          "when": "inuse"
        }
      },
      {
        "name": "ohos.permission.READ_IMAGEVIDEO",
        "reason": "$string:reason_read_imagevideo",
        "usedScene": {
          "abilities": ["EntryAbility"],
          "when": "inuse"
        }
      }
    ]
  }
}

Complete Demo Code (ArkTS V2, API 26+)

The following demo implements gallery image selection, on-device super-resolution reconstruction, and before/after comparison.

import { imageSuperResolution, visionBase } from '@kit.CoreVisionKit';
import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { fileIo } from '@kit.CoreFileKit';
import { photoAccessHelper } from '@kit.MediaLibraryKit';

// Super-resolution result data model
interface SuperResolutionResult {
  pixelMap: image.PixelMap;
  width: number;
  height: number;
}

// Image size info model
interface ImageSizeInfo {
  width: number;
  height: number;
}

@Entry
@ComponentV2
struct ImageSuperResolutionDemo {
  // Input original image
  @Local inputPixelMap: image.PixelMap | null = null;
  // Super-resolution output image
  @Local outputPixelMap: image.PixelMap | null = null;
  // Original image size info
  @Local inputSize: ImageSizeInfo | null = null;
  // Super-resolution result size info
  @Local outputSize: ImageSizeInfo | null = null;
  // Processing status message
  @Local statusMessage: string = 'Please select an image to start';
  // Processing flag
  @Local isProcessing: boolean = false;
  // Analyzer instance
  private analyzer: imageSuperResolution.ImageSRAnalyzer | null = null;
  // Analyzer ready flag
  private analyzerReady: boolean = false;

  async aboutToAppear(): Promise<void> {
    this.statusMessage = 'Initializing super-resolution analyzer...';
    try {
      // Create image super-resolution analyzer instance
      // First call downloads AI model over network, requires INTERNET permission
      this.analyzer = await imageSuperResolution.ImageSRAnalyzer.create();
      this.analyzerReady = true;
      this.statusMessage = 'Analyzer ready, please select an image';
      hilog.info(0x0000, 'ImageSRDemo', 'ImageSRAnalyzer created successfully');
    } catch (error) {
      const err = error as BusinessError;
      this.statusMessage = `Initialization failed: ${err.message}`;
      hilog.error(0x0000, 'ImageSRDemo', `Create analyzer failed. Code: ${err.code}, message: ${err.message}`);
    }
  }

  async aboutToDisappear(): Promise<void> {
    // Release analyzer resources, must pair with create()
    if (this.analyzer) {
      try {
        await this.analyzer.destroy();
        hilog.info(0x0000, 'ImageSRDemo', 'ImageSRAnalyzer destroyed');
      } catch (error) {
        const err = error as BusinessError;
        hilog.error(0x0000, 'ImageSRDemo', `Destroy analyzer failed: ${err.message}`);
      }
    }
  }

  // Select image from system gallery and convert to PixelMap
  private async selectImageFromGallery(): Promise<void> {
    if (!this.analyzerReady) {
      this.statusMessage = 'Analyzer not ready, please wait';
      return;
    }
    const photoPicker = new photoAccessHelper.PhotoViewPicker();
    try {
      const selectResult = await photoPicker.select({
        MIMEType: photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE,
        maxSelectNumber: 1
      });
      if (selectResult.photoUris.length === 0) {
        return;
      }
      const imageUri = selectResult.photoUris[0];
      // Open file and create ImageSource
      const fileSource = await fileIo.open(imageUri, fileIo.OpenMode.READ_ONLY);
      const imageSource = image.createImageSource(fileSource.fd);
      // Decode to RGBA_8888 PixelMap
      const decodingOptions: image.PixelMapDecodingOptions = {
        desiredPixelFormat: image.PixelMapFormat.RGBA_8888
      };
      this.inputPixelMap = await imageSource.createPixelMap(decodingOptions);
      // Record original image size
      const imageInfo = await imageSource.getImageInfo();
      this.inputSize = {
        width: imageInfo.size.width,
        height: imageInfo.size.height
      };
      await fileIo.close(fileSource);
      this.outputPixelMap = null;
      this.outputSize = null;
      this.statusMessage = `Original: ${this.inputSize.width}×${this.inputSize.height}, tap "Start Super-Resolution" to process`;
      hilog.info(0x0000, 'ImageSRDemo', `Image selected: ${this.inputSize.width}x${this.inputSize.height}`);
    } catch (error) {
      const err = error as BusinessError;
      this.statusMessage = `Image selection failed: ${err.message}`;
      hilog.error(0x0000, 'ImageSRDemo', `Select image failed: ${err.message}`);
    }
  }

  // Execute super-resolution processing
  private async processSuperResolution(): Promise<void> {
    if (!this.inputPixelMap || !this.analyzer) {
      this.statusMessage = 'Please select an image first';
      return;
    }
    this.isProcessing = true;
    this.statusMessage = 'Performing super-resolution reconstruction...';
    // Build super-resolution request
    const imageData: visionBase.ImageData = {
      pixelMap: this.inputPixelMap
    };
    const request: visionBase.Request = {
      inputData: imageData
    };
    try {
      const startTime = Date.now();
      // Call super-resolution interface
      const response = await this.analyzer.process(request);
      const elapsed = Date.now() - startTime;
      // Get super-resolution result
      this.outputPixelMap = response.pixelMap;
      // Calculate output size (expected 4x original)
      const outputImageInfo = await image.createImageSource(response.pixelMap).getImageInfo();
      this.outputSize = {
        width: outputImageInfo.size.width,
        height: outputImageInfo.size.height
      };
      this.statusMessage = `Super-resolution completed in ${elapsed}ms, output: ${this.outputSize.width}×${this.outputSize.height}`;
      hilog.info(0x0000, 'ImageSRDemo', `Super resolution completed in ${elapsed}ms`);
    } catch (error) {
      const err = error as BusinessError;
      this.statusMessage = `Super-resolution failed. Code: ${err.code}, message: ${err.message}`;
      hilog.error(0x0000, 'ImageSRDemo', `Process failed. Code: ${err.code}, message: ${err.message}`);
    } finally {
      this.isProcessing = false;
    }
  }

  build() {
    Column({ space: 12 }) {
      // Title
      Text('Image Super-Resolution Demo')
        .fontSize(20)
        .fontWeight(FontWeight.Bold)
        .margin({ top: 20 })

      // Status message
      Text(this.statusMessage)
        .fontSize(13)
        .fontColor(Color.Gray)
        .width('90%')
        .textAlign(TextAlign.Center)
        .maxLines(2)
        .textOverflow({ overflow: TextOverflow.Ellipsis })

      // Original image display area
      Text('Original')
        .fontSize(14)
        .fontColor(Color.Gray)
      if (this.inputPixelMap) {
        Image(this.inputPixelMap)
          .width('80%')
          .height(180)
          .objectFit(ImageFit.Contain)
          .border({ width: 1, color: Color.Gray })
      } else {
        Column() {
          Text('No image')
            .fontSize(14)
            .fontColor(Color.Gray)
        }
        .width('80%')
        .height(180)
        .backgroundColor('#F5F5F5')
        .justifyContent(FlexAlign.Center)
      }

      // Original image size label
      if (this.inputSize) {
        Text(`Size: ${this.inputSize.width} × ${this.inputSize.height}`)
          .fontSize(11)
          .fontColor(Color.Gray)
      }

      // Action buttons
      Row({ space: 10 }) {
        Button('Select Image')
          .type(ButtonType.Capsule)
          .fontColor(Color.White)
          .layoutWeight(1)
          .onClick(() => this.selectImageFromGallery())

        Button('Start Super-Resolution')
          .type(ButtonType.Capsule)
          .fontColor(Color.White)
          .layoutWeight(1)
          .enabled(this.inputPixelMap !== null && !this.isProcessing)
          .onClick(() => this.processSuperResolution())
      }
      .width('80%')

      // Super-resolution result display area
      Text('Super-Resolution Result (4x)')
        .fontSize(14)
        .fontColor(Color.Gray)
        .margin({ top: 8 })
      if (this.outputPixelMap) {
        Image(this.outputPixelMap)
          .width('80%')
          .height(180)
          .objectFit(ImageFit.Contain)
          .border({ width: 1, color: '#007DFF' })
      } else {
        Column() {
          Text('Waiting for super-resolution')
            .fontSize(14)
            .fontColor(Color.Gray)
        }
        .width('80%')
        .height(180)
        .backgroundColor('#F5F5F5')
        .justifyContent(FlexAlign.Center)
      }

      // Super-resolution result size label
      if (this.outputSize) {
        Text(`Size: ${this.outputSize.width} × ${this.outputSize.height}`)
          .fontSize(11)
          .fontColor('#007DFF')
      }
    }
    .width('100%')
    .height('100%')
    .justifyContent(FlexAlign.Center)
  }
}

Pitfall Records

ImageSRAnalyzer.create() Hangs Indefinitely

Symptom : Promise neither resolves nor rejects; page stuck at "initializing".

Root Cause : First call downloads AI model over network. Three situations cause hang:

Running on emulator (Core Vision Kit unsupported on emulator).

Device offline (model download required on first use).

System version below API 26.

Solution : Must debug on real device with API 26+, ensure network connectivity, declare ohos.permission.INTERNET in module.json5. Wrap create() with timeout protection.

Input Image Format Mismatch

Symptom : process() throws parameter error.

Root Cause : Input image must be converted to RGBA_8888 format PixelMap; other formats may cause failure.

Solution : Explicitly specify decoding format when creating PixelMap:

const decodingOptions: image.PixelMapDecodingOptions = {
  desiredPixelFormat: image.PixelMapFormat.RGBA_8888
};
const pixelMap = await imageSource.createPixelMap(decodingOptions);

Input Image Size Exceeds Limit

Symptom : Large image processing fails.

Root Cause : Maximum input dimension is 2048×2048. Images exceeding this must be downscaled first.

Solution : Check image dimensions before passing to super-resolution; if over limit, scale down via PixelMap.scale().

Analyzer Resource Leak

Symptom : Memory grows continuously after frequent page switches, eventually OOM.

Root Cause : create() and destroy() not strictly paired; missing destroy() leaks analyzer resources.

Solution : Ensure destroy() called in aboutToDisappear, with proper error handling to prevent destroy() failure from breaking subsequent logic.

Super-Resolution Output Size Not As Expected

Symptom : Output image dimensions not exactly 4x input.

Root Cause : Interface provides no scale parameter; output follows system's actual processing result. Some extreme input sizes may cause minor deviations due to internal alignment strategies.

Solution : Business logic must not hardcode expected output as 4x input; instead read actual dimensions from returned ISPResponse.pixelMap.

Real-World Use Cases

Summary

HarmonyOS 7's image super-resolution capability, via Core Vision Kit's imageSuperResolution module, provides a zero-algorithm-threshold path for on-device AI image enhancement.

Core takeaways:

Integration takes four steps: create analyzer → wrap input data → call process → get result PixelMap.

Analyzer instance is reusable; create() and destroy() must be strictly paired.

First call downloads model over network; ohos.permission.INTERNET is mandatory.

Only supports Stage model, API 26+, real devices; emulator not supported.

Suitable for perceptual quality improvement; not for scenarios requiring exact detail fidelity.