HarmonyOS Face Detection Tutorial: Core Vision Kit Implementation with Reusable Code

Face Detection Capabilities in HarmonyOS Core Vision Kit

Face detection in HarmonyOS Core Vision Kit enables apps to automatically locate faces in images and extract key information. Beyond simple face presence detection, the API provides:

Face position : Rectangle coordinates (top-left X, Y, width, height) marking the face region.

Facial landmarks : Precise coordinates for left eye, right eye, nose, and mouth.

Face orientation : Values 0–3 representing upright, 90° counter-clockwise, 180° (upside-down), and 90° clockwise rotation relative to world coordinates.

Confidence score : Float 0–1 indicating detection reliability; higher values mean more accurate results.

Typical use cases include beauty apps (filter placement via landmarks), photo album clustering, access-control pre-checks, and interactive entertainment (stickers or game control based on orientation).

Constraints and Limitations

Real device required : The API does not run on simulators; testing must be done on physical HarmonyOS phones or tablets.

Image quality : Recommended minimum 720p resolution; width 100–10,000 px, height 224–15,210 px, aspect ratio ≤ 10:1. Blurry or extremely elongated images may fail.

Latency : Detection call latency is noticeable, making it unsuitable for real-time scenarios like live camera beautification. It is designed for offline image processing.

Four-Step Development Workflow

Initialize the face detection service ( faceDetector.init()).

Let the user pick an image from the system gallery.

Convert the selected image to a PixelMap (the only format the detector accepts).

Call faceDetector.detect() with the PixelMap, parse the returned Face[] array, and release the service ( faceDetector.release()).

Complete Reusable ArkTS Implementation

The following component encapsulates the entire flow: image selection, format conversion, detection, result parsing, and resource cleanup. It uses @State for the original image and detection result text, and PhotoViewPicker for gallery access.

// Import required kits
import { faceDetector } from '@kit.CoreVisionKit';
import { image } from '@kit.ImageKit';
import { hilog } from '@kit.PerformanceAnalysisKit';
import { BusinessError } from '@kit.BasicServicesKit';
import { fileIo } from '@kit.CoreFileKit';
import { photoAccessHelper } from '@kit.MediaLibraryKit';

const TAG = "FaceDetectionDemo";

@Entry
@Component
struct FaceDetectorPage {
  @State originalImage: PixelMap | undefined = undefined;
  @State detectionResult: string = "Detection results will appear here...";
  private imageSource: image.ImageSource | undefined = undefined;

  build() {
    Column({ space: 20 }) {
      // Display selected image
      Image(this.originalImage)
        .objectFit(ImageFit.Contain)
        .height('40%')
        .width('90%')
        .border({ width: 2, color: 0x317AE7, radius: 8 })
        .backgroundColor('#F5F5F5')
        .accessibilityDescription("Image to be detected")

      // Scrollable result area
      Scroll() {
        Text(this.detectionResult)
          .copyOption(CopyOptions.LocalDevice)
          .margin(10)
          .width('90%')
          .fontSize(14)
      }
      .height('25%')
      .border({ width: 1, color: '#E0E0E0', radius: 8 })
      .width('90%')

      // Pick image button
      Button('Select from Gallery')
        .type(ButtonType.Capsule)
        .backgroundColor(0x317AE7)
        .fontColor(Color.White)
        .width('90%')
        .height(45)
        .onClick(() => this.selectImageFromGallery())

      // Detect button
      Button('Start Face Detection')
        .type(ButtonType.Capsule)
        .backgroundColor(0x317AE7)
        .fontColor(Color.White)
        .width('90%')
        .height(45)
        .onClick(() => this.startFaceDetection())
    }
    .padding(20)
    .width('100%')
    .height('100%')
    .justifyContent(FlexAlign.Center)
  }

  // Step 1: Pick image from gallery
  private async selectImageFromGallery() {
    try {
      const photoPicker = new photoAccessHelper.PhotoViewPicker();
      const selectResult = await photoPicker.select({
        MIMEType: photoAccessHelper.PhotoViewMIMETypes.IMAGE_TYPE,
        maxSelectNumber: 1
      });

      const imageUri = selectResult.photoUris[0];
      if (imageUri) {
        await this.loadImageToPixelMap(imageUri);
      } else {
        this.detectionResult = "No image selected, please try again";
      }
    } catch (err: BusinessError | any) {
      hilog.error(0x0000, TAG, `Pick failed: ${err.message}`);
      this.detectionResult = `Pick failed: ${err.message} (code: ${err.code})`;
    }
  }

  // Step 2: Convert URI to PixelMap
  private async loadImageToPixelMap(uri: string) {
    try {
      const file = await fileIo.open(uri, fileIo.OpenMode.READ_ONLY);
      this.imageSource = image.createImageSource(file.fd);
      this.originalImage = await this.imageSource.createPixelMap();
      await fileIo.close(file); // Prevent resource leak
      this.detectionResult = "Image loaded. Tap 'Start Face Detection'...";
    } catch (err: BusinessError | any) {
      hilog.error(0x0000, TAG, `Image load failed: ${err.message}`);
      this.detectionResult = `Image load failed: ${err.message}`;
    }
  }

  // Step 3 & 4: Initialize, detect, parse, release
  private async startFaceDetection() {
    if (!this.originalImage) {
      this.detectionResult = "Please select an image first!";
      return;
    }

    try {
      await faceDetector.init();
      this.detectionResult = "Detecting faces, please wait...";

      const visionInfo: faceDetector.VisionInfo = {
        pixelMap: this.originalImage
      };

      const faceResult: faceDetector.Face[] = await faceDetector.detect(visionInfo);

      if (faceResult.length === 0) {
        this.detectionResult = "No faces detected. Choose a clearer photo.";
      } else {
        let resultText = `Detected ${faceResult.length} face(s)

`;

        faceResult.forEach((face, index) => {
          resultText += `=== Face ${index + 1} Details ===
`;

          // 1. Face rectangle
          const faceRect = face.faceRectangle;
          resultText += `Face Position:
`;
          resultText += `Top-left X: ${faceRect.left.toFixed(2)}, Y: ${faceRect.top.toFixed(2)}
`;
          resultText += `Width: ${faceRect.width.toFixed(2)}, Height: ${faceRect.height.toFixed(2)}
`;

          // 2. Confidence
          resultText += `Confidence: ${face.confidence.toFixed(4)} (higher is more reliable)
`;

          // 3. Orientation
          resultText += `Orientation: ${this.getFaceOrientation(face.orientation)}
`;

          // 4. Facial landmarks (if available)
          if (face.faceFeatures) {
            resultText += `Landmarks:
`;
            resultText += `Left Eye: X=${face.faceFeatures.leftEye.x.toFixed(2)}, Y=${face.faceFeatures.leftEye.y.toFixed(2)}
`;
            resultText += `Right Eye: X=${face.faceFeatures.rightEye.x.toFixed(2)}, Y=${face.faceFeatures.rightEye.y.toFixed(2)}
`;
            resultText += `Nose: X=${face.faceFeatures.nose.x.toFixed(2)}, Y=${face.faceFeatures.nose.y.toFixed(2)}
`;
            resultText += `Mouth: X=${face.faceFeatures.mouth.x.toFixed(2)}, Y=${face.faceFeatures.mouth.y.toFixed(2)}
`;
          }

          resultText += `
`;
        });

        this.detectionResult = resultText;
      }

      // Always release after detection
      await faceDetector.release();
    } catch (error: BusinessError) {
      this.detectionResult = `Detection failed: ${error.message} (code: ${error.code})`;
      hilog.error(0x0000, TAG, `Detection failed: ${error.message}, code: ${error.code}`);
      await faceDetector.release(); // Release even on error
    }
  }

  // Helper: map orientation enum to readable string
  private getFaceOrientation(orientation: number): string {
    switch (orientation) {
      case 0: return "Upright (normal)";
      case 1: return "Rotated 90° counter-clockwise";
      case 2: return "Rotated 180° (upside-down)";
      case 3: return "Rotated 90° clockwise";
      default: return "Unknown orientation";
    }
  }
}

Key Code Sections Explained

1. Initialization and Release Pairing

faceDetector.init()

prepares the service; faceDetector.release() frees native resources. They must appear in pairs — both on success and in the catch block — to avoid memory leaks.

2. Mandatory PixelMap Conversion

The detector only accepts PixelMap. The code opens the file descriptor via fileIo.open, creates an ImageSource with image.createImageSource, generates the PixelMap via createPixelMap(), and immediately closes the file descriptor with fileIo.close.

3. Result Parsing Details

faceResult.length

: Number of detected faces. face.faceRectangle: Bounding box for drawing overlays. face.confidence: 0–1 confidence; filter low-confidence results (e.g., < 0.8) to reduce false positives. face.orientation: Integer 0–3 mapped to human-readable strings via the helper function. face.faceFeatures: Contains leftEye, rightEye, nose, mouth coordinates. This object can be null for blurry faces; the code guards with if (face.faceFeatures) before accessing.

4. Error Handling Strategy

All async operations are wrapped in try-catch catching BusinessError. Errors are logged via hilog.error and surfaced to the UI with both message and error code. The service is released in the catch block as well.

Pitfall Checklist

No simulator support — test only on real HarmonyOS devices.

Image specs — 720p+, 100–10,000 px width, 224–15,210 px height, aspect ratio ≤ 10:1.

Not real-time — latency makes it unfit for live camera feeds; use for static images.

Always release — call faceDetector.release() in both success and failure paths.

Null landmarks — face.faceFeatures may be absent; guard before reading.

Data Privacy and Security

On-device processing : Images and face data never leave the device; detection runs locally.

No data retention : The system service does not store any face information after the call returns.

Permission model : No extra sensitive permissions required; the system gallery picker and vision kit handle authorization internally.

Developers should disclose in their privacy policy that face data is processed locally and not uploaded.

Development Summary and Extensions

The core flow mirrors HarmonyOS OCR and subject segmentation: pick → convert → detect → parse. A beginner can ship a working feature in half a day. Suggested enhancements:

Draw bounding boxes and landmark dots on the displayed image.

Filter results by confidence threshold (e.g., > 0.8).

Auto-rotate images based on orientation or add AR stickers aligned to landmarks.

Batch-process multiple images from the gallery.

Although not used in the author's "Command Cube" app, face detection is broadly applicable across HarmonyOS apps. The provided code can be adapted directly.