Using AVFoundation and CoreImage for Camera Image Capture and Face Detection on iOS

AVFoundation is one of Apple’s media‑handling frameworks that enables real‑time capture of camera images, screen content (on macOS), and video recording. The article introduces the basic AVFoundation architecture and shows how it connects AVKit, UIKit, Core Audio, Core Media, and Core Animation.

The focus is on using AVFoundation to capture camera pictures. The capture pipeline consists of three parts: AVCaptureDeviceInput , AVCaptureSession , and AVCaptureOutput . Each part must be initialized correctly.

The AVCaptureDeviceInput initialization image demonstrates how to set up the iOS rear camera; modifying the AVCaptureDevice allows switching to other devices such as the screen.

The AVCaptureSession is created by initializing an AVCaptureSession object and selecting an appropriate SessionPreset to define resolution and quality.

The AVCaptureOutput configuration includes setting video settings to control image size and adding a dispatch queue to receive captured frames.

After all three modules are set up, images can be captured by invoking the appropriate delegate method on the session’s output.

CoreImage is Apple’s powerful image‑processing framework. It provides filters ( CIFilter ) and detectors ( CIDetector ) such as face detection. The article shows how to initialize a CIDetector for faces and use featuresInImage to obtain an array of CIFaceFeature objects, each containing a bounding‑box rectangle.

Because UIKit’s coordinate system has its origin at the top‑left while CoreImage’s origin is at the bottom‑left, the article notes that coordinate conversion is required before drawing rectangles around detected faces.

The practical demo includes the setUpCamera and cameraCapture functions for camera initialization and image capture, as well as the faceDetect function for face detection. Images of these functions are provided.

References to Apple documentation for AVFoundation static image capture, CoreImage face detection, and related open‑source projects are listed at the end of the article.

Author: Lin Xiangyu, a master’s student in Computer Science at Nanjing University of Posts and Telecommunications, currently working in Baidu’s platform testing department on iOS mobile testing.