Unlocking ARKit: How Apple’s AR Framework Powers iOS 11 Augmented Reality

ARKit Overview

Apple introduced ARKit at WWDC 2017 with iOS 11, enabling developers to create augmented reality experiences by merging virtual content with the real world.

Framework Architecture

ARKit sits between data analysis/processing and rendering. Data analysis generates support data; rendering is handled by three Apple frameworks: standard rendering views (SceneKit for 3D, SpriteKit for 2D) and a custom rendering view via Metal.

Tracking and Session Management

The core of ARKit is ARSession , which manages AVCaptureSession (video capture) and CMMotionManager (device motion). A session runs with an ARSessionConfiguration , combining camera frames with motion data to estimate device pose.

Scene Recognition

ARKit provides powerful scene‑recognition capabilities, including plane detection, hit‑testing, and light estimation. Plane detection supports horizontal planes, multiple simultaneous planes, automatic alignment of virtual objects to detected surfaces, and plane merging.

Plane Detection

Key points of plane detection: 1) detects horizontal planes relative to gravity, 2) can detect multiple planes concurrently, 3) automatically aligns 3D content to the detected plane, 4) merges adjacent planes.

Hit‑Testing

Hit‑testing casts rays from the camera into the scene to find intersecting points on detected planes, returning a set of ARHitTestResult objects. Four result types are available: featurePoint , estimatedHorizontalPlane , existingPlane , and existingPlaneUsingExtent .

Light Estimation

ARKit measures ambient light intensity (default 1000 lumens) to match virtual object lighting with the real environment, enhancing visual realism.

Demo Implementation

Using Xcode 9 AR templates, developers can choose SceneKit, SpriteKit, or Metal for rendering. A simple demo loads a 3D airplane model, configures ARWorldTrackingSessionConfiguration (6‑DOF on A9+ devices), runs the session, and anchors the model to a detected plane, allowing 360° inspection.

Conclusion

ARKit supports 3‑DOF on devices below A9 and 6‑DOF on A9+ devices, offering robust data analysis and rendering pipelines. Developers should master linear algebra, the ARKit framework, SceneKit/SpriteKit, and optionally Metal for custom rendering to build compelling AR applications.