Unlock Apple’s On‑Device LLM: Foundation Models Guide & Travel Planner Demo

1. Understanding Foundation Models and Apple On‑Device LLM

1.1 Overview

Foundation Models is Apple’s AI integration framework announced at WWDC 2025. Building on Apple Intelligence, it opens on‑device large language model capabilities to developers via a Swift API.

The Swift API lets apps access the on‑device model, extend its abilities, and enhance existing functionality with model outputs.

1.2 Advantages

Local model does not increase app bundle size.

On‑device inference eliminates network latency and errors, improving responsiveness.

Developers bypass API rate limits and token costs.

User privacy is protected.

The model is fine‑tuned for everyday user scenarios and can be customized at runtime via Foundation Models adapters.

1.3 Limitations

Despite its size, the on‑device model cannot perform advanced logical reasoning, stay up‑to‑date with world knowledge, or master domain‑specific expertise.

2. Technical Highlights of Foundation Models

2.1 Guided Generation

Developers annotate Swift structs or enums with @Generable macros, enabling the model to output rich Swift data structures directly. This relies on vertical integration of the model, OS, and Swift compiler, which injects the expected output format into prompts and uses optimized constraint decoding to ensure correct schema.

2.2 Snapshot Streaming

Instead of token‑by‑token streaming, the framework returns partially generated objects with optional properties. The @Generable macro creates a PartiallyGenerated class whose streamResponse method yields an asynchronous sequence, allowing seamless integration with SwiftUI animations.

2.3 Tool Calling

Tool calling lets the model execute custom code inside the app. The model decides when to invoke external tools (e.g., MapKit) to fetch up‑to‑date information, and the framework automatically runs the tool and inserts the result back into the conversation.

3. Hands‑On: Build a Travel‑Itinerary Planner with Foundation Models

The following Swift code demonstrates a complete workflow from data‑model definition to UI rendering.

3.1 Define the Guided‑Generation Itinerary Structure

@Generable
struct Itinerary: Equatable {
    @Guide(description: "An exciting name for the trip.")
    let title: String
    @Guide(.anyOf(ModelData.landmarkNames))
    let destinationName: String
    let description: String
    @Guide(description: "An explanation of how the itinerary meets the user's special requests.")
    let rationale: String
    @Guide(description: "A list of day‑by‑day plans.")
    @Guide(.count(3))
    let days: [DayPlan]
}

3.2 Define a Map Tool for Point‑of‑Interest Lookup

@Observable
final class FindPointsOfInterestTool: Tool {
    let name = "findPointsOfInterest"
    let description = "Finds points of interest for a landmark."
    let landmark: Landmark
    init(landmark: Landmark) { self.landmark = landmark }
    @Generable
    struct Arguments {
        @Guide(description: "This is the type of destination to look up for.")
        let pointOfInterest: Category
        @Guide(description: "The natural language query of what to search for.")
        let naturalLanguageQuery: String
    }
    func call(arguments: Arguments) async throws -> ToolOutput {
        let items = try await pointOfInterest(location: landmark.location, arguments: arguments)
        let result = items.prefix(10).compactMap { $0.name }
        return ToolOutput("There are these \(arguments.pointOfInterest) in \(landmark.name): \(result.formatted())")
    }
}

3.3 Create the Itinerary Planner Using the On‑Device Model

@Observable
final class ItineraryPlanner {
    private(set) var itinerary: Itinerary.PartiallyGenerated?
    private(set) var pointOfInterestTool: FindPointsOfInterestTool
    private var session: LanguageModelSession
    let landmark: Landmark
    init(landmark: Landmark) {
        self.landmark = landmark
        let tool = FindPointsOfInterestTool(landmark: landmark)
        self.pointOfInterestTool = tool
        self.session = LanguageModelSession(
            tools: [tool],
            instructions: Instructions {
                "Your job is to create an itinerary for the user."
                "Each day needs an activity, hotel and restaurant."
            }
        )
    }
    func suggestItinerary(dayCount: Int) async throws {
        let stream = session.streamResponse(
            generating: Itinerary.self,
            options: GenerationOptions(sampling: .greedy),
            includeSchemaInPrompt: false
        ) {
            "Generate a \(dayCount)-day itinerary to \(landmark.name)."
            "Give it a fun title and description."
        }
        for try await partial in stream {
            itinerary = partial
        }
    }
}

3.4 Render the Itinerary with SwiftUI Animations

struct ItineraryView: View {
    let landmark: Landmark
    let itinerary: Itinerary.PartiallyGenerated
    var body: some View {
        VStack(alignment: .leading, spacing: 16) {
            if let days = itinerary.days {
                ForEach(days) { plan in
                    DayView(landmark: landmark, plan: plan)
                        .transition(.blurReplace)
                }
            }
        }
        .animation(.easeOut, value: itinerary)
    }
}

4. Performance Debugging & User‑Experience Optimization

4.1 Playground

The Playground plugin provides instant feedback while editing Foundation Models code, similar to SwiftUI Previews, enabling rapid iteration.

4.2 Instruments

Instruments now includes a Foundation Models module that measures model load time, inference duration, and token count, helping developers locate bottlenecks.

4.3 UX Optimization Tips

4.3.1 Pre‑warming

Load the model in advance when the user navigates to a page that uses the LLM to reduce initial latency.

4.3.2 Use Full Examples Instead of Prompt‑Only Guided Generation

Setting IncludeSchemaInPrompt = false reduces token usage. After the first request, keep the option off and provide a complete example (with optional properties filled) when initializing the session.

extension Itinerary {
    static let exampleTripToJapan = Itinerary(
        title: "Onsen Trip to Japan",
        destinationName: "Mt. Fuji",
        description: "Sushi, hot springs, and ryokan with a toddler!",
        days: [
            DayPlan(
                title: "Sushi and Shopping Near Kawaguchiko",
                subtitle: "Spend your final day enjoying sushi and souvenir shopping.",
                destination: "Kawaguchiko Lake",
                activities: [
                    Activity(
                        type: .foodAndDining,
                        title: "The Restaurant serving Sushi",
                        description: "Visit an authentic sushi restaurant for lunch."
                    )
                ]
            )
        ]
    )
}