Engineering · 2026-05-16

Building an on-device AI coach with Apple's Foundation Models

There are a hundred "AI conversation coach" apps in the App Store. Almost all of them are wrappers around a server. We built ours on Apple's Foundation Models framework so the user's most sensitive conversations never leave the device. Here's what that actually looked like in code.

The product constraint that picked the architecture

Worldly Wisdom is a practice gym for the conversations people would rather avoid — tough feedback at work, family arguments, breakups. The value of the AI coach is that you can describe what's actually happening and get tailored practice. That value collapses if the user has to mentally sanitise their input first because the message is going to a server.

So privacy wasn't a feature we bolted on. It was the constraint that determined the architecture.

That ruled out the obvious approach (OpenAI/Anthropic API + a thin SwiftUI shell) and pushed us toward FoundationModels — Apple's on-device LLM framework that shipped with iOS 26 / Apple Intelligence. Free per user, fully local, works offline.

The service abstraction

Before any framework calls, we drew a line: nothing UI-facing should import FoundationModels. The framework is iOS 26+ and we still build for iOS 17+. The whole UI compiles regardless of availability; only the implementation crosses the version gate.

@MainActor
@Observable
final class AICoachService {
    static let shared = AICoachService()

    private(set) var availability: CoachAvailability

    private init() {
        if #available(iOS 26.0, *) {
            #if canImport(FoundationModels)
            self.availability = FoundationModelsCoach.currentAvailability()
            #else
            self.availability = .unsupportedOS
            #endif
        } else {
            self.availability = .unsupportedOS
        }
    }
    // reply, streamReply, generateScenario, beginSession, endSession ...
}

Two things matter here. First, the iOS-version if #available wraps the canImport check, not the other way around — canImport resolves at compile-time, but the iOS-26-only types inside FoundationModelsCoach can only be referenced inside an availability gate. Second, availability is @Observable so SwiftUI views automatically hide the AI surfaces when the device can't run them.

Sessions, not stateless calls

For the chat coach, we want a multi-turn conversation scoped to a single scenario. LanguageModelSession already does this — it tracks history internally — so we keep one session alive per scenario:

func startSession(scenario: Scenario, pickedLevel: Int, language: String) {
    let instructions = Self.buildInstructions(
        scenario: scenario,
        pickedLevel: pickedLevel,
        language: language
    )
    session = LanguageModelSession(instructions: instructions)
    session?.prewarm()
}

The instructions bake in the scenario text, the user's picked response (so the coach can critique it), the strategic tip, and the user's language. prewarm() kicks off model warmup while the user is still reading the scenario explanation; by the time they tap "Talk to AI Coach" the first token is usually <50ms away.

Why we don't pass history manually

The Foundation Models API has two patterns for multi-turn:

  1. Stateless: pass the full message history with each call.
  2. Stateful: keep a session, send only the new user turn.

The stateful path is meaningfully cheaper in tokens and avoids the prompt-tax of re-paying for a long history every turn. The downside is you can't trivially edit prior turns or rewind. We accepted that — coaching conversations are usually 2-5 turns and don't need a rewind affordance.

Guided generation for the structured surface

The other AI surface is "Your Situation" — user types a free-text description of their real situation, we return a complete practice item (situation summary, 4 ranked responses, explanation, takeaway tip). This needs a structured output, not a blob.

@Generable + @Guide lets us hand the model a Swift struct and get a strongly-typed result back:

@available(iOS 26.0, *)
@Generable
struct GeneratedScenarioPayload {
    @Guide(description: "A one-sentence retelling of the user's situation")
    let situation: String

    @Guide(description: "Exactly four response options, one at each EQ level 0..3")
    let options: [GeneratedScenarioOption]

    @Guide(description: "Why the level-3 (master) option is the best fit")
    let explanation: String

    @Guide(description: "One short, actionable takeaway sentence")
    let tip: String
}

Then the call site is plain:

let response = try await oneShot.respond(
    to: prompt,
    generating: GeneratedScenarioPayload.self
)
let payload = response.content   // fully-typed GeneratedScenarioPayload

No JSON parsing, no fallback for malformed model output. The framework's constrained decoding handles it.

A subtlety we hit

The model occasionally produced response options in the wrong order (level 3 listed first, or randomised). Two fixes:

Streaming the chat coach

Synchronous respond(to:) works but feels slow — the user taps "Talk to coach" and stares at a "Thinking…" placeholder for 1-2 seconds. streamResponse(to:) yields a cumulative snapshot of the response as tokens arrive:

return AsyncThrowingStream { continuation in
    let task = Task {
        do {
            let stream = session.streamResponse(to: userMessage)
            for try await partial in stream {
                continuation.yield(partial.content)
            }
            continuation.finish()
        } catch {
            continuation.finish(throwing: CoachError.modelError(error.localizedDescription))
        }
    }
    continuation.onTermination = { _ in task.cancel() }
}

Each partial is the full cumulative text, not a delta, so the SwiftUI view just renders the latest value directly with contentTransition(.opacity). onTermination cancels the underlying task if the sheet is dismissed mid-generation — important for not leaving inference running invisibly.

What about iOS 17-25 users?

Apple Intelligence requires iOS 26 + an iPhone 15 Pro / 16 / 17 / Air. That's a meaningful chunk of our addressable market who get the rest of the app (2,000 scenarios, six game modes, bilingual content) but not the AI layer.

We considered three approaches:

The "hide it" path means iOS 17-25 users don't see a teasing locked icon for a feature they can't unlock — App Review wouldn't love that either. The paywall is also conditional: the AI Coach feature row only appears when the device can actually run it.

What we'd do differently

The privacy claim, made concretely

"On-device" is a phrase that's been abused. To be specific:

Which means the privacy policy can say what it actually means, instead of the standard "we collect data to improve the service" tap-dance.

Worldly Wisdom launches on the App Store this month. If you'd like to know when it ships, the email list is here. Tech feedback welcome at support@worldlywisdom.app.
Back to the site →