Engineering · 2026-05-16 · Updated 2026-10-13

Building an on-device AI coach with Apple's Foundation Models

We built Worldly Wisdom's AI coach on Apple's Foundation Models framework so coaching conversations are processed on the user's device. Here's how that choice shapes the app.

The product constraint that picked the architecture

Worldly Wisdom is a practice gym for the conversations people would rather avoid — tough feedback at work, family arguments, breakups. The value of the AI coach is that you can describe what's actually happening and get tailored practice. That value collapses if the user has to mentally sanitise their input first because the message is going to a server.

So privacy wasn't a feature we bolted on. It was the constraint that determined the architecture.

That ruled out the obvious approach (OpenAI/Anthropic API + a thin SwiftUI shell) and pushed us toward FoundationModels — Apple's on-device LLM framework that shipped with iOS 26 / Apple Intelligence. Free per user, fully local, works offline.

The service abstraction

Before any framework calls, we drew a line: nothing UI-facing should import FoundationModels. The framework is iOS 26+ and we still build for iOS 17+. The whole UI compiles regardless of availability; only the implementation crosses the version gate.

@MainActor
@Observable
final class AICoachService {
    static let shared = AICoachService()

    private(set) var availability: CoachAvailability

    private init() {
        if #available(iOS 26.0, *) {
            #if canImport(FoundationModels)
            self.availability = FoundationModelsCoach.currentAvailability()
            #else
            self.availability = .unsupportedOS
            #endif
        } else {
            self.availability = .unsupportedOS
        }
    }
    // reply, streamReply, generateScenario, beginSession, endSession ...
}

Two things matter here. First, the iOS-version if #available wraps the canImport check, not the other way around — canImport resolves at compile-time, but the iOS-26-only types inside FoundationModelsCoach can only be referenced inside an availability gate. Second, availability is @Observable so SwiftUI views automatically hide the AI surfaces when the device can't run them.

Sessions, not stateless calls

For the chat coach, we want a multi-turn conversation scoped to a single scenario. LanguageModelSession already does this — it tracks history internally — so we keep one session alive per scenario:

func startSession(scenario: Scenario, pickedLevel: Int, language: String) {
    let instructions = Self.buildInstructions(
        scenario: scenario,
        pickedLevel: pickedLevel,
        language: language
    )
    session = LanguageModelSession(instructions: instructions)
    session?.prewarm()
}

The instructions bake in the scenario text, the user's picked response (so the coach can critique it), the strategic tip, and the user's language. prewarm() kicks off model warmup while the user is still reading the scenario explanation; by the time they tap "Talk to AI Coach" the first token is usually <50ms away.

Why we don't pass history manually

The Foundation Models API has two patterns for multi-turn:

  1. Stateless: pass the full message history with each call.
  2. Stateful: keep a session, send only the new user turn.

The stateful path is meaningfully cheaper in tokens and avoids the prompt-tax of re-paying for a long history every turn. The downside is you can't trivially edit prior turns or rewind. We accepted that — coaching conversations are usually 2-5 turns and don't need a rewind affordance.

Guided generation for the structured surface

The other AI surface is "Your Situation" — user types a free-text description of their real situation, we return a complete practice item (situation summary, 4 ranked responses, explanation, takeaway tip). This needs a structured output, not a blob.

@Generable + @Guide lets us hand the model a Swift struct and get a strongly-typed result back:

@available(iOS 26.0, *)
@Generable
struct GeneratedScenarioPayload {
    @Guide(description: "A one-sentence retelling of the user's situation")
    let situation: String

    @Guide(description: "Exactly four response options, one at each EQ level 0..3")
    let options: [GeneratedScenarioOption]

    @Guide(description: "Why the level-3 (master) option is the best fit")
    let explanation: String

    @Guide(description: "One short, actionable takeaway sentence")
    let tip: String
}

Then the call site is plain:

let response = try await oneShot.respond(
    to: prompt,
    generating: GeneratedScenarioPayload.self
)
let payload = response.content   // fully-typed GeneratedScenarioPayload

No JSON parsing, no fallback for malformed model output. The framework's constrained decoding handles it.

A subtlety we hit

The model occasionally produced response options in the wrong order (level 3 listed first, or randomised). Two fixes:

Streaming the chat coach

Synchronous respond(to:) works but feels slow — the user taps "Talk to coach" and stares at a "Thinking…" placeholder for 1-2 seconds. streamResponse(to:) yields a cumulative snapshot of the response as tokens arrive:

return AsyncThrowingStream { continuation in
    let task = Task {
        do {
            let stream = session.streamResponse(to: userMessage)
            for try await partial in stream {
                continuation.yield(partial.content)
            }
            continuation.finish()
        } catch {
            continuation.finish(throwing: CoachError.modelError(error.localizedDescription))
        }
    }
    continuation.onTermination = { _ in task.cancel() }
}

Each partial is the full cumulative text, not a delta, so the SwiftUI view just renders the latest value directly with contentTransition(.opacity). onTermination cancels the underlying task if the sheet is dismissed mid-generation — important for not leaving inference running invisibly.

What about iOS 17-25 users?

Our AI features require iOS or iPadOS 26 or later, a supported device, and Apple Intelligence enabled. The scenario library, six practice modes and bilingual content remain available without AI. Full Access is required for AI Coach and Your Situation when the device supports them.

We considered three approaches:

The "hide it" path means iOS 17-25 users don't see a teasing locked icon for a feature they can't unlock — App Review wouldn't love that either. The paywall is also conditional: the AI Coach feature row only appears when the device can actually run it.

What we'd do differently

The privacy claim, made concretely

"On-device" is a phrase that's been abused. To be specific:

Which means the privacy policy can say what it actually means, instead of the standard "we collect data to improve the service" tap-dance.

Worldly Wisdom is available on the App Store. Technical feedback is welcome at sspkobe@gmail.com.
Back to the site →