Building a keyboard inside iOS's limits

On iOS, a third-party keyboard isn't really an app. It's an extension: a small process the system starts inside whatever app you're typing in, and kills whenever it wants.

It gets a fraction of the memory a normal app gets. It can't use the microphone. It only sees a small window of text around your cursor. And it has to respond to every single keystroke without you noticing any delay.

Keygram runs a language model, a personal n-gram model, a spatial touch model and an autocorrect lexicon inside that box. This post is about what that actually felt like, and why privacy ended up being an engineering decision, not a line in a policy.

The two limits

Time: 20ms per keystroke. Suggestions have to update before your next tap. Fast typists hit keys every 100 to 200ms, and the suggestion strip, autocorrect and touch handling all share that window. I set 20ms as the budget for the whole suggestion pipeline. Anything slower and suggestions show up for the word you already finished.

Memory: much less than you think. Apple doesn't publish a fixed number for keyboard extensions, and it's far below what a normal app gets. Go over it and iOS kills the keyboard mid-sentence, which looks like a crash to the user. Every model, lexicon and cache has to fit, with headroom.

The annoying part is that both limits are easy to pass on your desk and fail on a real phone, in a real app, after an hour of use.

114ms, and where it actually went

The first real measurement was humbling:

Five times over budget on average, sixteen times at p95. And users were taking our suggestions about once in 111 words. The strip was basically decoration.

My first guess was that the model was too big. It wasn't. A single forward pass benchmarked at 0.38ms on an M3. The model was never the problem.

The problem was how many times we called it. Beam search was making up to 96 separate ONNX calls per suggestion, one after another, and copying a big block of output scores back into Swift every time. Each call was cheap. Ninety-six of them, plus the copying, was not.

The fixes were boring, which is usually a good sign:

• Measure the real call count before touching anything, instead of trusting the configured limits
• Batch the beams into one call. This needed the model re-exported with a dynamic batch size, so it was a model export fix, not a Swift fix
• Do top-k inside the model graph, so only a handful of scores cross into Swift instead of the whole vocabulary

Along the way we also found a memory check that was quietly switching the neural model off on device, so the keyboard ran on n-grams alone and nothing told us.

The lesson: profile the pipeline, not the model. A fast model inside a slow loop is a slow model.

Privacy as architecture, not policy

A keyboard sees everything. Passwords, bank OTPs, fights with your family, medical questions. If a keyboard is careless with that, nothing else it does matters.

So the rule from day one was strict: your keyboard text never reaches third-party infrastructure. Not for analytics, not for model training, not to call someone else's AI API.

A privacy policy is a promise. I wanted something harder to break, so the rule shaped the design:

• Personalization happens on the phone. Your spellings and phrases live in a local store on the device. There is no server copy of how you type.
• Models run on the device. The language model, n-grams and autocorrect all run locally. Typing works with no network at all.
• Anything server-side has to be ours. For features that might need a server, like translation, the plan was self-hosted open models on infrastructure we run. No third-party AI APIs touching keyboard text.
• Commercial-friendly open source only. Partly for licensing, partly because I wanted to know exactly what every component does.

The point of building it this way: if a future me gets tempted to add "just a little" telemetry, the system has nowhere to send it.

Your typing rhythm is a fingerprint

This one surprised me. For better autocorrect, we wanted to use keystroke timing: how long between taps. A word you type fast and evenly is probably a habit ("gud", "nhi"). A word you type slowly and unevenly might be new, or a mistake.

Useful signal. But keystroke dynamics is a known biometric. The rhythm of how you type can identify you with decent accuracy, even without reading a single word of content. It's closer to a fingerprint than to a typing log.

So timing data got the strictest treatment in the app:

• It stays on the device and is only used locally by the autocorrect logic
• It is structurally excluded from cloud backup. The timing store has no export path, and the rule is backed by a test that fails if timing fields ever show up in a backup payload

The reason for a test and not a rule in a doc: policies drift, tests don't. Six months later, someone adds a field to the backup, and the build fails before anything leaks.

What I took away

• Measure on the device, early. Everything was fine on my Mac. The phone told a different story.
• The model is rarely the bottleneck. The loop around it, the copying and the loading usually are.
• Make privacy a property of the system. If the data can't leave, you don't have to trust anyone to not send it, including yourself.

Next and last post in this series: what building Keygram taught me, and why it's going into maintenance mode.