On-Device Models: The Business Is in the Ground Apple Doesn't Cover

Jiang Wu · 2026-09-16

On July 3, 2026, Desert Ant Labs, a Dutch company, published its own model license: each model, on each platform, is free to use up to 100,000 monthly active devices, and beyond that a commercial license is negotiated. There are two conditions. A line reading "Powered by Desert Ant Labs" must appear somewhere users can see it, and its models and outputs may not be used to train competing models. The license is perpetual, can be amended, and can ship inside an app.

The company has left only a faint trace. Its GitHub organization was created on June 14, every repository dates from July or later, and the main SDK, desert-ant-core, has 13 stars. The official press page gives its founding as August 2026, and the first blog post came on September 3. Five people in total have commits, and I found no funding announcement.

The founding team is three people: Paul Veugen (CEO), Laurier Rochon (CTO), and Fredrik Wallin (engineering). Together they built Human, which Mapbox acquired in 2016, then the video creation app Detail (named 2025 iPad App of the Year at the App Store Awards) and the video platform Subwave. The company says these models already run in those two products.

What it has released is 12 usable models and 6 in closed beta, each doing one thing: speech enhancement, language identification, titling, PII redaction, topic tagging. They share no common base. Sizes run from under 1 MB to 467 MB, and none reaches the GB scale.

On its about page it describes itself as "a small cerebellum embedded in every product." The cerebellum is a tenth of the brain's volume and holds four fifths of its neurons, and it governs balance, timing, and coordination, movements that are fast and reliable and need no thought once learned. It says it wants not the largest model but "the fastest one for the job," because "people carry more compute in their pockets than data centers have."

The door did open, and Apple opened it

In February 2024 I wrote a piece on local models (Chinese) with three judgments: the decision to use a local model hinges almost entirely on cost, phone processors sit idle most of the time, and apps would sooner or later move to a mix of cloud and local. In October that year, when the Snapdragon 8 Elite launched, I wrote another (Chinese), saying that once chips packaged the interfaces and capabilities for apps to call, the on-device door would be open.

Two and a half years later the door has opened, though Apple opened it and not Qualcomm, and more completely than I expected then.

The Foundation Models framework ships with the system from iOS 26 and macOS 26: an on-device model of about 3 billion parameters with a 4,096-token context, supporting Simplified Chinese, structured output, and tool calling, and able to load a self-trained LoRA adapter. Developers pay nothing and apply for no API key. The version announced at WWDC 2026 and landing with iOS 27 goes a step further: the context doubles to 8,192 tokens, image input is added, and a model abstraction layer opens so that third-party cloud models and local MLX models plug into the same Swift API. A chipmaker opening an NPU interface is one thing, and an operating system putting a model in front of every developer is another.

The door comes with three thresholds.

The first is system version. Foundation Models, SpeechAnalyzer for transcription, and RecognizeDocumentsRequest in Vision for structured document recognition are all pinned to the 26 line. iOS 27 was released in mid-September, so the bar only rises with new versions.

The second is the device. It takes an iPhone 15 Pro and the A17 Pro chip at minimum. Counterpoint's figure is that Apple has shipped more than 450 million Apple Intelligence-capable iPhones cumulatively (through Q1 2026), while Apple itself reports more than 2.5 billion active devices worldwide. The two can't be divided directly, since one is cumulative iPhone shipments and the other is active devices of every kind, but the order of magnitude shows that system-level on-device models cover far from all devices.

The third is region. iOS 27 shipped on September 14 with the new generation of Apple Intelligence, and the support page Apple updated that day is blunt: for devices bought in mainland China, these features are not available for now. Earlier, on July 15, Apple received filing clearance from the Cyberspace Administration of China, with the cloud portion handled by Alibaba's Qwen, but a filing is no launch. In the early hours of March 31 some China-market iPhones accidentally received the push, and it was withdrawn in under 6 hours. For Foundation Models to work, three conditions must hold together: device support, user opt-in, and regional availability. If the third fails, the first two make no difference.

The EU offers a useful comparison. The Foundation Models framework itself works normally there, while iOS 27's new Siri is not launching in the EU for now because of the DMA's requirements on system permissions for third-party assistants. The same layer of capability from the same company takes a different shape in each jurisdiction, so this foundation is political in itself.

The other road is open-source runtimes: llama.cpp, whisper.cpp, sherpa-onnx, MLX, and ONNX Runtime. It works, at the price of every indie developer moonlighting as an ML integration engineer. In several apps I maintain myself, three or four inference runtimes run at once, each with its own model format, quantization strategy, and memory curve. Model weights are hosted on Hugging Face by default and users in mainland China can't pull them, so a mirror and a verification process need to be prepared separately, and if I want to swap a model, all of this is done over again. None of it has anything to do with the product, and without it the product doesn't run.

So Apple's road is free but gated, and the open-source road has no gate but you build the road yourself. Desert Ant Labs wants to stand between the two.

Parts and models are different businesses

Laid out, its model catalog is surprisingly narrow, model by model.

Clear does speech enhancement, with 9 MB of Core ML weights based on DeepFilterNet 3, independent of language. Redact does PII redaction at 11.6 MB. Tongue identifies the language of text, 84 languages at 2 MB, built into the SDK package so it needs no download. Gist tags topics across 36 fixed categories at 74 MB. Schemer does structured extraction with 211 million parameters, 111 MB after int4 quantization. Title writes titles and descriptions, fine-tuned from IBM Granite 4.0's hybrid architecture, about 286 MB after 6-bit quantization. Align is only 0.7 MB and exists to refine word-level timestamps.

Voz best illustrates the approach. It transcribes, built on NVIDIA's Parakeet TDT 0.6B v3, converted to Core ML at 467 MB, resident on the Neural Engine, with peak memory that does not grow with audio length. The stated figure is 10 minutes of audio transcribed in 2 seconds on an iPhone. On quality, its own reported average WER across six Open ASR Leaderboard datasets is 7.40%, against 7.00% for Whisper large-v3-turbo, a gap of 0.4 percentage points, at less than a third of that model's 1.6 GB. That is the official basis, and the direction of the trade-off is clear: it is not chasing first place, it is chasing fitting into a phone and still running.

Integration follows that shape: native SDKs in Swift, Kotlin, and JavaScript for iOS 18+, macOS 15+, and Android API 24+. Models download from Hugging Face at a pinned revision with verification, or can be bundled into the app or pointed at a self-hosted address. The selling point it stresses is simple: a few lines of code.

What really separates it from model companies is the unit of billing: by monthly active devices and not by calls.

Behind that is a physical fact about on-device inference. Cloud models charge by token because every call consumes the vendor's GPUs. On-device inference runs on the user's own chip, so the vendor's marginal cost is zero, and per-call charging loses its matching cost. The one thing left to meter is how many devices it is installed on.

So the chain runs like this: free below 100,000 devices in exchange for distribution, the "Powered by Desert Ant Labs" credit in exchange for brand, and charging only starts past 100,000. It is the developer-first playbook of Twilio and Stripe moved onto the device: let developers run it at zero cost first, then charge at scale. I once wrote about how companies that don't build models grow another layer on top of them, on how cloud middle layers accumulate position over model vendors. Desert Ant accumulates position by the same logic, only it stands between the operating system and developers, and what it prices is the device and no longer the call.

Billing per device also carries an unavoidable cost: someone has to count devices. Clause 7 of the license states that the SDK runtime sends a self-generated identifier to its servers, used only to count monthly active devices, containing no input or output and not linked across apps. Developers must disclose it truthfully in their own privacy policy and may not interfere with the report. On-device does not mean zero network. The content really does not leave the device, but the fact that "this device has this model installed" does. For products with "all processing is local" on their front page, this clause needs more forethought than the credit requirement.

One more clause is easy to skip: the license is perpetual, amendable, and distributable with the app, the weights can be self-hosted, and versions can be pinned. So even if this company disappeared tomorrow, apps already shipped would not break. They would simply stop getting updates. For a company with under three months of public trace and no disclosed funding, this clause matters more than any technical metric, because it turns the risk of depending on a small company from uncontrollable to estimable.

"Can't reason" is the selling point here

The difference between discriminative small models and generative models lies less in parameter count than in what each can guarantee.

Schemer is the clearest case. It does structured extraction: given a text and a schema, it fills the fields. It makes three guarantees. Field types are guaranteed by construction, with no prompting the model to cooperate. If a field can't be found in the source, it returns null instead of inventing a plausible value. And every output is a substring of the source with character offsets, traceable back to its place in the original.

The third matters most. A generative model's output is sampled, so you can't prove that "this sentence the model said is that sentence in the source." Discriminative extraction hands back a pointer into the source that can be audited word for word. The internal figure reported officially is 0.91 on missing-field detection, against 0.18 to 0.43 for the LLMs it tested. That is the company's own claim and no third party has reproduced it yet, but it points at the same thing: when a general model is asked to fill a blank, it tends to fill it.

On specs, it uses 200 MB of memory at runtime, takes 8.7 milliseconds per encoding on the Neural Engine, and caps text at about 1,100 tokens. You can't chat with it, and it never meant to be chatted with.

The about page also carries an energy ledger: editing 100,000 thirty-minute videos takes 0.3 kWh with its Clips model and 140 kWh with Claude Sonnet, a 470-fold difference, with blind-test scores level. This too is the official basis, and the method is stated openly: the Sonnet side uses measured token counts from 38,308 tasks plus an MLPerf estimate, and the Clips side assumes 2.4 seconds and 3 W on an iPhone 17 Pro. There are many assumptions, and precision to the decimal point means little, but the order of magnitude will not be wrong. A dedicated model of a few hundred MB doing one narrow job and a frontier model doing the same job are separated by orders of magnitude.

More than parameters, Title shows the character of this company. Its documentation says it deliberately produces no emoji, no hashtags, and no clickbait. That is no capability gap. It is a product decision made at training time: what a model's inputs and outputs look like is itself part of product design, and prompts and post-processing are not there to patch it after the model has spoken. Anyone who has built AI features inside an app knows post-processing can never fully clean up emoji.

Its technical choices are unfussy too. The same document explains why only Title runs on MLX: generative decoding is bandwidth-bound, and Core ML trails on time to first token, throughput, load time, and resident memory. The other discriminative models all run on Core ML, resident on the Neural Engine. A five-person team willing to maintain a separate runtime for one model means it takes "use the best part for each job" seriously.

The ground outside Apple's range

The space left for third-party on-device models falls roughly into three kinds.

One is regions Apple does not serve. Mainland China is the big market Apple Intelligence has still not entered. On China-market devices and mainland Apple IDs, third-party apps calling Foundation Models will most likely get "unavailable," and the block is the region and not the device.

Another is devices and systems Apple does not serve. Devices still on iOS 18, models that can upgrade to iOS 26 but aren't Pro, and Intel Macs are not covered by Apple's line at all. Desert Ant's SDK needs only iOS 18+ and macOS 15+, one generation below Apple's (after iOS 18 the next version is iOS 26), which falls into exactly that range.

The third is tasks Apple does not do: structured extraction, speech enhancement, speaker separation, filler-word detection, and word-level timestamp refinement are not in the system APIs. Align is the plainest example. At 0.7 MB, its only use is refining the word-level timestamps from Apple's own SpeechAnalyzer, which the company says brings average error on clean audio from 113.5 ms down to 44.9 ms. Its documentation even teaches how to chain Apple's transcript with it. It was a patch from the start and no replacement.

This ground is not stable. Apple clears it once a year, at WWDC.

On iOS 26, SpeechAnalyzer's WER on an English earnings-call dataset, as measured by Argmax, is 2.12%, better than Whisper Small, Base, and Tiny in the same test (Whisper Small is 3.74%). That test covers English only, but transcription in English is already mostly handled by system APIs, and Voz's differentiation shrinks to working on iOS 18 and word-level timestamps. iOS 27's Foundation Models adds image input, which will squash a stretch of third-party vision parts in passing. Redact faces the long-existing NSDataDetector in the system, and Gist faces a text classifier that NaturalLanguage and Create ML can train, both within Apple's range.

So the lifespan of a third-party part roughly equals the time it stays outside Apple's range. For Desert Ant that is a risk, and for the developers using it a design constraint: any third-party on-device model should be built as a replaceable layer, so the core logic of the product doesn't grow onto one particular model. It counts as a part only if it can be taken off.

The constraint holds for versions too. desert-ant-core went from v0.5.0 on July 27 to v3.1.0 on August 28, 15 releases in a month, jumping from v1.2.0 to v3.0.0 on August 19 alone. A September 2 commit pinning Align's weights to a fixed version shows that, for a while, the weights floated with a branch. On-device does not mean behavior stands still, and nobody pins versions for you.

Its biggest open ground is one it hasn't reached

Go through the language lists and something looks fractured.

Voz supports 25 languages, all European, with no Chinese. Redact supports 27, also all European, with no Chinese. Uhm, which detects filler words, supports only German, English, Spanish, French, and Dutch. The three most presentable models all lack Chinese, Japanese, and Korean.

The ones that include Chinese sit at the edges. Tongue and Ear do language identification (84 and 99 languages). Gist covers 101 languages but has only 36 fixed categories aimed at posts and articles, a granularity that doesn't fit real content on personal devices. Align's 9 languages include Chinese, but it requires iOS 26, which puts it back on Apple's line. Schemer's 13 languages include Chinese and Japanese, but Schemer is still in closed beta, with weights public and no SDK.

Its website, meanwhile, is localized into 12 languages, including Simplified Chinese, Japanese, and Korean. CJK developers are clearly among its target customers, and model coverage hasn't kept up.

The position of this gap is a little delicate. Apple Intelligence has not entered mainland China, which should have been the biggest blank for third-party on-device models, and it is exactly the area this Amsterdam company cannot reach right now.

The ones likely to fill it are Chinese teams, and the parts are in fact already on the table. SenseVoice (Alibaba) does speech recognition for Mandarin, Cantonese, and dialects. sherpa-onnx (k2-fsa, linked to Xiaomi) packages transcription, speaker separation, speech enhancement, and VAD into a cross-platform offline runtime. FireRedASR (Xiaohongshu) does industrial-grade Chinese ASR. RapidOCR and the PaddleOCR line do on-device OCR, and Qwen's small models provide a general base. These are no weaker in capability than Desert Ant's catalog, and many are fully open source and commercially usable.

What's missing is the productization layer. They are still "a model, weights, and a README," and developers have to choose, quantize, wrap for mobile, and carry model updates and version drift themselves. Desert Ant offers "a few lines of code, per-device billing, pinned versions, and a consistent SDK on three platforms." The same capability is loose material on one side and an assembled part on the other. Nobody I can see is seriously turning Chinese on-device capability into such a parts box.

The contest is who gets installed on more devices first

Model companies compete on the ceiling of intelligence, pushing it up with parameters and data. Parts companies compete on who gets installed on 100,000 devices first, which depends on how low the friction of the SDK is. Those are two different curves.

The hybrid pattern from the piece two and a half years ago did arrive, but the dividing line is not where I thought. Then I thought the line would fall between cloud and device, deciding which tasks go down to the device and which stay in the cloud. A closer line to reality is another one: what Apple provides and what it doesn't. The first is free, with three gates of system version, device, and region. The second is the entire living space of third-party parts.

Whether Desert Ant Labs itself survives, I don't know yet. Five people, no public funding, no public customer cases beyond its own products, and almost no independent discussion on Hacker News or the Chinese internet: any one of those could make it vanish. But it has drawn the shape of this layer clearly: a model narrow enough to do one thing, size squeezed to tens of MB, an interface narrowed to a few lines of code, and billing changed from calls to devices. The shape is one thing, and the company's survival is another.

The real thing to watch isn't in Amsterdam. Every September Apple stuffs more free capability into this layer, and the room for third-party parts gets thinner each year, leaving only the corners Apple can't enter. Whoever first turns Chinese on-device capability into an SDK that installs in a few lines of code gets the market Apple has not yet entered. That window will not stay open, and so far there is no queue at the door.

Sources

Home · All articles · Topics · 简体中文