Three Things to Watch from Apple's WWDC 2024

Jiang Wu · 2024-06-11

*Originally published in Chinese on June 11, 2024.*

Apple's WWDC 2024 keynote covered a long list of system, hardware and service updates. I will group them by type first, then pick out the three details I think matter most.

Regrouping the keynote

I set aside the keynote's own order and sorted the announcements into systems, apps, hardware and new services.

Across these product and capability updates, Apple's decisions took the pace of hardware and technology development into account together. The more central point is how it thinks about the relationship between hardware and AI: AI added to hardware, or a new kind of AI hardware. That is a question of standpoint.

Open or closed is a trade-off

The biggest highlight this time is the interface that lets Apple and model makers call each other. As things stand, the capability and value that models give applications exceed what operating systems used to give them, and that raises the user experience at a basic level. Siri's old goal was the same: build an integrated service for users at the system level.

A hardware maker has to avoid becoming only an input and output terminal for data, one that does nothing to improve how the application is served. That is a delicate choice. Users need the hardware, but the need adds nothing to the hardware's own value, since models or applications deliver the whole service. With the imagination around hardware shrinking and its bottlenecks getting worse, hardware makers do not want that outcome.

In the past a closed system held a monopoly on matching hardware and system services into one complete experience. But if models can create newer experiences or change how users work through the whole process, and the system and ecosystem built on hardware cannot give users more than they expect, users may move to a new platform.

So hardware makers have to think about how they relate to models. The core is how a hardware maker defines the relationship between its own hardware capability and a model maker's capability. Apple's logic is to open the interface: you come and connect, and the two sides cooperate to some degree.

My feeling is that this is a long-term state, and the future depends on how hardware capability and model capability each iterate.

Where the privacy boundary sits

Apple's other effort is to build a layer of privacy isolation for user data into the hardware. Hardware is the terminal that collects data and the first carrier of it. A user who owns the hardware should in theory have their rights over that data protected. Most users may not be aware enough to protect their data privacy, so the hardware wraps a shell around it to help.

Take an example. In future, when I want to ask a model about a detail in a photo, do I send the photo directly to the model, or does the hardware wrap a protective layer and pass only the key data to the model? They look like two ways of processing, but the privacy requirements behind them are of completely different orders: one has almost no privacy, and the other exposes almost none.

This will become more prominent. User voice, images, video and everything else collected through the hardware interface faces the same choice: send it straight to the model maker (or application maker), or bridge it through a privacy protection system that provides only the key elements needed to serve the user. Beyond data handling, different approaches will face different regulatory review and different privacy standards in different scenarios, and they may also affect how models improve and how accurate their service to users is later. The bargaining over privacy and experience may soon be between users and models directly, not between users and applications.

Today Apple looks ready to step in itself, to deal with this part of the privacy problem and define the privacy boundary between user data and models.

Trying out on-device models

Another thing to watch at this keynote is the logic of trying on-device models. Most of the time a phone's processing power is capped and underused. Models and applications choose the cloud today partly after weighing user experience and results. A hardware maker that exposes its on-device capability can cut part of an application's service cost, improve the user experience and increase how much applications and users depend on the device.

At this keynote I did not see Apple package its phone's processing power into APIs for other applications to call, or at least it was not obvious. But Apple should now focus on optimizing on-device capability and exporting it through APIs.

For example, some of the image-processing capability in the built-in Photos app is integrated only into Apple's own system apps today. Could it become an interface for other applications to integrate, or let them use local capability directly, such as image recognition and translation, background removal, even higher-level content recognition and output? I believe Apple is willing to open this up. Microsoft has already made its approach public, which to a degree shows the path is feasible.

The saddest part is that, in the past, whenever Apple upgraded a system capability, a pile of applications announced the end of their service. In the model era, that will be the norm.

Home · All articles · Topics · 简体中文