At Apple’s recent “It’s Glowtime” event, the spotlight was on how Apple has addressed the complexities of integrating large language models with cloud services while maintaining strict privacy standards. The key to this achievement lies in Apple’s Private Cloud Compute, a breakthrough technology designed to ensure user data remains secure even when processed off-device.
Achieving Privacy in the Cloud
Craig Federighi discussed the major goal behind Apple Intelligence: extending the privacy guarantees traditionally associated with on-device processing to the cloud. Federighi emphasized that this mission required significant innovation, stating, “We set out from the beginning with a goal of how can we extend the kinds of privacy guarantees that we've established with processing on-device with iPhone to the cloud— that was the mission statement. It took breakthroughs on every level to pull this together, but what we've done is achieve our goal.”
A major challenge with using large language models like ChatGPT involves ensuring that data sent to the cloud remains protected. Typically, end-to-end encryption is used to keep data private, but this wasn’t feasible for processing requests in the cloud. Federighi explained, “What was really unique about the problem of doing large language model inference in the cloud was that the data had to at some level be readable by the server so it could perform the inference. And yet, we needed to make sure that that processing was hermetically sealed inside of a privacy bubble with your phone.”
Innovations Behind Private Cloud Compute
To overcome these challenges, Apple developed what they call Private Cloud Compute, a system designed to keep data secure even when processed in the cloud. Federighi elaborated on the technical aspects, stating, “We had to do something new there. The technique of end-to-end encryption — where the server knows nothing — wasn't possible here, so we had to come up with another solution to achieve a similar level of security.”
Apple’s solution involved several key innovations:
Custom Servers and Operating Systems: Apple built new servers specifically designed for use with their technology. This included creating a custom OS for data centers and developing a trust model that ensures only verified software can process user requests.
Transparency and Trust: The trust model requires that the software running on servers must have its signature published to a transparency log. This ensures that the device will only send requests to servers that are verified and trustworthy.
Federighi praised the implementation, noting, “Building Apple Silicon servers in the data center when we didn't have any before, building a custom OS to run in the data center was huge. [Creating] the trust model where your device will refuse to issue a request to a server unless the signature of all the software the server is running has been published to a transparency log was certainly one of the most unique elements of the solution - and totally critical to the trust model.”
He concluded that the rollout of Private Cloud Compute has been largely smooth, stating, “I think this sets a new standard for processing in the cloud in the industry. And following all of the development work, the rollout of Private Cloud Compute has been delightfully uneventful.”
Apple Intelligence's Challenges in Europe
Despite the technical successes, Apple Intelligence faces regulatory hurdles in Europe. The implementation of this new technology is currently unavailable in the EU due to uncertainties surrounding the Digital Markets Act (DMA). Federighi addressed this issue, saying, “Our goal is to bring ideally everything we can to provide the best capabilities to our customers everywhere we can. But we do have to comply with regulations, and there is uncertainty in certain environments we're trying to sort out so we can bring these features to our customers as soon as possible.”
He added that Apple is committed to localizing Apple Intelligence in France by 2025 and is actively working to resolve regulatory challenges. “So, we're trying,” Federighi concluded, indicating Apple’s dedication to navigating these complexities and expanding their services in compliance with local regulations.