In a recent interview, Apple engineering manager Jenny Chen and product manager Ty Jordan provided insights into the development of two groundbreaking features in iPadOS 18: Math Notes and Smart Script. These new capabilities represent the culmination of years of technological advancements at Apple, enhancing the functionality and user experience of the iPad.
Key Features and Origins:
-
Math Notes:
- Handwritten Equation Recognition: Math Notes allows iPadOS 18 to recognize handwritten sums and equations, solving them directly in the user’s handwriting.
- Technological Evolution: Jenny Chen explained that Math Notes builds on the technology developed for Scribble, Apple’s handwriting recognition tool. Scribble converts handwritten text into typed text, and Math Notes takes this a step further by not only recognizing handwriting but also synthesizing and solving mathematical equations in the user’s own handwriting.
-
Smart Script:
- Handwriting Smoothing: Smart Script intelligently smooths out handwritten text, maintaining the unique characteristics of the user’s handwriting while enhancing its legibility.
- Interactive Text Manipulation: Ty Jordan highlighted how this feature allows users to interact with their handwritten text as they would with typed text. For instance, users can tap and hold to move words around, reflow text in a paragraph, and even paste typed text as handwritten text within a paragraph.
The Development Journey:
-
Building on Scribble: Chen noted that the foundation for Math Notes and Smart Script lies in the technology developed for Scribble. By leveraging the ability to recognize and convert handwriting to typed text, Apple was able to advance to recognizing and synthesizing handwritten mathematical equations and improving the legibility of handwritten notes.
-
Enhancing User Interaction: Jordan emphasized the seamless integration of handwritten and typed text functionalities. This integration allows for dynamic text manipulation, such as moving and adjusting handwritten words within paragraphs, akin to editing typed text. This feature brings a new level of flexibility and usability to handwritten notes on the iPad.
Impact and Availability:
-
User Experience: The advancements in Math Notes and Smart Script are designed to enhance the user experience by making handwritten notes more interactive and versatile. These features allow users to maintain the personal touch of handwriting while benefiting from the functionality of typed text.
-
Launch Timeline: Math Notes and Smart Script will be officially launched as part of iPadOS 18 in the fall. Currently, a public beta is available, with several rounds of beta testing already completed with developers.
Apple's continued focus on enhancing handwriting recognition technology underscores its commitment to improving the user experience. With Math Notes and Smart Script, iPadOS 18 promises to deliver powerful new tools for users who rely on handwritten notes, bridging the gap between the personal touch of handwriting and the efficiency of digital text.
