Apple discusses iPadOS 16.4’s new Pencil hover features

In December of last year, Apple announced that hover was being added to the Pencil’s bag of tricks. What might sound like a minor update to non-artists represents an important addition to workflow. While digital drawing is more forgiving in terms of correcting errors than its real-world counterpart, stopping and starting is still an extremely frustrating dimension that can ultimately hamper the creative process.

Hover adds the ability to preview lines before committing them to a piece. By hovering a Pencil 2 up to 12 millimeters above the new iPad Pro (Apple notes that it’s the M2 that makes the feature possible, hence the limited hardware options), the system offers line and line width and color previews, along with the ability to choose drawing implements in Markup.

“The pencil is emitting signals that the iPad is then interpreting and figuring out the location of the tip of the Pencil in 3D and the angle which it’s being held at,” Apple’s Director of Input Experience Leslie Ikemoto tells TechCrunch. “That gets complicated the further away you are from the screen, but we did a number of studies of how people hold their Pencil when they’re resting and hold their Pencil when they’re drawing. We have determined we needed 12 millimeters. That’s what our process gets us, that ability to detect these signals, even from 12 millimeters away.”

Savage Interactive, which makes the graphics editing and digital painting iOS/iPadOS app Procreate, seemed especially bullish at the time of the product’s release. “It’s truly made a profound impact in our design phase,” CEO James Cuda noted at the time. I imagine such things make an especially big difference when your app is an iPad/iPhone exclusive.

“IPad is, first and foremost, a touch-first device. You hold this sheet of glass,” says Director of Platform Product Marketing Stephen Tonna. “As you add different accessories to it, a keyboard or a trackpad, it unlocks different capabilities. The Pencil adds to the incredible experience. “That’s another way to think about the Pencil and iPad.”

This morning, iPadOS 16.4 arrives with further refinements to the feature, in the forms of tilt and azimuth. Tilt is effectively what it sounds like. Azimuth is a new one to me (and, it turns out, our staff designer, Bryce). Given how poorly I did in high school geometry, I’m going to do the most high school term paper thing I’ve ever done on TechCrunch and quote the Oxford Languages dictionary:

The direction of a celestial object from the observer, expressed as the angular distance from the north or south point of the horizon to the point at which a vertical circle passing through the object intersects the horizon.

Those tools are now open to developers looking to build on top of existing Pencil functionality. “If you look at Procreate, they have the pencil brush, which is small and thin when you’re perpendicular to the display, and then gets wider as you tilt for shading,” says Ikemoto. “With tilt and azimuth, Procreate can render an exact outline of the mark that you’re going to make when you set down your pencil brush, and that’s a huge accelerator for their users.”

The company got into the tablet stylus game around 7.5 years ago, announcing the first Pencil alongside the original iPad Pro. The Pencil 2, meanwhile, arrived alongside the third-gen Pro three years later, bringing wireless magnetic charging to the table. Apple doesn’t comment on roadmaps or the development that goes into new and upcoming features, of course. But along with the aforementioned studies, the company says updates are, in part, informed by user feedback. Artists are, after all, often a particular bunch, committed to specific workflows developed over time.

“We’re always listening to customer feedback,” says Tonna. “One important part of building great products is taking that feedback and listening to what’s going on. And, of course, building things our customers didn’t even think of. They see it, they love it, they take it further. We listen.”

The features are made available to developers through various APIs. “We wanted hover to be as easy to adapt as possible, so we used the existing APIs that we used for the Magic Keyboard Trackpad,” Ikemoto adds. “That’s UI pointer interaction and UI hover recognizer. For developers who have already adopted UI pointer interaction in their app, they get Apple Pencil hover for free without doing any work. It will just work. Developers who want a more custom experience with hover can use UI hover gesture recognizer, which we’ve extended to report the location of the tip of the Pencil in 3D and also the angle at which the Pencil is being held.”

Procreate, naturally, is an early recipient of the new tech. “We’ve been really looking forward to the possibility of Apple Pencil hover ever since the Apple Pencil launched,” Procreate Chief Product Officer Claire d’Este said in a comment to TechCrunch.

The company tells TechCrunch, however, that drawing and painting on the iPad isn’t a more popular activity for the Pencil than handwriting. The hover feature works with a variety of apps, as well, including Notes and Safari. It accesses similar tool and preview features as the kind you’ll find in places like Procreate.

“You have this device that you can lay down on your table and take notes in a lecture, and then you can put it in a Magic Keyboard when you’re done,” says Tonna. “You can use the keyboard to write your term paper. […] You can take those handwritten notes and paste them right into your paper, and they turn into typed text.”

Working on a product line invariably involves a fair amount of dogfooding. For Pencil, that means making sure there are some amateur artists on the team, actually using new additions to the line before rolling them out more widely to developers and users.

“The team participates every October in Inktober, where we draw or paint a different concept every day,” says Ikemoto. “Inktober is a big event in the artistic community. There’s a word published a day, like ‘plant,’ and everyone will draw and paint a plant. It’s a great way to share that artwork across the team.”

The freshly arrived iPadOS 16.4 also brings Safari web app push notifications for the home screen, 21 new emojis and assorted bug fixes.