Apple Vision Pro: Here’s everything you need to know | TechCrunch

At its WWDC 2023 conference, Apple took the wraps off the Vision Pro, its long-rumored augmented reality (AR) headset. The Vision Pro is one of Apple’s most ambitious products to date — one year in the making and packed with technology, including a new operating system, visionOS, that might just warrant the sky-high price tag.

It’s nearly arrived. The Vision Pro goes on sale Friday, February 2, and preorders are currently open on the Apple Store.

Despite the imminent launch, there’s a surprisingly small number of natively developed apps for the Vision Pro — and big omissions like Netflix, Spotify and YouTube. That being said, early headset owners can expect content such as 3D movies and films, streaming services like Disney+ and 180-degree recordings captured with Apple’s Spatial Audio tech.

It’s a lot to keep track of — which is why we created a handy, condensed guide to everything Apple announced around the Vision Pro and visionOS.

Apple Vision Pro has a ski goggle–like design

The first thing you’ll notice about the Vision Pro is the chassis, which evokes a pair of high-end ski goggles.

A tinted, continuous front panel wraps around the Vision Pro wearer’s eyes, acting as a lens and concealing a fan that draws air through the headset to cool the electronics. A “light seal” — magnetically attached to the headset — keeps ambient light from leaking in.

Around the back of the Vision Pro, there’s a button and digital crown for fine-tuning, navigation and calibration (although pupillary distance calibration is handled in software), and a band that can be easily swapped out for one of several included in the box.

The Vision Pro has two displays (plus an external OLED panel), one for each eye, with 23 million pixels across both. A custom 3D lens ensures that the UI always remains in view, while features like HDR and “wide color” deliver an ostensibly superior image.

Built-in earbuds pipe in audio for wearers who don’t opt to plug in a pair of headphones. They support specs like Dolby Atmos and lossless audio, and leverage “audio ray tracing” to analyze the acoustic properties of the surroundings to optimize sound for a space, Apple says.

Zeiss worked in partnership with Apple to create magnetic lenses for prescription wearers, which will be available for custom order through the Apple Store.

IR cameras inside the Vision Pro track a wearer’s eyes, while downward-facing cameras on the exterior of the chassis track their hands. A third set of sensors — lidar sensors — sense objects and people around the Vision Pro in real time, keeping tabs on their positions.

Speaking of, Brian reports that the hand tracking on the Vision Pro is quite good. You don’t have to lift your hands — just ensure that they’re not occluded from the line of sight. Using the spatial computing element gives the effect of picking up app windows and moving them around in front of you, while the Vision Pro’s people-detection feature automatically reveals people’s figures when you turn your head toward them with an app open.

All of the Vision Pro’s sensors — 23 in total, including a dozen cameras, five sensors and six mics — feed into the R1, a new Apple-originated chip designed to “eliminate lag” and stream images to the headset’s display “eight times faster than the blink of an eye” (as per Apple PR). The Vision Pro’s passthrough effect has a little latency (around 12 milliseconds) and blurriness — but it’s still the best version of the tech Brian has experienced on a headset, he says.

Apple Vision Pro Apps at launch

At launch, the Vision Pro will feature apps — including Unity apps, which run natively on Apple’s headset — from Adobe (specifically Lightroom), Microsoft (Office) and other major developers.

Apple’s own apps are also available for the Vision Pro, as are others you may know, like Temu, Uber and Uber Eats, Tinder, Notion, CNN, Washington Post, Reddit and Discord. To spur the development of more, Apple’s launched a dev kit program for approved account holders in the Apple Developer Program.

Now, out of the box, the Vision Pro can run most iPhone and iPad apps, which can be installed in the standard way from the App Store (unless developers choose to hide them). That’s good news for wearers who plan to use YouTube, Netflix and Spotify, none of which currently offer native Vision Pro apps. But these emulated iOS and iPadOS apps won’t perform like native Vision Pro apps in the sense that they won’t tap many of the headset’s more powerful functions, like motion tracking.

Apple apps

The Vision Pro, being an Apple product, ships with native Apple apps upgraded to take advantage of the headset’s immersive capabilities.

There’s Vision Pro-specific versions of Calendar, Mail, Messages and FaceTime to help you keep organized and in touch with people while you’re wearing the Vision Pro. In the case of FaceTime, the virtual persona used in the Vision Pro’s EyeSight feature (more on that later) stands in for your actual face, approximating the gestures and expressions you make while wearing the headset.

On the productivity and entertainment end, Apple’s releasing enhanced Messages, Safari, Freeform, Notes and Files experiences as well as media viewers (i.e., the Photos, Music and TV apps). Keynote, the company’s presentation software, is present and accounted for also — but no other apps from Apple’s iWork suite just yet.

The Vision Pro’s app catalog features the standard array of enterprise meeting apps, namely Microsoft Teams, Webex and Zoom, all of which support the Vision Pro’s virtual avatars.

Zoom’s app is particularly robust, taking advantage of the Vision Pro’s AR capabilities to blend in with users’ physical environments while surfacing as a floating window. Future updates will add “real-world pinning,” 3D object sharing and Zoom’s Slack competitor, Team Chat. (Slack, speaking of, has a native — albeit bare-bones — app for Vision Pro).

Microsoft is contributing apps beyond Teams to the Vision Pro’s cohort, specifically Word and Excel. Word’s and Excel’s UIs on the headset use a simplified ribbon interface and can be controlled with your eyes.

Among the early crop of third-party apps for the Vision Pro, there’s:

• Fantastical, a calendar and task app to organize your workflow.
• JigSpace, an AR app that lets you create 3D presentations with exploded views, animated slides and cutaway images to see inside objects.
• PCalc, a port of the iPad calculator of the same name.
• Sky Guide, which lets you explore constellations in AR.
• Djay, which puts a turntable at your fingertips in AR.
• Complete HeartX, an anatomical model simulator with realistic renderings of organs.
• OmniPlan, the long-standing project management app.
• Bills to Budget, a budgeting app with bill-tracking features and more.
• Subjects, a school planner with organizers for homework and assignments.
• TV Remote, a remote app for popular TV sets and set-top boxes.

Games

Beyond apps, hundreds of Apple Arcade titles will be available to play on visionOS on “day one,” Apple says. And there’s other games besides.

See Demeo, a mixed-reality Dungeons & Dragons–inspired game with animated pieces, and Just Hoops, which lets you play arcade-style basketball with free throws in your living room. There’s also NowPlaying, a music trivia app that links up with Apple Music to give you fun facts about your favorite artists.

One of the day-one Apple Arcade titles for the Vision Pro is What the Golf?, a companion to the previously released What the Car? A puzzle game about golf, What the Golf? requires you to use a club, your own body and a host of other items to get the ball in the hole.

As for Super Fruit Ninja, it takes the classic Fruit Ninja formula — fruit-cutting katanas — and adds eye and hand tracking to the mix.

Apple Vision Pro TV and movies at launch

New TV and movie experiences are launching with the Vision Pro, both from Apple and third parties.

Apple’s introducing a new format, Immersive Video, that features 180-degree 3D 8K recordings captured with spatial audio (more on that below).

The Immersive Video content available today includes:

• Alicia Keys: Rehearsal Room, which offers a glimpse into the artist’s creative process with a rehearsal session featuring renditions of “No One,” “If I Ain’t Got You” and “You Don’t Know My Name.”

• Adventure, which follows athletes as they take on challenges in some of the world’s most “sensational” locations.

• Wild Life, an up-close-and-personal view of wild creatures and what makes them unique.

• Prehistoric Planet Immersive, a spiritual sequel to the Favreau series on Apple TV, Prehistoric Planet, following the struggles and triumphs of dinosaurs.

Vision Pro users will be able to access 3D versions of movies when they become available to rent or purchase from the Apple TV app, Apple says, and users who own or purchase movies with a 3D edition can access that version on the Vision Pro at no cost. Several streaming apps, including Disney+, will stream 3D versions of their latest and most popular movies on Vision Pro at launch and will debut new 3D editions alongside or shortly after the standard 2D releases.

Wearers will find an array of natively built third-party streaming apps on the Vision Pro, including MLB, ESPN, PGA Tour, Max, Discovery+ and Paramount+ in addition to Peacock, Pluto TV, Tubi, Fubo, Crunchyroll, Red Bull TV, IMAX, TikTok and MUBI.

Disney+ will also natively support the Vision Pro at launch, with its app offering a selection of “3D” movies to watch such as “Avatar: The Way of Water”; “Avengers: Endgame”; “Star Wars: The Force Awakens”; “Elemental”; and “Encanto.” The app optionally projects background scenes that wrap around you while videos play, with the four choices being Disney+ Theater, the Scare Floor from Pixar’s “Monsters Inc.,” Marvel’s Avengers Tower and the cockpit of Luke Skywalker’s landspeeder.

What is the Vision Pro’s EyeSight feature?

The Vision Pro’s EyeSight feature is intended to, as Apple explains in a press release, “help users stay connected to the people around them.” When a person approaches the Vision Pro, the headset looks transparent — letting the wearer see them while also displaying the wearer’s eyes.

The R1 powers EyeSight, which essentially projects a virtual version of the wearer’s face — an avatar Apple calls a “persona” — onto the Vision Pro’s aforementioned external display. The personas, created during the Vision Pro’s initial setup, are impressively high-fidelity, with simulated skin and muscle tension and a full range of facial contortions (e.g., eyes blinking) animated with the help of AI models.

When a wearer’s using an app, EyeSight gives visual cues to lookers-on about what the user’s focused on. EyeSight also includes a visual indicator that makes it clear when a wearer’s capturing a photo or video.

For example, when a wearer’s looking at an app, a bluish light appears. When they launch an AR “environment,” the screen turns into an opaque shimmer. And when they take a photo, the screen flashes like a shutter.

What is the Vision Pro’s Optic ID feature?

Because Face ID and fingerprint scans in AR would almost certainly be an exercise in futility, Apple’s introducing a new biometric authentication system, Optic ID, with the Vision Pro.

Optic ID analyzes a user’s iris under invisible LED light to unlock the Vision Pro, autofill passwords and complete payments with Apple Pay. Apple says that Optic ID data is protected by an encrypted hardware chip in the Vision Pro and that it’s not accessible to apps — and never leaves their device.

What is spatial video and spatial audio?

The Vision Pro can play back spatial video, the 3D video format that the iPhone 15 Pro and iPhone 15 Pro Max — and the Vision Pro itself — can capture. While spatial videos appear as regular 2D videos on the iPhone and other devices, on the Vision Pro, they can be expanded into an “immersive” view.

Spatial videos are recorded with spatial audio, Apple’s binaural audio tech. Thanks to dynamic head tracking, spatial audio can deliver what Apple describes as “theater-like” sound from the video you’re watching — so sounds seem like they’re coming from all around you.

Spatial audio isn’t new — it’s supported on many of the more recent iPhone, iPad, Mac, Apple TV and AirPod models. But paired with spatial video on the Vision Pro, it seems poised to deliver pretty novel experiences in AR.

What is the Vision Pro’s battery life?

Unlike some AR headsets on the market, the Vision Pro’s battery isn’t built into the chassis itself. A five-foot wire running from the rear of the headset connects to a pocket-sized, USB-C-compatible pack, which provides up to two hours of use. (Brian writes that the battery pack’s long enough to run behind you while you sit.)

Apple quotes a battery life of about two hours of “general use” and two and a half hours of video watching. One presumes that more demanding apps and games will sap the battery faster.

The Vision Pro can be plugged into a wall outlet for an extended, battery-free experience and be used while charging — but that of course requires dealing with a tether.

visionOS

The software powering the Vision Pro is visionOS, which Apple describes as “the first OS designed for the ground up for spatial computing” — spatial computing being Apple’s phrase of choice for AR and virtual reality experiences. At the architecture level, visionOS shares core blocks in common with MacOS and iOS but adds a “real-time subsystem” for processing interactive visuals on the Vision Pro.

The three-dimensional interface of visionOS frees apps from the boundaries of a traditional display so that they can appear side by side at different scales. The UI responds dynamically to natural light, casting shadows to help communicate scale and distance.

As one would expect of a PC-level operating system, visionOS supports accessories like keyboard and trackpads, and — beyond those inputs — responds to finger and wrist gestures, and tracks one or both eyes for app focusing and app window selection.

VisionOS also supports SharePlay, Apple’s tech that lets wearers watch TV shows and movies, listen to music and play games from the App Store with users on other Apple devices. A “Travel Mode” stabilizes the Vision Pro’s visuals for use on planes, while Guest User shares specific apps and experiences with family and friends, like Photos or Safari.

Environments are perhaps visionOS’ standout feature. They surround the Vision Pro wearer with “dynamic” landscapes — like Haleakalā, Mount Hood, movie theater, Joshua Tree and Yosemite national parks and the surface of the moon — that evolve based on the time of day, aiming to help with focus or reduce clutter in busy surroundings. Twisting the digital crown controls the level of immersion and volume of ambient sounds.

Where and how to buy the Apple Vision Pro

The Vision Pro is available starting February 2 in the U.S., priced at $3,499 for 256GB of storage, from the Apple Store (online or in-person), with more countries to come later in the year. Zeiss optical inserts are an additional $99 for readers and $149 for a prescription.

Every Vision Pro purchase comes with adjustable bands, a light seal, two light seal cushions, a cover for the Vision Pro, a polishing cloth, a battery, and a USB-C charging cable and power adapter.

What to keep in mind before buying an Apple Vision Pro

Customers who book an appointment in an Apple Store will get a demo and a chance to personalize their fit before they buy the Vision Pro. Expect demo times to be limited — they’re on a first-come, first-served basis.

When you order the Vision Pro either in store or online, you’ll need to scan your face using an iPhone or iPad with Face ID. (The scan requires the Apple Store app, which is available from the App Store.) This, Apple says, helps determine the right-sized Vision Pro light seal and headbands.

You’ll also be asked a few questions to find out if you need optical inserts. If you do, you’ll have to upload a valid, unexpired prescription from a U.S. eye care professional after checkout.

If you can’t wait until Friday, watch a guided tour

Getting pumped about the Vision Pro? Good news — we’ll have a steady stream of coverage over the coming weeks and months. But also, Apple recently dropped a lengthy rundown showcasing features like EyeSight, Immersive Video, environments and more. Watch it here.

So clearly there’s a lot to learn about the Vision Pro — and lots of unknowns. Keep your eyes glued to TechCrunch, though, and we’ll do our best to answer as many questions as we can.