“Let our rigorous testing and reviews be your guidelines to A/V equipment – not marketing slogans”

Apple Vision Pro: The Good, the Bad and the Creepy

by Wayde Robson — July 10, 2023

Apple Introduces Vision Pro at WWDC23

By now you’ve heard that Apple has finally announced and demonstrated the rumored mixed-reality (MR) headset it had been working on for over a decade. Apple has created the audio-visual device it believes not only changes how we use computers, but how we interact with the world around us. From the Audioholics perspective, VisionPro potentially marks a disruptive, watershed moment in home theater and immersive audio just as the iPod did to the music industry. Think of a single device where large-scale TVs, projectors, and immersive audio setups are consolidated into a single, portable device. And in Vision Pro (and its new operating system VisionOS), we could also be witnessing the start of a change in Human-Computer Interaction almost as dramatic as the launch of the original iPhone. The naysayer might remark that might this is an expensive misstep Vision Pro someday to remembered alongside Apple Newton. It will certainly be interesting to look back at this article in 5 years and see what evolution has transpired. Nevertheless, whatever the future holds, nearly everyone who has tried Vision Pro seems to agree it's a breakthrough MR headset technology. And Vision Pro is only the first iteration of a device running VisionOS.

In usual Apple form, Vision Pro is not completely new. It's an evolution and refinement of technologies to create an entirely new experience. Apple CEO Tim Cook calls that new experience “spatial computing”—the ultimate progression of its Spatial Audio format and a fair description of the device’s new VisionOS. Since Apple considers the Vision Pro a new category, at its Worldwide Developer Conference (WWDC23) presentation, Apple avoided using broader terminology associated with Vision Pro's underlying technology. It's not just an augmented reality (AR) device, nor is it virtual reality (VR) headset. Most tech-press are calling it a mixed reality (MR) device. I'll go with the wisdom of the online crowd here.

Make no mistake: Vision Pro is not an accessory. Instead, it’s a fully functional, wearable computing platform (computer, monitor, keyboard, audio device) that sits on your face. That last sentence conveys both Vision Pro’s greatest value proposition, and its biggest hurdle to broad adoption.

The Good! A seamlessly conceived device that promises to disrupt Home Theater as we know it today

Vision Pro wears like a typical VR headset. Your eyes are sealed in an isolated environment in the visor. The brilliant interface breakthrough in Vision Pro is its high-performance rendition of your surroundings. This is only possible because Vision Pro has enough outward-facing sensors to rival a Galaxy Class Starship (a nod to our Star Trek fans). Sensors include an array of cameras; IR illuminators for low-lighting; TrueDepth cameras and LiDAR scanning for real-time 3D mapping; and microphones all working together to generate a 3D audio/visual map of your surroundings. Apple designed the device to let you see the finest details.

Vision Pro's experience is said to be stellar, in typical Apple fashion. Those who have tried it say it provides smooth motion at just better-than 4K resolution of the world outside. The headset sports a 90-Hz refresh rate with wide-color gamut and high dynamic range. You’ll gaze into an unprecedented 23-million pixels across two separate screens, one for each eye. Vision Pro boasts a sensor-to-display latency at only 12-milliseconds. That's effectively instantaneous—at about 8x the speed of a blink of an eye. It's the sensors and VisionOS overlay you're seeing at 12-ms, Apple did not demonstrate that speed using fully rendered 3D virtual environments at WWDC23.

VisionOS

VisionOS, overlays itself in the foreground of your environment, enveloping your field of view while you control the Vision Pro experience in the most organic ways possible: You use your hands and gestures for control. Cameras around the outside of the goggles track your hands and identify simple control gestures. For example, you can sit in a room lifting a hand into the sensors while pinching thumbs and forefingers together like a zen-meditating orchestra conductor. Meanwhile, on the inside, IR cameras and a ring of IR illuminators track each eye with such precision that tech YouTuber Marques Brownlee simply called it "magic". The personal accounts from those who tested it agree that VisionOS' graphics, controls and environmental display are smooth and responsive. Some are calling it the first headset to bring a true high-performance rendition of AR as VR into one device.

VisionOS Spatial Computing

Organic VisionOS Experience

I’ll admit, I've oft' been a critic of Apple’s track record of forcing its faithful inside a tightly controlled ecosystem and paying inflated costs—like its $1000 Apple Pro Stand. But I’ve never doubted the company’s ability to bring new directions to familiar technology—or blaze new ones. Today, I have to admit - Vision Pro is a tremendous feat of equal parts design, engineering, user-experience, and a bold step into MR—and this is only the first version of VisionOS' future.

Apple has a consistent design style when it develops new products. It elevates existing technology by introducing the organic Human-Computer Interaction technology always needs. Apple did it for the computer by introducing the mouse to Lisa back in ‘83. Then the mouse became a standard computer accessory. It did it again to the mini-keyboards on mobile devices when it introduced the iPhone touch screen. Touch-screens are now practically mandatory on smartphones. None of these were Apple inventions of course, Apple just made them essential. AR, VR and MR headsets have been around awhile, but again Apple has made them more intuitive by introducing one that works well without handheld controllers. Instead, Apple used our built-in human communication systems—eyes, hands and voice—to control the device. When using VisionOS you just look in the direction you want to navigate and eye-tracking highlights the object, then just pinch to select or pinch and flick to scroll. These simple controls let you master virtual environments of content in three-dimensions. You can even shortcut to what you want by using voice control.

Vision Pro Audio

There’s little to be learned about Vision Pro’s sound quality from Apple's presentation, but certainly a lot to speculate on. In images we can see the device's audio system consisting of what Apple calls dual-driver audio pods. We know they’re capable of Apple’s spatial audio rendition of 3D sound and head tracking capabilities. It’s a safe bet that Vision Pro’s sound will be a worthy acoustic successor to Apple’s AirPods Pro series of earbuds. But that probably won’t get most audiophiles excited.

We’re able to see in the video that Vision Pro’s audio pods don't cover the ears and an animation shows the pods sending sound downward toward the user’s ears. Perhaps this brings benefits in wider dispersion of the spatial audio effect. But in my humble opinion, this can’t be ideal for sound quality, unless Apple has developed its own Focusonics-like sound-beam technology. They’re not likely to provide acoustic isolation or Active Noise Cancelling and will almost certainly bleed some sound into your environment. Perhaps that's by design.

Bluetooth Earbuds on a Plane!

Designing Vision Pro so as not to invade outer-ear's airspace may have been a great choice, provided we can use our own Bluetooth earbuds or headphones with Vision Pro. It would be ideal if we can achieve a good fit with our own headphones over top of the headset. The presentation’s shot of the woman on a plane supports Bluetooth audio capability as she’s clearly using a pair of AirPods Pro. That's likely for noise cancelling and to prevent audio bleed into the cabin. If it can pair with non-Apple Bluetooth devices, we should be able to use a Bluetooth A/V receiver or DAC to watch a movie as an audio alternative to Vision Pro's audio pods. But whether through audio pods or anything over Bluetooth, the home theater audio experience just isn't the same as a real-world competent home theater system with subwoofers that can shake your insides.

Audio Ray Tracing

With the promise of immersive spatial audio, Vision Pro doesn’t neglect environmental ambient sound. The company created a new technology it calls Audio Ray Tracing that uses the device’s comprehensive room-scan data to reproduce ambient acoustic properties. The name Audio Ray Tracing, appears to be borrowed from a graphics rendering algorithm used to model light within a given 3D environment. Consumer-grade Ray Tracing was brought to gaming by Nvidia through its RTX-series of GPUs. I also wonder if Apple’s Audio Ray Tracing research has any application in home audio room correction. Perhaps Apple is already working on significant upgrades to its HomePod lineup.

3D Room-Data Mapping For Sound

Solo Entertainment

Disrupting personal entertainment is perhaps one of the most intriguing aspects of Vision Pro. One need not be a futurist to see that VisionPro is positioned as a turnkey immersive audio, home theater alternative. Through VisionPro, you can experience entertainment from a nearly infinitely large screen; experience entertainment in 360-degrees and in three dimensions in a simulated immersive audio environment. Put another way, what may have previously cost you five or six figures between display, projection, processors, amplification, speakers, room size, acoustical treatment, and lighting control you now have in a single headset for $3,500. And let's not forget the WAF factor too! (Though the Vision Pro certainly introduces new WAF factors of its own). Moreover, if you're in a small NYC apartment, imagine the potential something like the Vision Pro represents. Through nothing more than a headset, you'll overcome any and all physical confines of your space—and there's no more confined space than a NYC apartment or an airplane seat! This is a game-changing concept when one truly thinks through the typical limitations most enthusiasts grapple with.

A full entertainment experience regardless of space may be Vision Pro's unheralded strength. I can’t wait to try it as a home theater alternative.

Apple's doing everything right to advance this concept. For example, Disney has agreed to build a native VisionOS Disney+ app, loaded with unspecific interactive experiences and 3D-effects that will surely captivate users for minutes-on-end. (Your mileage may vary, but it's not enough to get me to subscribe to Disney+). Reports from Vision Pro testers indicate that Apple has an amazing live sports experience in store too. Users saw a demos including an "at bat" perspective of a baseball game and a court-side view of an NBA game both of which are designed to immerse you into the game as if you were there. When the NHL finally trickles down to VisionOS this could easily become my preferred way to watch a hockey game. While these are unique experiences, they may end up being largely impractical or unused features like the multi-angle in Blu-rays.

Unique Disney+ Experience on Apple Vision Pro

While the ability to create custom-sized virtual screens before your eyes seems amazing, most movies will still playback on a flat 2D screen. Will the Vision Pro be a catalyst to bring 3D movies back to consumer consciousness? Vision Pro could be the best 3D-movie platform ever as it overcomes the fundamental obstacles that made 3D problematic for so many people. Forget 3D-TV without the glasses, Vision Pro will give you 3D-TV without the physical TV! Long after the 3D-movie novelty has worn off, Vision Pro’s inherent control over ambient lighting, brightness and contrast is sure to create a visual feast on a flat 2D virtual screen. I know 3D movies has its supporters, and more power to 'em! But I’ve always avoided theatrical 3D showings if there’s a 2D option available. In fact, whenever I hear 3D movies are making another comeback, I think of Dr. Tongue’s 3D House of Pancakes.

3D making a comeback? See those pancakes coming right out at you!

The Bad

No Home Theater Substitute. At least for now

There are going to be steep limitations to watching movies and shows on Vision Pro. Foremost is the 2-hour battery life. But since you can plug Vision Pro into a constant power source and let it run indefinitely, the stationary movie experience is solved with a tether. The biggest problem however is that it will only be a solitary experience. I often watch movies alone when the rest of the family is occupied and I can get a precious hour or two's escape into the man cave, and I'm sure it's a unique, incredible experience. The question yet to be answered is "What will Vision Pro's impact be on communal entertainment experiences?" However, for all of Apple’s effort to promote interacting with other terrestrial beings entering our augmented space, it’s still a solo experience. I do not see it replacing the communal experience of a real home theater system. This is why I can only give home theater capability a half thumbs up.

Office Productivity?

I work at a computer almost all day long doing standard document editing tasks. The ability to edit docs, spreadsheets and presentations across multiple virtual screens will certainly be an interesting way to get work done. Maybe it's only because I've never experienced it myself, but I just don’t see spatial computing making my own office productivity use-case any more productive than a laptop. For me at least, it's not solving any specific problem nor lending any efficiencies. The virtual keyboard included in VisionOS that you can only see and not feel might inevitably make for a clumsy typing experience. I tend to type through sense of touch and almost never look down at my fingers. Fortunately, you can use a regular keyboard and mouse sitting at a desk with Vision Pro. But would we really want to sit at a desk working with keyboard-and-mouse while wearing a face computer if you didn't have to? If heat and discomfort doesn’t get to you after a couple of hours, Vision Pro’s 2-hour battery life will. But hey, this is only version 1.0 of the spatial computing experience, so there's always hope for future iterations. After all, unlike watching movies and sports which I prefer with family and friends, I work like George Thorogood drinks, alone. Tangible improvements to my solo work experience would be welcomed.

VisionOS Virtual Keyboard

On one hand, I never expect Vision Pro to excel at spreadsheets, but a high-performance AR headset like Vision Pro will almost certainly open new possibilities to 3D CAD engineering and design. There are plenty of workplace settings that already use or could benefit from AR in many industries. With the right apps for VisionOS, Vision Pro might bring some stiff competition to devices like Microsoft’s Hololens. Maybe that’s just enough to get Vision Pro’s foot into the doors of some workplaces.

Apple did gloss over the issue of video conferencing with the Vision Pro. I suspect this feature, which displays an avatar of you to participants, is still a work in progress.

Vision Pro is not for Real Gaming Just Yet

Unfortunately, at this point in the product's evolution Vision Pro is not the gaming platform or VR gamers are looking for. The Oculus seems to maintain an edge here. The Vision Pro demo included a shot of someone playing the mobile version of NBA 2K ‘23 on a flat, 2D virtual screen with what looks like a PS5 controller as the voice-over says:

“Vision Pro is a wonderful way to play your favorite games, just connect your controller and play on a massive screen with incredible audio.”

That line was aimed squarely at casuals, not hard core gamers. For subscribers, Apple Arcade promises over 100 of its games to be Vision Pro-ready by launch. But it’s unlikely serious gamers will find their favorite games among them. Don't expect to see games like Elden Ring or Hogwarts Legacy to be playable on a Vision Pro virtual screen, nor any fully rendered 3D VR games. Some Apple Arcade games may include some smaller virtual 3D elements but the majority will likely be the kinds of games found on Apple Arcade including a lot of ported mobile games played on a 2D virtual screens. While panoramic 3D images and video clips that envelop users into virtual environments are sure to be stunning with Vision Pro, these aren't the same as today’s 3D VR gaming experiences with fully rendered objects. The closest we're likely to 3D VR gaming will be games like Rec Room which is coming to Vision Pro. Rec Room is a lightweight VR game with graphics that look as if they were stolen from Roblox. Perhaps Vision Pro's M2 CPU/GPU and cooling system isn't designed to handle the 3D VR assets like a dedicated VR headset.

Vision Pro, the ultimate mobile gaming platform!

Apple computers have historically never really targeted the gaming market and the Vision Pro demo nor did Apple even use the term VR at all. Serious gaming was just not the direction Apple took with Vision Pro. But Apple might be changing to become more hardcore gamer-friendly in the near future.

Another presentation at WWDC23 announced new gaming initiatives for Macs running its new breed of Apple Silicon chips, and Vision Pro uses an Apple Silicon M2 chip found in last year's Macbooks. The new Game Porting Toolkit will allow developers to port Windows DirectX games to Mac and the new MacOS Sonoma includes a Game Mode function to optimize resources that let you play. It's easy to believe that future VisionOS devices or perhaps updates for Vision Pro itself could bring some DirectX games to the platform that might be playable on a virtual 2D screen. But for now, no such announcements have been made.

The Creepy

How are the people of 2024 going to react the first time someone is seen out in the wild wearing their Vision Pro? What will we think seeing that person, uncanny-eyeballs forward, knowing they're probably making a video? The public backlash against Google Glass was the stuff of urban legend. Academic institutions like MIT have grappled with it. It could be even worse for Vision Pro wearers next year. In the Google Glass-era, we didn't have the term, "Tik-Tok prankster" yet. Now we're watching out for them. To Apple’s credit however, public wearing of Vision Pro was not among the use-cases promoted in its demonstration—even if the shot of the interaction between a presumed father and his two kids during the Vision Pro presentation was rightly noted to be a "little weird".

Home video goggles

Tinfoil Headset

Apple’s Vision Pro research involves an estimated 5000 new patents. Yet for a product with so much high tech potential, WWDC23 may have been the first new Apple product announcement where the presenting CEO wasn’t shown actually using the product. During all the subsequent media appearances I found with Tim Cook, he always avoids putting the headset on himself. Why is Tim Cook avoiding putting on the device? We already know Apple’s eye-tracking is so groundbreaking that I expect an upcoming optometrist checkup app. Unless the American Optometric Association bans any such apps. I’m sure Vision Pro could spot a torn retina or astigmatism in seconds. But they say the eyes are the windows to the soul, and maybe your emotional state and intentions. And this is where unbridled technology advances can get downright creepy.

Former Apple software developer Stirling Crispin says he performed much of Vision Pro’s foundational work as a Neurotechnology Prototyping Reseacher. According to Crispin, while he was working on the project, Apple had been exploring “mind reading."

Vision Pro is looking at more than just your eyes with those inward facing sensors. Crispin says they were working on a litany of other measurements including: Electrical brain activity, heart rate and rhythms, muscle activity and blood density in the brain, blood pressure and skin conductance. These measurements are taken so Apple could read your emotions in real-time and make predictions about what you would do next with the device. Their research paid off in predictive lessons, such as how pupils react before you even click. The feedback Apple gets from these mind-probes allows the device to adjust to your cognitive state in real time. According to Cripin they even explored:

“Other tricks to infer cognitive state [that] involved quickly flashing visuals or sounds to a user in ways they may not perceive, and then measuring their reactions to it.”

Former Apple Developer on Twitter

There's quite a bit to unpack there. And this is only the research Crispin could talk about. While Apple may or may not accumulate such data eventually, we know all-too-well, that the widespread, unchecked availability of such technology could have unintended consequences. Too many companies (Ahem, Google and Meta) make their whole existence centered around accumulating and selling data about people. Whether it's in the form of corporate abuse or a hack that exfiltrates that data, we should all be wary and vocal in our opposition to such unprecedented data gathering.

Conclusion

Vision Pro is truly a breakthrough technology. It's the best performing MR headset of any kind. Apple is clearly years ahead of the competition. The interface is brilliant and refined as are the fundamental gestures to interact with Vision Pro. That natural, intuitive interface has the potential to perform nearly unlimited practical computing functions. Its $3,500 price will keep it aspirational for most of us. However, taken in context from a home theater perspective, that $3,500 cost to substitute for a 120-inch display, AVR, and 7.1.6 speaker array suddenly becomes immensely affordable. And, as we saw with the original AppleTV, iPhone, iPad, and Apple Watch, prices will drop as the technology improves and imitators (hi there Google) step into the market. Rumor has it that Apple is already at work developing a cheaper version of Vision Pro, without the Pro moniker.

If there was ever a serious attempt to bring a face-computing device to the normies, the kinds of people who don't have patience for immature technologies, this is it! But there’s one problem not even Apple can solve in trying to bring a computer capable of placing virtual & augmented worlds before your eyes. The problem of wearable technology for your face. Despite high-profile efforts, no tech company has been able to crack the “face computer” acceptance code. They’ve remained niche tech enthusiast devices.

Wearable Acceptance by 2024?

Wearable technology has limits, we’ll gladly wear a watch, we’ll even put on headphones. Not long ago earbuds were a step too far, but Apple has since found a broad market for its AirPods despite being a literally invasive wearable technology we stick inside our ear-holes. But will the average person ever go so far as wear technology on their face? The answer is certainly a "no" until battery life, physical size, and wearable fashion improve. And there's the unanswered social reaction to wearing goggles in communal settings.

I give it an easy thumbs up, I'd use it personally. Even if it is reading my mind for nefarious purposes, I'd be happy if they're finding anything useful in there. But that’s my own bias as a techie nerd, not a prediction of market acceptance. For that, I’ll defer to someone wiser than myself.

One of the best Marketing teachers is also a culture & tech critic, NYU Prof. Scott Galloway, pictured right with his own rendition of Vision Pro. Galloway just doesn’t see a mass audience coming for Apple's device. He breaks it down to our evolutionary biology when he claims that one important social function of personal technology is that it confers status to potential mates. But in his opinion, wearing a pair of Apple goggles on your face sends the opposite signal, it says-don’t mate with me, there might be something wrong with my genes. In a Medium article Prof. Scott Galloway says:

“We’re highly discerning about what we put on our face, as it must enhance, not impair our ability to assert dominance, attract mates, and make connections.” - Prof. Scott Galloway

Is he right? Will Apple Vision Pro or a follow-up VisionOS device disrupt and change or home theater and immersive audio hobby? Will a product like the Vision Pro ever achieve mass appeal if Apple can overcome production issues recently reported? Tell us what you think in the comments.

Discuss This Article