plain,

Working in the Mixed Reality group at Mozilla was the most fun I ever had in my 30 year career. Helping to usher a new technology into the world. One with such potential. I’d really love to go back there. Unfortunately it seems there isn’t really a there to go back to. Cue memory fade.

plain, WebXR

Mixed Reality (MR) refers to the spectrum of interfaces between and including Virtual Reality (VR, fully opaque) and Augmented Reality (AR: fully transparent). The Mixed Reality Group was a team at Mozilla researching MR, building web standards around it, and making a MR first browser for multiple MR devices. At one point we had tech running on the Oculus Go, Oculus Quest (1), MS’s HoloLens, and some Android-derived devices like the Pico (who I just learned was acquired by Bytedance). 

At the center of all of this was WebXR: a set of web standards (actual W3C standards) for building and securing mixed reality applications on the web. And I got to be a part of it. My team and I built developer tools, epic demos, wrote tutorials (since scrubbed from the web), and promoted WebXR content.  It was a thrilling time and a highlight of my career.

plain, The failure of WebXR

After the WebXR group was shut down at Mozilla a lot of other MR projects have been canceled, scaled back, or put on the back burner. In the following five years MR for the open web hasn’t happened and WebXR is largely DOA.

At Mozilla, after laying off most of the Emerging Technologies division in 2020, Hubs survived but mostly for the non-immersive cases, and even that was cut last summer. Everything else is gone. Our blogs are gone. The github repos have been archived. Almost wiped clean from the face of the web.

Apple supports the WebXR standard but only on their VR headset (more on that in a bit). We (Mozilla) had it AR based WebXR running smoothly on iPhones, but Apple never unlocked the API in Safari. In fact, Apple’s native ARKit efforts seem to have completely dropped off the face of the earth. They are still supporting it and occasionally add new features, but don’t highlight apps built with it or use it any way I can see. 

Microsoft was a WebXR contributor but they killed off their Windows MR initiative and HoloLens 2 is stuck on an OS that will soon exit support and they have shown no indication there will ever be version 3.

Google abandoned Daydream and has done nothing with Android’s AR APIs. They also shut down multiple web-based systems that supported AR (visual tours of maps, a 3d model store, and I think a few others). Chrome itself still supports WebXR, presumably on any PC based VR headsets, but they don’t seem to do much work on it anymore.

Meta still supports WebXR on the Quest, but I never hear about websites being built to use it anymore. They don’t advertise it or seem to encourage development. 

So MR on the web hasn’t happened. Or it happened and then it died. This makes me sad. Just writing this section of the article makes me sad. I really wanted to build the metaverse as an open network of websites that visitors could jump between. To make immersive a native part of the web. In the end.. maybe the world just didn’t want it

plain, Beyond the Web

The non-web MR platforms have slowly improved but not matching what we dreamed about. Meta shipped the Quest 3 (and the expensive Quest Pro, for some reason) and it has a few popular games and exercise apps, but it hasn’t grown into a thriving platform beyond games. They also tried to build a VR social network but if none of my VR using teenagers have even heard of it then clearly it flopped.

Sony has lost interest in the Playstation VR2, but that was always a closed platform anyway. Pico has continued to make headsets but I don’t think they are sold in the US. And of course Microsoft and Google shut down their own offerings.

Google recently announced Android XR, but I question if it will go anywhere. Google has a habit of starting big initiatives and then abandoning them when they get bored. At least that’s what it looks like from the outside. I imagine internal politics are different. I’d love more information here.

plain, Apple Vision Pro

And finally, a latecomer to the game (or maybe still too early) is the Apple Vision Pro. It’s amazing hardware but doesn’t do much and is very expensive. How is it that the best designers at the wealthiest company in the world can’t come up with better uses for AR than 3D photo galleries and VNC?  And if that’s the best they’ve got, then why did they release it at all?

Usually when Apple introduces a new technology they have some amazing showcase uses to drive adoption and excite developers. However, sometimes it’s simply impossible to nail the killer uses without a bunch of real people trying it out in the real world and seeing what works. The Apple Watch was like this. 

The first versions of the Apple Watch were slow, had terrible battery life, and couldn’t do very much. In most ways they were worse than everything else on the market. The one true bit of value they added was notifications from your phone on your watch, which ended up being easy for others to replicate anyway thanks to BLE. But Apple didn't give up.

Apple’s Watch OS team pivoted the UI and core uses several times before nailing the wellness functions around version 5. So why did they ship before then? Why did watches 1-4 exist outside the lab? Because without them they could never get to Watch 5. 

I think, or at least I hope, Apple Vision Pro is like this. There will have to be a few crappy versions until we get to the good one that has the right mix of weight, features, and battery life; and has a few (or even just one) truly killer use that justifies the product. 

I’ll say this about Apple. They don’t give up. They keep iterating until they nail it. Of course, once they find product market fit they sometimes let products atrophy, but that’s a story for another day.

plain, A different path forward

Maybe evolving VR to AR was the wrong approach. Maybe instead we should focus on regular glasses with a heads up display and actual useful functions before trying to nail the real world overlay and occlusion problems. Meta’s RayBan smart glasses are far more interesting to me than either of their separate VR or AI initiatives.

Maybe we need an AI agent looking over your shoulder analyzing what you are doing. Being the guy in the chair to our superhero selves. Secure AI agents could be huge here.