808 days in, designing for visionOS
What I learned, what was challenging, what I still don't know... A love letter to my favourite OS.
Day 1: July 2023
WWDC23, had to digest an entire new OS, design paradigm, spatial what?, what is AR nowadays, what is passthrough, why it's not VR but Spatial, what is transparency, scale, volume, floating font.
The most difficult part was not being able at first to rely on years of experience. I never really enjoyed the clunky XR, AR on iPhone was not something you could make a living from.
And today, back in July 23, the fruity company launched a high-res screen computer, we need to design for it.
August 23
Ok, ok, I think I can design for this thing. visionOS 1 is quite a wild beast, my dev comrade still swearing a lot of French words, I do too, because I have no clue how to make Figma look like SwiftUI views.
Even with Apple's Figma template, as always, you have no clue how it works. I had to slow down WWDC's videos, I tried to grasp anything, any ideas.
Gods were not sitting at the design table, we had no device to test our ideas, just an Xcode simulator and a lot of guesses. Painful!
Apple visionOS Labs
We arrived the day before in London. Excitement at its peak, the promise was to fianally touch it, giving us 5 hours to play. It's difficult to describe how I felt that day, because first, I was back at Apple. I met some old friends, discovered Battersea office. So grandiose.
After a short intro, we came to a meeting room, with 12 other devs. As we sat, there was this lovely wooden tray with our own AVP. The electricity was palpable in the room. The Vision Pro was waiting for me.
First Impressions
It's just a detail, a tiny little detail I started with: the computer made its infamous boing sound, then a white Apple logo appeared in front of my eyes in the meeting room. It was so crisp, so perfect I could touch it. Then the springboard, floating but perfectly designed. BTW, they won a Black Pencil for that design!
Designer's Frustrations
First question I asked: "Can we take screenshots?", guess the answer. So everything I was seeing, experiencing couldn't be recorded. There were so many details, the curves, glass effects, the colours and textures. Music app was so cool, Photos was also very very well designed. There was nothing I could copy/paste to Figma for later analysis.
Analog Tech
I knew it could happen. I had a backup plan, a notebook and few pencils. So I spent my first hour drawing, annotating, trying to explain to my future me back home, how it works, how these tiny interactions play out. It was so different from the simulator, it was live and absolutely gorgeous.
A Small Bunch of Crazy Ones
We spent the afternoon chatting with other devs. We shared ideas and early findings. We had to rush to test our apps, some ideas that were worse when tested on device. I met a lot of really great people there.
Even as a designer (and I think I was the only one that day) the dev community around this device was just the most excited I'd ever seen since iOS 1.
One Lab, Then 7
Yes, we were lucky. The way Florent and I pushed the visionOS boundaries made some impact. At one lab we had senior Apple VPs looking at one of our demos, asking how the heck we managed to do that kind of SharePlay experience. Very proud moment. We met our future clients there. We became friends with a lot of cool devs, hey
@jordibruin
Feb 24 Launch
In October 23 we met this team who had not one but two devices in their offices. Yes, very rare, and almost crazy to think they had two. So we built our first App Store app for them. Apple was very discreet about any launch date, but we felt it might be beginning of 24.
So we rushed the development cycle, spending hours in a locked room with a secured vault. One device each few hacks, haha! Can't really tell you, but the designer in me was happy.
We launched Swan Dive, first 360 arty videos. Florent had to invent an MV-HEVC encoder and a player, lol. Still used by tons of AVP people. Respect.
Fast Forward y 2024
We all learned a lot with this new OS. As a designer, I was not happy with Apple's Figma template, so I built my own. 150h of craft, had to hire a design system designer to help me clean it. I still use it today.
I also helped a lot of teams, designed some cool apps for the Vision Pro, and hit a massive wall.
Learning Curve
Once you understand how this OS behaves, it was time to push it. For example, all visionOS apps look the same. You have a great glassy effect, but no personalisation. Once your app is launched, there's a lack of branding.
Basically, after opening 6-7 apps around me, I couldn't tell who was who. So I had to hack it somehow. I found TabBar icons could be branded. You could add background blurs, like in Music, etc.
808 Days In
My little contribution to this ecosystem was to make a few cool apps, learned spatial computing design and most importantly, made some friends.
Do I regret it? It's not a strong NO.
I do like the OS, the potential, the free space as a canvas. The only thing that is difficult for most people I met, from indies, clients, agencies, filmmakers is the barriers to content creation. It's super costly, even with our new tools, like the URSA Blackmagic cam.
The AVP is a pixel-perfect beast, anything that is not high-res looks instantly wrong.
You want to make a 180 immersive video, that's not what you do with an iPhone.
You want some 3D artifacts, this is not a simple USDZ that you bought for $24.
That is my daily frustration.
Design, Code & Content
TBH, I haven't seen yet the quality on Vision Pro's apps the way we can experience some of them on iOS. There's a lack of interest from the design community for this spatial computer. I can understand, I feel the pain.
In fact, I think there's still a lot of opportunities for young talented designers to flex their muscles on spatial design. I can help, just message me.
Code is kind of easy, that's mainly SwiftUI. The logic from iOS applies well, with some tweaks around volumes, environments. But content is the true wall. You can fake it to demo things, but it will be THE problem to solve.
You're Still Here?
Well, it's just a love letter to the whole visionOS internal Apple team, really. To the dev community who pushed amazing hacks and experiences every day. To the 3D geniuses we work with on Neom (đ Gilles), to the video producers, sound designers, composers...
Last.
I won't do any more client apps. I'm done. We have better ideas, building the shovels for the next generation of spatial explorers.
Thanks for reading this piece.