2016 has turned immediately into the year of living virtually.
In 2015 there was the sense that it was definitely going to be that way, both for proper VR and its cousin 360° video, but it’s still surprising how fast it’s happened: headlong without the necessity for language or codes. Naturally it’s a time of experimentation.
Here are some reflections on the experience.
Describing the spectrum of experience:
VR – Virtual Experience – 360° video
Clear parlance and definitions are really important. A pure VR experience is a quite separate thing to a 360° video, although the former can incorporate the latter.
Witness ADR1FT for instance, which although it can be played in 2D, shines through an HMD (head mounted display) – a really seminal experience of VR’s new wave.
Then there is a form of media that intersects both, which here is referred to as a virtual experience and commonly comprises a dimensional space of simple geometry, one that shares polygons and video elements, and which The Guardian’s intense 6×9: Solitary Confinement is a great example (also featuring excellent binaural sound design).
Basically, the geometry defines a space around you that enhances the sense of perspective and therefore presence, but is in immersive terms a step below purer forms of VR – trading immediacy for qualities associated with higher production values.
Simple geometry is commonly used to enhance the space and therefore presence of a 360° experience. Car ads abound, and it’s an emerging solution for fast turnaround communication media, such as advertorial and real estate.
However, as below, it’s also an excellent opportunity for more sophisticated storytelling, and a brilliant approach to experimenting within the evolving language of the medium.
Finally, at the other end of this spectrum form pure VR is pure 360° video.
As an aside in this post, the output equipment across an array of cameras and lens types, is a wholly exciting subject in its own right (behold Lytro’s Immerge).
The distinctions along this spectrum of experience are fluid, but it’s really important, when describing the output or end product to distinguish between them: from pure VR to the more basic 360° video, and the intersecting media between. Otherwise the quality of presence, immersion and subtleties of these high fidelity formats become arbitrarily mixed up with the lo-fi.
And to show how tricky this can be, here’s my favourite virtual experience to date – much more than the 360° video it pegs itself as. Watching this experience on Oculus DK2 is completely different to watching on Google Cardboard.
Dead slow cross-dissolves work
Just as I write, the tools for editing and post production are beginning to emerge. When I was sitting down to work out logistically what could be achieved with the small time and smell-of-an-oily-rag-budget we had to produce the work (the enthusiasm and effort by great and talented people around you are as always the trick), a big consideration was flow.
It turns out that the humble cross-dissolve is ideal for establishing a transition in a 360° environment which put a general audience at ease. Dead slow pace also opens up interesting aesthetics in the transition moments – not only graphic forms in motion, but it affects the sense of travel, presence and temporality in about as positive and soothing a way as possible. Dissembling the world by cutting away parts of its structure to reveal the scene the audience would be seeing next – a sort of breaking down of the walls of one scene to reveal another – just felt artificial and flat, next to a dead slow dissolve, which retains the sense of volume between locations nicely.
If not binaural, pan instead
Although we recorded binaural sound across the scenes, we weren’t able to implement it in the final film. Andrew Stevenson, sound designer for the project remixed elements of the sound to emulate the experience of spatial sound instead. It may be a poorer cousin but it’s huge entirely worth it. Even for the below-conscious level acknowledgement that the drift of Nicole Car’s voice emerges from the location of the Joan Sutherland Theatre (to the user’s left), it’s a nice detail.
I also used sound as a pre-cursor and intermediary to visual transition, basically a thread to carry the user through the experience. Throughout the video, we either hear the next scene before seeing it or are led through it wit the existing soundscape.
Check your heights
Maintaining, or being very conscious of the reason you’re changing the consistency of the height of the camera (and therefore the end experience) is vital, and a basic checklist item when moving between setups. If you want to maintain consistency for the user across the experience, this is a crucial point and especially for fixed camera work.
It also gives you a powerful position to break from. I had initially wanted to come into the Joan Sutherland Theatre via the stage left well – coming up onto stage, with Nicole singing to the camera. When that was not possible, we inverted the camera rig, strapped it to a mid-stage fly line and sent it up over Nicole’s head instead. Especially if you’re viewing on a good headset, this experience of gentle lift works nicely as a disruptive thrill to the rest of the locked-off shots.
Spatial elements, grade
We paired right back on ambitions to define geometry and move the 360° video production more toward a virtual experience, but took the opportunities where they were available. I used perspective typography to contextualise the performances, and the grade study for the scenes gave them a slightly enhanced depth and saturation, to offset the lack of defined space – the ‘flatness’ of the final video surface. This works particularly well in the organ space of the Concert Hall, and also in the Utzon Room, with the intense golden light of early morning that pours into the room at that hour.
Title cards and credits are also a good place to break out into a simple sense of 3D. At the end of the film, the credits are set at the four cardinal points.
This was a production that was very much characterised by what we could do with the means to hand. Sure, one of the most incredible buildings in the world is a good start! and being surrounded by enthusiastic and talented individuals is the key. But what I mean is the intention of understatement and incidental observation really made the limitations strengths rather than the weaknesses they always promise to be.
It was a such a privilege to be given the opportunity to make what’s essentially a small arty film about Sydney Opera House. Huge thanks to crew, artists and friends who poured their efforts into the production. Please check out them out in the credits above.
Finally, here’s how I describe the finished video:
The Sydney Opera House 360° video intends to lead its viewer through the Sydney Opera House from daybreak to evening, immersing them in a handful of incidental experiences along the way. Deliberately low-key and vérité in its approach, the idea of observing the ‘nature’ of the house is explored – an experience that is not commonly available to visitors, but is the magic of the place that people who are around it all the time can easily recognise. Making this experience universally available is the core value proposition.
The performances arose in the vein of this approach: Nicole Car’s enthusiasm to ad lib in-between performances; Sydney Symphony Orchestra’s generous permission to let us record them in rehearsal; Benjamin Schwartz giving up his time to noodle in the Utzon Room early one morning. The spirit of generosity and creativity that is poured into the Sydney Opera House by all who work and perform there is what makes it a special place and infuses the visitor/audience experience. The 360° experience reflects on these qualities, the sense that behind the performance are the talents, enthusiasm and generosity of people who are inspired by the extraordinary place they work and perform.
Musically, the pieces all resonate with this intention. Rimsky-Korsakov’s orchestral work, Scheherazade, speaks to the grand romance of the building itself, a place and experience that is by its nature bigger than any of us, classical yet contemporary, an immersive feeling that sweeps us off our feet! Dvořák’s Song to the Moon is an operatic love letter to the incredible legacy of intimate opera in the Joan Sutherland Theatre. Benjamin Schwartz noodles around in the ‘key of Bach’, evoking one of Utzon’s inspirations for his mural, which Ben faces.