AR-VR • Inspiration Are We There Yet? And Does It Matter? Sundance New Frontiers 2020

At the Sundance Film Festival, the transition from the busy and snow-bright Park City streets into the dark halls of the New Frontiers program is dramatic. The transition into the head-setted virtual spaces inside is otherworldly.


Sundance New Frontiers is the cinema-adjacent sideshow to the Sundance Film Festival, featuring media projects and artists that push the edges of technology in the service of storytelling. Since 2007, New Frontiers has been showcasing new forms of artistic media expression, and along the way rapidly advancing technology has fueled a similarly rapid creative evolution. Long a playground for VR, AR and 360-video, this year New Frontiers projects also took advantage of artificial intelligence, smartphone technology, big data, biofeedback, and facial recognition, among other things. It even spilled out of the dark rooms and onto the Park City streets—and in one case, into the hotel swimming pool.

Sundance New Frontiers 2020 showcased new forms of artistic media expression, including VR, AR, 360 video, smartphone tech, big data, biofeedback, and facial recognition. Click to watch a brief video shot at this year's event.

The artists working in this space come from an equally broad spectrum, such as the film world, the arts community, the gaming industry, and elsewhere. VR, AR, and all their associated flavors are often criticized for being alienating or gimmicky, or always but not quite on the edge of reaching critical mass. Shari Frilot, the New Frontiers Chief Curator, doesn’t see it that way, and neither do the artists who create using these tools. “It's emerging and evolving and iterating, and that's where my interest lies: to find out how art, film, and technology converge to manifest something that's larger than the sum of its parts,” she says. “And sometimes that manifests in different ways of storytelling. And sometimes those different ways of storytelling come to create new technologies.”

New Frontiers 2020 was popular with festival goers; some works and installations were almost impossible to get in to. Scarecrow was a continuously booked crowd favorite, and it brought a strong emotional component to its story of a lonely straw man. Scarecrow creator Sngmoo Lee is also a filmmaker, and he wanted to bring narrative and humanity into a medium that many criticize as cold, even using human physical intervention at some key points when he reached the limits of digital and haptic technology.

An image from Breathe, a popular piece at Sundance New Frontiers 2020. 

Breathe, another popular piece from this year’s program, used an interactive, multi-user model toshow the connections we all share and the impact we have on one another. Mixing traditional storytelling with experiential data visualization, Breathe’s mult-user and interactive environment breaks through some of the traditional barriers to adoption. All Kinds of Limbo, another multi-user VR piece, allowed for 20 users at a time. “You're starting to see [more multi-user experiences], and I selected those from a whole body of works out there that were designed for [multi-users]. There are way more than last year.”

Click to watch a brief video showing elements of BLKNWS.

One high-profile project was Kahlil Joseph’s BLKNWS. Joseph is the artist/director behind some beautifully original music videos like Beyonce’s Lemonade and the amazing Until the Quiet Comes for Flying Lotus, as well as distinctive short films and video installations. With BLKNWS Joseph and his team have created a piece that combines appropriated footage with originally produced segments to comment on the “news-industrial complex” and to provide a different perspective.

Both the aforementioned All Kinds of Limbo as well as Persuasion Machines used leading-edge VR to create rich experiences. All Kinds of Limbo, from the National Theatre of Great Britain, took you from a small black physical room into a huge virtual hall, placing you in the middle of a series of musical performances tracing the influence of the West Indies in the UK music scene. Persuasion Machines added some personal privacy-related tricks to make its point.

Click to watch a brief video showing elements of Spaced Out.

If stepping out of the snow and into VR space wasn't disorienting enough, try taking off your clothes and jumping into a swimming pool, as Spaced Out requires of its audience. Combining a new underwater VR headset from Ballast with abstract George Melies-influenced art and audio from the Apollo 11 mission, Spaced Out wants to be a sensory deprivation tank-like happening. And maybe it would have been if the water hadn’t been so cold and didn’t get in your mask. There is potential here, but there are a lot of variables to get under control for this to feel like a real weightless out-of-body experience. It probably would have been amazing in a heated pool, but as it was, the viewer achieved only a semi-altered and mostly uncomfortable state.

Another favorite was The Book of Distance, an intimate, graceful piece on the lesser known story of Canadian WWII Japanese internment camps. The Book of Distance turned some of the traditional limitations of VR—such as the single user paradigm—into a strength. It also used audio and subtle interaction to create the most intimate and moving piece I’ve seen in a while, in this medium or any other. It also succeeded in terms of narrative storytelling, and explored themes like xenophobia and racism that are as relevant today as they were in the 1940s. It’s an empathy-building machine for a world that needs it as much as ever.

VR/AR has long been a space where critics wring their hands and big media companies demand scalability and mass adoption. Those criticisms are being addressed, even while they miss the point. “One big thing we talk about in the world of AR/VR is the access, and the ability to allow anyone to experience this type of content much more easily.” says Elizabeth Barelli, Adobe’s principal product marketing manager for AR and VR products. “I think we are at a turning point, where not only can you experience AR on most mobile phones, but we're also seeing a lot of new headsets that are becoming more accessible. This technology will continue to develop in innovative ways in the next few years. ”

A scene from The Book of Distance.

Barelli also thinks that adoption is inevitable and that digital overlays in the physical world are an expected next step in the evolution of a screen. “I think there's going to be a time in history where we'll look back at today and think, ‘Oh, do you remember when we were experiencing all these stories on a small, flat screen and how limited that was?’ The same way we look back now at TVs that were just stuck in your living room. Humans crave to sense things and interact in a 3D, physical environment. And I think that the fantasy of AR, holograms, and seeing things in 3D isn't actually totally new. It's been something that we dreamed about in the last century; a lot of science fiction literature and movies can attest to tht. The real question is how we will be using that technology, and how it will add value to our daily lives.”

In any event, the artists and makers don’t really care much whether these storytelling tools are widely adoped yet. It’s either the right medium or it isn’t, and the intimacy, control, and ability to tell certain kinds of stories, or to put the viewer in unique circumstances, is unequaled. As Shari Frilot says, “The artists are absolutely convinced by this. You can't keep them away.”

You may also like