Blazing Trails in Augmented Reality’s Wild West

By Kati Krause

Illustrator Nadine Kolodziey is perhaps best known for her physical work in fused plastic foil. Now she's exploring completely new terrain: augmented reality (AR). On a recent cold morning, I joined Kolodziey in her Berlin apartment. We ate her mother’s Christmas cookies while we discussed AR; Kolodziey's Undrawn Drawing Tour; and how the relationship between illustrators and their work is like the one between humans and their dogs.

Create: You started working in Augmented Reality, or AR, in the summer of 2018. What drew you to it?

Kolodziey: I have always looked for new techniques to question the way we design or construct an image. But I felt very limited by the physical: I can experience it on my own but can’t pass it on. With AR, you can create an experience on the basis of an illustration and share it. That’s very difficult in another medium.

Create: I want to understand your approach to illustration before you started with AR. How did you develop your very specific style?

Kolodziey: I never set out to force a style. I simply designed and made things, and it has developed by itself. I think illustrators and their work are always alike.

Create: Like humans and their dogs?

Kolodziey: Exactly.

Create: In what way is your style like you?

Kolodziey: Bold, loud and unfiltered, like me. I’m not tactical or guarded. You don’t have to go through ten layers to know something about me. I’m pretty open about what I do and how I do things. And you see that in the illustrations. They have their ideas and little hints, but only at second glance.

Create: So you never set out to create this aesthetic?

Kolodziey: For me, it’s more about how you construct an image in your mind. When I work with fused plastic foil, I have a beginning and an end, a front and a back. Every layer is dynamic. When I draw, it looks completely different. Layers overlap completely, it’s hard to recognize individual aspects. That also influences the content.

Create: How?

Kolodziey: When I melt foil, images are more iconographic, they look more like logos. They communicate differently than a drawn image. We read and memorize them differently, simply because the layers are flat and easier to grasp.

Create: Do dimensions always play a role in your work?

Kolodziey: I think that’s the most interesting part. Mixing analog and digital, choosing a technique according to content… I’m pretty conceptual in my way of thinking.

Create: You became an Adobe Creative Resident in May 2018. Was that an incentive in applying for the Adobe Residency?

Kolodziey: Yes, to have the freedom to experiment. Because it’s difficult to communicate that you have a conceptual approach when you work in illustration or editorial. You have short deadlines, people buy a certain style. But what I’m really interested in is how we construct images, how we facilitate discovery and how we transmit experiences.

Create: When you applied for the Creative Residency, what project did you propose to do during the year?

Kolodziey: I call the project the Undrawn Drawing Tour. I want to research different types of design and illustration that explicitly combine the analog and the digital through interactive exhibitions that people can walk through.

The exhibitions take place in different cities and involve the user creating their own content. I want people to not just look at pictures, but to add something.

Create: So you didn’t reference AR in your residency application?

Kolodziey: Not explicitly. The first exhibition, in August 2018, took place without any AR. The room was a kind of cave, with sculptures of stalactites and stalagmites. You walked into another dimension. I’d made illustrations based on emojis people sent me and projected those illustrations onto the sculptures. People simply took photos or selfies of the projected illustrations. You don’t need AR to share an experience.

Then I did another exhibition in Los Angeles in October 2018. It was a booth at the Adobe MAX conference. There, the idea was to have a big display that involves even the floor so you’re swallowed up completely. I created an abstract landscape with references to nature. That’s where I first used an AR translation with Adobe's Project Aero, a new Adobe application for which I’m a beta user. Using tablets, people could take the exact shapes I’d used to build the park and place them in the room. The next visitor would then see what the previous visitor had built.

Click the image above for a virtual walkthrough of Nadine Kolodziey's exhibit at the Adobe MAX conference.


Create: What types of AR are you currently working with?

Kolodziey: With Project Aero, I can place shapes freely in any space and add an animation, too. I’d like to try to use a Photoshop file—for example, a gate with 40 layers—upload it to Project Aero and split it up, which would allow me to walk through it. It would be like a pop-up book.

For another AR project, I'm partnering with a friend in Japan who has a young programming business called Sogendo, and we support each other: I have beautiful graphics he can use to test things, while for me it’s great because I can work with programmers much more closely to execute customized ideas.

Create: As a creator, what are the possibilities of AR that most excite you?

Kolodziey: Everything in that field excites me! I’d love more stuff in the direction of what I’m creating with Sogendo: visual triggers for added information. Figures that come to life. It would be great to add a second, third, or fourth layer to any image I create, one that only exists digitally; the continuation of a story, for example. Or the interior lives of characters. Or the same story in another dimension. It would allow for a lot of education. You could say, for example, this story is very gendered—what would it look like if it were neutral?

Create: How did you use this technology in your exhibition in Japan?

Kolodziey: An association wanted to create a mural in a pedestrian tunnel the members walk through every day in Matsudo, a suburb of Tokyo. The tunnel connects two very important places: a temple and a historic house that was once the home of the last shogun’s brother. For the association, it was important that the mural had something to do with the city. So I did some research and found several stories, which I included in the mural. Some of the illustrations come to life in AR when you look at them with an app we built. It works really well to capture people’s attention. It was important for me to make this part of the Undrawn Drawing Tour.

Create: Will your next exhibitions also be part of this tour?

Kolodziey: Yes, I have three coming up in January 2019. At the In/Visible Talks in San Francisco, the idea is to build a proper garden—but it will exist only in AR, and the garden will grow over the course of the conference. People can take pictures and videos and share them. I’ll make it with Project Aero.

Then an exhibition in Hollywood, at a gallery called 1520 that belongs to Urban Outfitters. There’ll be Project Aero shapes that people can add, so it’ll be interactive. Afterwards, I’ll do another installation in San Francisco called Hooper Project that will also have an AR aspect.

The Hollywood exhibition and the Hooper Project will have some similarities to the booth I made for Adobe MAX. But the Invisible Conference is completely new terrain for me, so I’m a little nervous. I think they are, too.

Create: Will the In/Visible Talks installation include VR?

Kolodziey: No. I mean, you could film it and turn it into something you can experience after the fact. But that’s not the idea, to have a VR experience. People should experience it with devices that are more personal.

Create: Why do you prefer AR to VR?

Kolodziey: I think that in VR, you disappear into a dimension. You wear those goggles and aren’t in your own reality anymore. You look at a memory. What I like about AR is that it takes our present and adds a digital component. I find that more interesting because it’s closer to the experience. With VR, I move in a recording, in another place or another time. AR uses the here and now. I can explore it myself. Also, AR is something I can experience with friends. It’s more about others.

Create: You’ve compared AR to the Wild West. Are there any people you look to as pioneers in the area?

Kolodziey: Not really. I think that especially when you’re discovering something new, it’s difficult to look too much left and right at what others are doing. That influences you and you limit yourself, especially when you first work with a technique—it’s restricting.

Create: What types of content do you find most interesting to create with AR?

Kolodziey: Allowing people to move freely inside an artwork and make it part of their physical experience. But I also find adding something graphic to our world super interesting. I’ve done that with my project Lost Shapes: I can look from a skyscraper down onto a square and project a huge sculpture onto it. Usually, that takes years of conversation with the city authorities, permits, material… There are far too many hurdles for an artist to create this kind of artwork. But with an app I can simulate what it would look like if I placed a huge yellow stroke in the middle of Potsdamer Platz.

Create: Do you see the value in the image itself, or as a model that leads to a real-world sculpture?

Kolodziey: I find it difficult to measure the two against each other and decide which one is more real. My credo is, the analog and the digital in combination are the realest. That’s the next frontier of experiencing reality. That’s why a digital sculpture isn’t worth less than an analog one—it simply allows for more experimentation.

December 12, 2018