Blogs

Chris Leblanc, co-creator & video engineer, speaks of their project Un:Heard Resonance, the original tech created for the piece as well as the major themes for each movement and the underlying cautionary message overall. 

The art installation, Un:Heard Resonance, stirred up quite an interest among attendees at last month’s Northern Spark 2017. While among other exhibits surrounding the major theme of climate awareness, this interactive display granted attendees the ability to hear and visualize the “unheard” through subtle “resonances” from human interaction with plants, water, rocks and more. The resonant signals were then transferred to technological systems that allowed artists John Keston, Lucas Melchior, Mike Hodnick, and Chris Leblanc to manipulate and perform each interaction through audio and video mediums.

I had the opportunity to ask a few questions with lead video engineer and co-creator, Chris Leblanc. The following Q/A are the results of that interview.

Q: What is Un:Heard Resonance?

A: The piece was an interactive art installation that allowed attendees to perform on custom electronic instruments and feed the sounds into a larger system for processing. The sounds could then be altered, re-looped and sampled by our team of 3 sound artists before hitting the speakers, while each movement built up layers and layers of expanding soundscapes. The video portion had corresponding patches for each movement in the piece to follow the theme. It also reacted to the sounds that attendees were making. Our project focused on musical movements in the themes of Geology, Biology, and Technology, rotating every 15 minutes for 9 hours from sunset to sunrise Saturday night.

Q: Can you tell us how Un:Heard Resonance became an exhibit at Northern Spark?

A: John Keston has performed at several Northern Spark festivals now and our involvement began when the Weismann Art Gallery decided to have John’s piece outside at the entrance of their building during Spark. I’ve attended the festival for several years and think it’s one of the best showcases for the Twin Cities art and technological creativity. Plus, it has always been entirely free to attend so it’s a great community event.

Q: What inspired you to get involved with this project?

A: John Keston pulled me in early to his project when he decided that eye-grabbing video would add to the overall presentation and help pull people towards the piece. His main idea behind the project was to have three different movements of a musical piece that would rotate every 15 minutes for the nine hours that the festival lasted. The three movements were “Geology,” “Biology,” and “Technology.” The Northern Spark overall theme this year dealt with climate change, and our project reflected a few facets influencing it. An underlying theme of the project was that technology may not save our species and that it actually has great potential for total devastation of both geological and biological processes.

Q: What was the overall message you were trying to convey with this piece?

A: From the main designer John Keston:

“The project will bring awareness to sonic activity rarely experienced within the environments we live in. The combination of micro-sonics and accompaniment will non-verbally stress hidden geological processes, the fragility and jeopardy of the ecosystem as it faces climate change, and the rapid, global expansion of technology. It will also imply that technology may eventually replace the geological and biological states of the world. A precedent for this idea resides in the concept of “Computronium” theorized by Norman Margolus and Tommaso Toffoli at MIT, a hypothetical state of matter that would yield the most efficient and powerful atomic arrangement for computer processing. The Geology and Biology sonatas represent the first two sequential stages in the evolution of the planet, while Technology suggests the dystopian possibility of the world becoming a giant computer that no longer supports life as we know it.”

Q: How long did it take you and your team to develop this interactive display?

A: John had been developing the concept behind the project for about a year and the development of all of the instruments and construction took about a month. My video patch took about 3 weeks to plan.

Q: What experiences might someone have if they were to stop by your art installation? What kinds of interactions could you experience with the different pieces? 

A: The geology portion used contact microphones embedded in a sandbox, attached to rocks, and within a water system to make very percussive sounds. Attendees could hit the rocks together or slide them along one another for different sounds. There was a drummer that showed up and he was banging the rocks together like it was a drum kit while the sound artists in back mastered the different sounds and looped the beats into a frenzied rhythm.

The biology portion had three houseplants, each with a different take on a touch responsive synthesizer. You could touch the plants in different ways to make different sounds. One of the plants would sort of scream if you grabbed it, which was pretty funny. There was also a custom synthesizer made by Mike that used a wireless brainwave sensor headset to read 7 different types of brainwaves and shape the sound of the instrument depending on what you were thinking about. It was a really wild concept that worked beautifully in the project’s context. Mike is a total mad scientist.

The technology portion let people use a light sensor to play chasing LEDs as an instrument, and also picked up electromagnetic interference from people’s cell phones. It was an ominous and droning effect that played into the cautionary message.

Q: Is this all original programming? What was used?

A: All of the synths and instruments were custom pieces by the sound team. There were four arduinos powering most of the audio portion. My video setup was the Nintendo I modified, a box by Tachyons, and the heart was the LZX modular system. There were also 2 old video mixers in there too.

Q: How did you decide what sounds/video paired with each of the different types of interaction?

A: The geology section was extremely percussive because of the contact microphones on the rocks, and I used live video of rotating fossils and stone slices processed and colorized for texture.

The biology section was very lively and had a variety of types of synth sounds to represent the lush diversity of life on earth and its constant activity. For the video portion, I used a video microscope with various biological specimens for textures.

The technology section had sharper sounds and underlying ominous drone. The video blended live shots of performers being keyed through the Nintendo’s glitch textures.

Q: Can you give us a brief breakdown on what you had in your video rig?

A: My video system had a larger “master” element where the signals were all blended by older Roland VJ mixers and fed into the LZX modular system. The modular is the heart of my whole system, and it gets blended with feedback loops and a glitch effect box which introduces controlled chaos into video signals.

There were three sub-mixed systems, one for each part of the movement. For geology, I used a visual presenter lightbox with spinning agate slices and fossils mixed into video of people performing live. For the biology patch, I was using a cheap video microscope to feed magnified images of biological laboratory slides into the system as textures. For the technology movement, I blended textures generated by a 30-year old Nintendo that I circuit bent. It has a patch bay that connects different points on the graphics circuits that weren’t meant to be connected. It generates very glitchy sprites that dance around unpredictably. Anyone who approached my video system and asked me about it was handed a Nintendo joystick and told to help build some textures!

All my gear runs on ancient composite video standards, which is analog and lets you build video content immediately without any latency whatsoever. The analog signal path also means that the modular video system operates a lot like a traditional audio synthesizer, where shapes, waves, and animation are all constructed from voltages.

Many people in live video have obviously abandoned hardware entirely in favor of high-powered computer systems and software, but there is great artistic potential in a hardware approach. The glitch processors offer a method of what I would consider a “pure” form of video glitch art, where chaos enters the system as the system and circuits are not functioning properly. You are just guiding them into not working properly the way you want in a trial and error method.

Q: Tell us about how you performed the video footage?

A: My video portion was all hardware based, with analog modular synthesis and modified video equipment from the 90s for glitch effects. I blended a camera feed of people interacting with the machines with live images of fossils, agate slices, microscope slides of dog stomachs etc. I also had a glitched out Nintendo I built, which went over well in the technology section.

Q: Did you have a personal favorite moment that you experienced in your live performance?

A: Probably when the drummer was banging out beats using the rock microphones.

Q: Was this your first interactive exhibit?

A: I approach each time I run live videos for a musician as an interactive piece, since my gear reacts to the audio of a room and blends it with various camera feeds of the environment. If all of your content is being made live then it has to be more interactive by nature, since you’re making it all in response to whatever else is going on.

Q: How does it feel witnessing the public interact with your exhibit?

A: This project was a larger scale than what I usually work on for my art video so it was really overwhelming at times. It was extremely satisfying to have a constant stream of people showing up and finding new ways of playing these strange instruments. Northern Spark is great because it’s a night where the entire city seems to embrace the weirder end of art and really stay up all night and celebrate it.

Overall I think they liked it. I didn’t hear any complaining and no one smashed anything!

Q: What’s next? Where can we find more of your work?

A: I perform around town at various music venues when I’m not on work shows performing collage or liquid light show based video. I work with electronic musicians and psych rock bands mostly. Come say hi if you see a guy sweating over half of a 1980s broadcast studio in your favorite venue.