The brain is a fascinating mass of constant connections, or synapses, where neurons are constantly forming constellations to help us make sense of our world. We were intrigued at the thought of helping visualize what the brain’s activities look like in a live performance when approached by founder and choreographer Jodi Lomask of Capacitor, a performance art company that explores non-traditional combinations of arts and sciences through movement. Their latest project Synaptic Motion debuted at the Yerba Buena Center in San Francisco this September.

Synaptic Motion intertwined music, visuals, and dance to show the brain’s many complex processes.

The performance comprised of custom composed audio, a live MC, an array of costumes, dance, large-scale movement sculptures, and floor and wall projections.

Jodi had EEG data recorded from her brain at the UCSF Neuroscape Lab in order to see what her thought processes looked like and how the outputs changed as she moved from her most active states to sitting completely still. We analyzed the scans and began to ideate graphical metaphors representing the transitions of the brain and the different phases of thought. With EEG data in hand and the vision in mind, the new challenge was synchronizing the brain and sound wave data with movement.

Sound artist, Oni Martin Dobrzanski, composed a soundtrack with specific pieces for each scene of the performance that audibly represented how the brain responds to hopelessness, a caffeine rush, a seizure, or an idea. Each song had so many granular layers that we felt our visualizations would be most complementary if they were layers of simple shapes that formed grids and dynamic compositions.

Using Processing, we loaded Dobrzanki’s audio and synced our sketches to the frequencies. But, being that we are a data visualization studio and new to the dance production process, we felt we needed to better understand what it means to be in performance arts. So, our team dropped by their rehearsals to speak with the dancers, scientists, and media artists. We demoed initial sketches at a few of them and felt good about the direction we took and so we continued to iterate.

For one of those iterations, we developed a ‘constellation’ algorithm that generated lines moving through a grid to mimic connections made between neurons. Whenever the lines passed through the composition’s threshold a new constellation is generated. This visualization ended up being one of the favorites despite its simplicity. To make it more dynamic and even more awesome we made a 3D version.

We encoded, packaged, and sent our favorite animations to Mary Franck, the projection artist of Synaptic Motion and the mastermind of the show’s visuals. Mary pieced together our work, weaving her graphics and ours into the storyline and overall concept of the shifting states of the mind. It was exciting to build visualizations for a piece where we had no idea what the final output would look like. Only at Yerba Buena’s opening night did we finally see the production in its entirety.

Even as collaborators on this project, we could not have anticipated how powerful and immersive the experience would be for the audience. Throughout the show, the crowd was encouraged to roam and take in all perspectives while dancers emerged from the dark, weaving through the crowds, alluring our eyes to the center of it all; the dance floor. As viewers and contributors, we were captivated by the complex idea, completely engaged with the performance, and left feeling like we just exited a scene from a Sci-Fi film.

The thing we valued most about working with Capacitor and all the creatives behind Synaptic Motion is how collectively, we took something difficult to imagine visually and made it tangible and stimulating.

Our Team on this project:

Wesley Grubbs: Creative Director
Anna Hodgson: Art Direction
Shujian Bu: Lead Engineer
Nick Yahnke: Engineer