TECHNOLOGY

Interaction: Real-Time Media Systems

Interactivity has been an essential and integral component of Troika Ranch’s work since the company’s inception. Interactivity is the energetic linkage of human action to digital content and the feedback loop that is formed by this relationship – performer responds to system, system reacts to performer, performer makes informed choice to respond to system, etc. This linkage is achieved by using cutting-edge computer technology in tandem with custom sensory technology that tracks the movement and vocalization of performers on stage or viewers in an installation. By using these systems a bending arm can warp a video image or the kick of a leg can recall a musical phrase. This interaction allows a performer or viewer to follow their instincts from moment to moment, making improvisational choices that subtly or profoundly change the visual and aural content of a work. Coniglio and Stoppiello want all of the digital media in their work to have the same sense of dynamism, vitality and “liveness” as the performers themselves.

 

Tools like video, digital media software packages and tele-presence are important to us because art, to be relevant in contemporary society, must simultaneously embrace and examine contemporary culture and currency. Integrating emergent media keeps live theater vital in a world where the most widely experienced channel of aesthetic expression is broadcast media. But, most importantly, it imparts a real and necessary density that mirrors the media intense world in which we live.

 

Isadora® is the software “brain” behind Troika Ranch performances and is the main technology used to generate the imagery and sound for Troika Ranch’s current projects. Created by Co-Director Mark Coniglio, Isadora is a graphic programming environment that provides human control over digital media, with special emphasis on real-time manipulation of digital video. Isadora gathers movement information from various sensory devices and uses that information to control and manipulate digital video, music synthesizers, sound modulation devices, theatrical lighting and robotic set pieces. (http://www.troikatronix.com)

 

A typical setup for a performance or installation includes one or more movement sensing devices that communicate with an offstage computer. A sensing device is a piece of equipment programmed to measure the movements or vocalizations of performers. One sensing device Troika Ranch developed and implemented is the MidiDancer; a bodysuit outfitted with plastic fibers that measure the flexion and extension of the major joints on the body. When a joint moves, it sends a signal to the computer. This signal is then transformed into code that manipulates other theatrical equipment – video cameras or digital sound devices. Video and sound are thereby manipulated directly in relation to a specific movement. For example, when a performer’s arm is straight, the sound is very soft; as the performer bends her arm, the sound amplifies. This system was first used while Coniglio and Stoppiello were students at CalArts in 1989. The two went on to create several works that feature the MidiDancer including their seminal work In Plane (1994), a duet for a dancer and her video representation. Using the MidiDancer, the performer controls the generation of music, the recall of video images, the manipulation of theatrical lighting and the movements of a robotically controlled video projector. This work has been analyzed and written about in numerous academic papers and journals.

 

Since the late 1980’s Troika Ranch has continued to advance and improve their sensory devices. Troika Ranch’s current sensory device uses a single video camera placed on stage. This video camera is connected to a computer that runs specially designed software (EyesWeb, http://www.eyesweb.org) that imposes a 12-point virtual skeleton onto the body as seen by the camera. The points of the skeleton are then linked to Isadora, which generates imagery in real-time response to the movement of one or all of those points. Troika Ranch also uses Infrared light as a way to be able to track bodies and project images into the same space. By flooding the “back wall” with infrared light, an infrared camera can “see” a black silhouette of a performer in front of a white wall and “ignore” any video projected onto that wall.