That Which Pulls

fixed video
In his book Digital Harmony, John Whitney describes his technique of "differential motion", an approach to animation derived from musical theories of harmonic structure. Studying his techniques has led me to apply these methods to sonic and visual content as a method of generating integrated audiovisual works. While Whitney's pieces generate gestalt formations by positioning visual elements along similar paths at harmonically related rates, That Which Pulls resynthesizes sounds and images by using differential motion to shuffle their data. Within this piece I apply this process to the pixels of a single image and the samples of a single note to generate the audiovisual material.
The manipulated image takes on gestural qualities that are functionally similar to Whitney's animated elements: the pixels arrange and rearrange themselves, alternately descending into chaos and resurfacing in a new order. An unexpected side effect was the Moiré patterns that emerge as the pixels realign near the different nodes, emphasizing the resolution. Within the audio, the single note quickly becomes multiple notes as the samples realign near the nodes, with complex syncopated rhythms appearing in the later iterations. These two streams of raw material were cut up, layered, and recombined to create a piece that explores the dynamic interplay between the emergent auditory and visual gestures, focusing on the counterpoint between their patterns of chaos and resolution.
Note that the vimeo link has some texture degradation due to bit rate restrictions. You can see examples of the full detail by clicking the images above, and also download this hd file if you want to see it at full quality.
The score for That Which Pulls: click to see it full size along with its text. For a more detailed description of the piece, you can view my paper, "Harmonic shuffling: Waveform and image resynthesis in That Which Pulls.