GMS The Gestural Music Sequencer
For my final project I would like to do a performance using the GMS, a gestural music sequencer that I've been developing in Processing (processing.org). The application samples video and displays it inverted so it looks as though you’re looking into a mirror. Each frame is analyzed for brightness, then the X and Y data of the brightest pixel is converted into a MIDI note. The X axis is used to select a pitch, while the Y axis determines the dynamics. As users move, dance, gesture, or draw, notes are generated based on a predetermined scale. Currently the available scales are pentatonic minor, whole tone, major, minor, and chromatic, all of which can be dynamically selected during a performance. Other dynamic controls include transposition, sustain, duration selection (manual or randomized with probability distributions), BPM adjustment, and note randomization. A "free" mode allows the durations to be manipulated by the mean brightness of the video input. The application works especially well in dark lighting while using a light source to control the sequencer.
Using this tool for an outing may produce some interesting results. I would like the piece to be collaborative involving a minimum of two artists and potentially the involvement of passers by. This way someone can be producing the visual input while another person is controlling the software. The controller can then react to the visual input by changing the key, speeding up the tempo, or manipulating the duration and color of the notes. Currently the software is still under development, so I have not yet written in any video processing. If this remains the case it will be important to either use a camera that has built in effects, or prepare visuals that work in the piece without the need for processing.
The sound will be an integral part of the performance; therefore, amplification and location are important factors for the success of the piece. On Main Street, Northeast under the Hennepin Avenue overpass by St. Anthony Main might be a good location. The sound is likely to reverberate prominently with plenty of volume. Another important factor is rehearsal. Although anyone can make music instantly with the GMS it takes some practice to understand how to use it effectively.
Thanks to Ali for all his suggestions including the use of probability distributions, Markov chains, vectors, and loads of other brilliant ideas that made my head spin. More details on the GMS including images, video, sounds, and writings about the development process can be found here: