End of project results: video

This final video is the culmination of my semester's work, investigating pixel-based music notation and the hidden aural beauty within the shapes and patterns of everyday visuals.


Progress and Patterns

Pixel-based musical notation. That's what I'm still experimenting with.
Formerly I was taking a photographic narrative approach and creating a musical score based on a series of images. This was okay, but it turns out people are actually more interested in how changing the graphic changes the resulting audio. It's also interesting to see what type of image (photograph, in this case) creates the most appealing or dynamic range of sounds.

Through breaking down many different photographs into pixel based musical notation, I've found the most exciting audio comes from images that have a varying, somewhat vertical pattern. Diagonally balanced patterns are also very nice.


(As a reminder, the audio notation reads from left to right. The pitch is determined by the height of a pixel, and the brighter the pixel, the louder the audio value.)

I will edit these examples to play in the same key for comparison's sake.

A graphic of simple horizontal lines has no rhythm and no variations in melody:


A graphic of vertical lines has an obvious rhythm but also has no movement of melody:


So there is definite motivation to use images that actually will make for interesting audio.

For my final end product this semester, I will still be making a video with the images and their corresponding sounds, but instead of using narrative photographs, I have chosen to just photograph things I see within walking distance of my house that I think will sound the best. I am mostly looking for interesting patterns that transform both horizontally and vertically.

Here is one example:



Depending on how I change this bitmap image, the sound also changes. To show this relationship, I will also include different variations for each photograph (which will be included in the video).

Compare the following graphic and audio to the previous one. Notice how the more elongated tones in the second example match the longer horizontal groups of pixels.



I will also include all the edited graphics of the notation in my final body of work.

Pixel-Based Music Project

I've decided to do some work exploring the relationship between aural and visual language. I have been experimenting with different ways of making sounds based on bitmap images (playing a picture like a musical score).

Through a software program I've been able to open a bitmap file of a photo, where it's then broken down into fewer pixels. The musical "score" is based on these pixels, and played from left to right. The height of each pixel determines the pitch, the color determines where the note will be played on the stereo field (left or right speaker), and the brightness determines the volume.

Within these guidelines, the pitches can be redefined to fit a particular musical scale. Removing specific horizontal lines of pixels accomplishes this by eliminating all notes that do not fit within that scale. Further manipulation of the image results in multiple variations of sounds.

Here are a couple examples, showing the original photograph, the pixelized png (stretched out to match the length of the audio file), the further edited png, and the resulting MP3 file (looped).









Using this method, I plan on composing a longer-length (2-3 minute) song, based on a series of original photographs that will tell a narrative on their own. After this process is complete, I will create a movie file that scrolls through the line of photographs as the resulting audio file plays along in real time, showing both ways of communicating the same story via distinctly different processes.

Digital Focus | Project Overview

| 1 Comment

Throughout the next 13 weeks I will be conducting a series of experiments that explore the impact of specific environmental factors on visual communication.

I will be updating my blog on an experiment by experiment basis, documenting my process and reflecting on the conceptual aspects of each.

Some environmental factors that I may explore include:

Forms cut from reflective vinyl laid on plain vinyl of the same color, the image appearing only under certain lighting circumstances. Some of my inspiration:

Bright Bike from Michael Mandiberg on Vimeo.

Freezing images and typography in ice. This idea was inspired by a discarded love letter I noticed frozen under about 1/2 inch of ice on Tischer Creek in Duluth, the translucent nature of ice allowing only certain words to be visible.

Andy Goldsworthy
View image


View image

Recording the audio waveforms of ugly, beautiful, or otherwise pointed phrases, and showing the resulting visual aesthetic, juxtaposed against contrasting words or images to expose new meaning. Here's an example of some of the visual forms audio waveforms can take:

Cymatics - the study of visible sound and vibration

Project 1: Avatar

My avatar comes from the virtual video game world of Morrowind.
The base character and scenery are from Morrowind. I've changed the face and added a few objects in Photoshop (bike, kayak, etc.).

About This Blog

This blog is a requirement for Digital Studio II to archive my process for the class.

Recent Comments

  • nels5607: This sounds very interesting. I'm looking forward to seeing how read more

Recent Assets

  • coiledwire2bsmall.jpg
  • coiledwire2_small.jpg
  • coiledwire_small.jpg
  • vert_lines.bmp
  • horiz_lines.bmp
  • gma3.jpg
  • gma2.jpg
  • gma1.jpg
  • rug3.jpg
  • rug2.jpg

Find recent content on the main index or look in the archives to find all content.