The twins jumping on the bed trying out my virtual keyboard. Using techniques from the book “Making Things See” by Greg Borenstein, I was able to create a virtual matrix that floats above the bed. As the girls move through the matrix it plays different sounds based on their location.
This is one of the first iterations. I was trying out sounds to get a nice interaction that would be easy for the girls to anticipate and would hopefully spark more intrigue. This was a pentatonic scale of a single octave offset by 1 across the matrix. It still sounds fairly difficult to anticipate for the girls, but I think the piano works well. Next I hope to add more music and extend it by adding the skeleton mapping to allow for instrument selection that produces the sound. The instrument will be chosen by the location of the hands when an event is triggered. If the hands are at the side it will play piano. If one hand is at the shoulders it will play a stringed instrument along with piano and so on.
I want to add thanks to my neighbor Brian for his help getting the piano sounds, and to my sister Lorri for instructing me to use a pentatonic scale as well as provide a useful placement those notes on the matrix.
Mar 252012