Piano Playing Robot
Submitted by William M, Malta
Introduction: As the title of this article suggests, this robot was designed to play the piano. The interesting thing about the whole project is that the program, which I wrote in Clarion, was designed to play any song imaginable. I will explain this further on in the Software section of this article. To begin with, I do not know music, I never studied music, so even less so did I know how to play the piano; so my first task was to do all three. I bought a second hand 61-keyboard MK-928 synthesizer on the eve of the New Year of 2011, and through the internet, I started learning the basic theory of music. One month down the line and I was ready to start my project. The parts I purchased from PC Control to use in this project were the Maxi Bee, two Bee Drivers and 24 pull-type 12V solenoids. The project was basically divided into two parts; on the one hand I had to write the software to play the music. On the other hand I had to translate the notes/chords I wrote in the program into actual movements in the physical world.
The software part of this project can be broadly divided into four parts
• the part to create the wave files (.wav) of all the possible notes’ and chords’ to be played when running my program, and the corresponding jpegs (.jpg) to see what is being played on screen.
• the part to convert a note/chord into a decimal number.
• the part to create the particular song file.
• and finally that part of the software to send out the signals to the piano in the physical world. These are the signals I am sending to the Maxi Bee and the two Bee Drivers to actuate the solenoids on the piano, which ultimately trigger the hammers to press or release the piano keys.
Software - Creating the notes and chords: To begin with, the piano is a 61-key piano, but from the start, and particularly to keep costs low (for every note played on the piano I have to use one PC Control solenoid), I wanted to play two tunes that only utilised 24 keys (notes/chords) from the possible 61 keys, where all the notes/chords in the songs fell clearly within the two octaves. The two songs I chose were Danny Boy (Piano) and Jesu, Joy of Man’s Desiring (Church Organ). For the music savvy, music is played in octaves, and each octave comprises 12 notes, each starting from note C; hence the 24 keys covered two octaves. .
In the image on the left, the yellow hammer signals the start of an octave, starting from note C. The yellow and grey LEGO hammers hit the white keys on the piano keyboard. The five black LEGO hammers hit the black keys on the piano keyboard. Since a 61-key piano has five octaves, I chose to start from the octave containing Middle C. In this case Middle C was in the 3rd octave (yellow hammer on the left). My piano robot covered the 3rd and 4th octaves. As my intention is to, one day cover the entire piano to play more complex songs, I created all 56 notes (it’s the most I can cover with the Maxi Bee) and a number of chords in my program. To create each note/chord I had to create the wave file and also the jpeg of that note/chord. In the figure below you can see the jpeg for Middle C
In this other figure below you can see this same Middle C note in the Table of Notes.
I also had to define simple chords. A chord is a number of notes (generally three or more notes) played together simultaneously. In the figure below you can see the chord Octave 3 A Minor
Software - Converting a note or a chord into a decimal number: It was important for me to give a unique decimal value for each note I created. In the table below I am showing the value for every note in the 3rd and 4th octaves. .
Middle C is the first note in the 3rd octave. It’s value is 65536 (see to the left and below). I created a tool below to automatically convert every note in both octaves on the piano into a unique decimal number. Since, as you can see from above, the numbers can be quite large, this tool helped me mitigate any assignment errors in my coding. It proved especially useful when calculating the value of a chord.
The decimal value of a chord is arrived at by adding the individual values of each of the notes making up that chord (see following section to convert notes and chords into decimal numbers). In the case of Octave 3 A Minor, the value of the chord is 67,108,881 and this is made up of note A in the 3rd octave (value = 67,108,864), note C in the 4th octave (value = 1) and note E also in the 4th octave (value = 16). Together these add up to 67,108,881. This is the value I send out to the Maxi Bee to play chord A Minor in Octave 3.
Software - Creating a Song: Next, I had to create a Songs Table. Every song had to be broken down into its constituent notes or chords each having certain parameters attributed to it. These parameters were primarily the sequence number of the note/chord, the output value in decimal to send to the Maxi Bee, the decimal number representing the duration of the note/chord, a repeat note/chord flag where a note had to be repeated as and where necessary, as well as the .wav file location and the .jpg file location to play and see whilst the song was being played on my computer. The wave file was only needed for testing purposes (without using the piano robot), because it was the only way I could ensure that I was playing the right notes/chords and to listen to the duration of each. When the song was completed, I used a software switch to disable playing the wave file whilst playing the song directly on the piano robot. In music, duration refers to an amount of time or a particular time interval. It is an important property of a note/chord that forms one of the bases of rhythm within a song. Durations may be described as short, long or taking a certain amount of time. The duration is the time that has to elapse from the moment a note/chord is played (the key or keys pressed) to the time when the note/chord is to be released. In the figure below you can see that the song Jesu, Joy of Man’s Desiring by Johann Sebastian Bach is made up of 168 notes and chords. Every record in the grid (table) is a note or a chord. The 85th note in the song is in fact a chord.
The decimal output for this chord is 71,303,168 and its duration is 320. This latter number is then multiplied to an internal constant note duration multiplier, which is 1,000,000 for this song, to act as the limit to be used in a time delay loop. Duration for each note chord is very much a trial and error process and I changed the value until I felt that the duration of the note/chord sounded right. The duration multiplier remained constant throughout a song.
Software - Sending out the signals to the piano: When playing a song the program reads the Song Table and reads and executes one record (note/chord) after another in incrementing sequence order, sending out a decimal number to the Maxi Bee. In reality, this is not quite correct. The dll supplied with the Maxi Bee provides for two output signals, both in decimal format; one value is stored in the variable Outputs1 and the other in Outputs2. This is a big plus for my project as with one Maxi Bee and four Bee Drivers I could control 48 keys on the piano. In reality, I could control 56 keys as I have two unused outputs on each Bee Drivers board which I am not using, and which I can use to control an additional eight notes in the fifth octave
Hardware: For the purpose of this project I only had to use one variable, Outputs1, to control two octaves. So I only needed one Maxi Bee board and two Digital Bee boards, one board for each octave. I also needed twenty four 12V pull type solenoids supplied by PC Control. To put it simply, this was more or less equivalent to playing the piano with one hand only, in this case, the right hand. Once the Maxi Bee receives the decimal value in the variable Outpus1, it outputs that signal to the corresponding Digital Bee which in turn actuates one or a number of solenoids in the two octaves controlled by that Digital Bee. As I said earlier, the solenoids are of the pull type. So once a solenoid is actuated, the electromagnetism created will pull the plunger down. The plunger has a machined internal thread to accept an M3 screw which is supplied with the solenoid. This makes it very easy to attach external mechanical parts, which is precisely what I did. I attached some LEGO parts to the plunger to act as a hammer on top of each key. Additionally, the pull type solenoids come with a spring which is used to provide the “return” force when the solenoid is de-activated. The return force sets in once the duration of the note is exhausted and the electric current to the solenoid is terminated
Conclusions: It is my intention, one day, to extend the piano robot to cover the entire keyboard. One Maxi Bee board will cover the entire keyboard, but I would have to increase the Bee Drivers from two boards to four boards. I would also need to increase the solenoids from twenty four to fifty six. In so doing, I would be able to play the piano robot with “two hands”. It would be interesting to see how the piano robot would handle some complex music like Beethoven’s 5th Symphony, or Coldplay’s Viva la Vida.
See my piano robot on YouTube by clicking on this link ..
You may also see my other robots on You Tube...
Rubik Cube Solver robot at http://www.youtube.com/watch?v=-oYLL6kr1nM
Robot Realtime Weatherman at http://www.youtube.com/watch?v=tLrNp5sJBlo
Solving 9-Letter Word at http://www.youtube.com/watch?v=nErjb7vO3cQ
PC Control Ltd: All articles are publish as submitted in good faith but we do not accept any responsibility for the authenticity or content of these articles. Any article breaching copyright will be removed from this website as soon as we are made aware.