I couple of days ago I posted a blog entitled Amazing Invention that linked to a mega-cool 3D animation of a complex musical instrument on YouTube.
At that time I thought this was a one-off, but a reader sent an email to inform me that this was actually the work of a company called AniMusic (for "Animated Music").
The stuff on the AniMusic site looked so cool that I immediately contacted the folks there to find out more. First, take a look at the image of the drums below. Now imagine this as a 3D animated video with the drum sticks playing in exact time with the music (you can see low-resolution video clips on the AniMusic site – also the images I show here are but pale reflections of the high-resolution frames on the DVDs).
"Stick Figures" from AniMusic 1 (Courtesy of AniMusic)
(Click this image to see a larger, more detailed version)
So, why did I emphasize the point about the drum sticks playing in exact time with the music? Well, the way animations are usually done (I won't say "animations like these" because I don't believe that there are any other animations like these) is either...
a) Someone creates an animation and then someone else lays down a music track to accompany it...
... or ...
b) Someone lays down a music track and then someone else creates an animation to accompany it...
In both cases, synchronizing the animation and the music together is a pain in the rear end and you can usually spot "inconsistencies" if you look hard enough. The point is that this is not the way in which AniMusic is created. But before we move to consider how the folks at AniMusic do perform their magic, let's first take a step back to remind ourselves as to what MIDI is and what it does...
MIDI is an acronym that stands for Musical Instrument Digital Interface. This refers to a set of standards that originated circa the mid-1980s to allow different musical instruments to "talk" with each other via digital messages.
In a typical MIDI usage model you need three things: an input device (this is typically a MIDI-enabled keyboard), a sequencer (a hardware device or a software application running on a computer that stores and replays the MIDI data), and an output device in the form of a sound generator such as a synthesizer.
When you press keys on the keyboard, the system stores the speed with which you strike each key, the duration of each note, and so forth (this is, of course, a simplification – there's a lot more to it than we have time for here – but this high-level view will serve the purposes of these discussions). When you come to play your music back, you can tell the synthesizer that you want it to sound like a piano, or a trumpet, or ...
There are also 3D animation programs that allow you to create three-dimensional models of "things", to give these models colors and textures, to gather groups of these models together in a three dimensional world, to set up lights and camera locations, to describe how the objects and cameras will move (and, in the case of the objects, possibly change shape) over time, and to render a 3D animation of the scene.
OK, this is where things start to get clever. The folks at AniMusic have created some software that allows them to create Music-Driven 3D Animations (I'm told that we're now looking at over a million lines of code). First of all they create an excruciatingly accurate 3D model of an instrument that fully details the way in which each part will move, how long it takes to move, and so forth. They also describe the "sound" associated with the instrument (the tone, pitch, timbre, etc. of each note). They then repeat this process to build up an "orchestra" of different instruments.
The clever bit is where the software looks at the music to see what's coming up and decides what it has to start doing now in order to make each note occur at the exact right time in the future. What do I mean by this? Well, suppose that we return to the real world and that we have a row of different sized bells hanging from a rod. Let's assume that each bell has an air-powered "cannon" associated with it.
Now let's suppose that the row of cannons is about 20 feet from the row of bells. When we press a button on one of our cannons, it fires a tennis ball towards its associated bell. Of course the balls take some amount of time to reach the bells, so if we are trying to play a tune, we will have to keep this "time-of-flight" in mind. (Things will get more complicated if we have the ability to fire the balls at different speeds depending on how loud we want the bells to sound, because the time-of-flight will vary depending on the velocity of the ball.) Now replace our pressing the cannon buttons with the AniMusic software working in conjunction with a MIDI sequencer ... and you sort of have the idea (if you revisit that Amazing Invention video on YouTube you'l see it in a new light).
Suffice it to say that this is all mega-cool. The previous image was from the AniMusic 1 DVD. Titled Stick Figures. This obviously features drums (although there are other instruments). Another AniMusic 1 favorite for me was the Aqua Harp track from whence the following image originates:
"Aqua Harp" from AniMusic 1 (Courtesy of AniMusic)
(Click this image to see a larger, more detailed version)
As you can see, this is a complicated instrument (as far as the AniMusic software in concerned, this is actually a number of different instruments gathered together in a single frame). The blue plectrums in the middle are plucking the harp-like strings; the plectrums at either end are holding the strings that are being played by the bows; the little pipes at the top act like a miniature steam-powered organ; while the two hammers shown resting on the base eventually start to ring the "tubular bells" hanging underneath the frame. And then things start to get complicated...
Now, don't get me wrong with what I am about to say, because I absolutely love all of the tracks and animations on AniMusic 1. However, a lot (not all) of the instruments were playing themselves as if by magic (drumsticks and plectrums flaying around on their own with no means of support, and so forth), which – for me – detracted from the "realism" just a tad.
Having said this, and on further contemplation, I think that my real issues here are that (a) I love robots and complex mechanical assemblies and (b) I was spoiled by the AniMusic 2 DVD. To put this another way... I would die a happy man if I had only ever seen AniMusic 1, but AniMusic 2 took things to a completely new level for me. Consider the following image from Resonant Chamber on AniMusic 2:
" Resonant Chamber " from AniMusic 2 (Courtesy of AniMusic)
(Click this image to see a larger, more detailed version)
Now, imagine all of these mechanical fingers pressing and plucking strings in perfect time with the most amazing music. Also, you have to understand that we're not talking about fixed camera positions for each piece of music. The whole thing is completely choreographed with multiple camera angles and moving cameras and changing lighting conditions and ... it blows your mind away.
My favorite? They are all my favorites but – if you were to push me – I have especial fondness for Pipe Dream from AniMusic 1 and Pogo Sticks from AniMusic 2. And I for one CANNOT WAIT for the forthcoming AniMusic 3, which does not yet have a firm release date ("it will be ready when it's ready") but I am more than happy to wait because I know the result is going to blow the socks off my feet!
You really MUST see these DVDs ... speaking of which ... the folks at AniMusic were kind enough to give me a spare copy of AniMusic 1 and another of AniMusic 2 (no one is going to get their hands on my own personal copies). But how am I to decide to whom I should present these little beauties?
Hmmm... do you remember my previous blogs on the Bulwer-Lytton Fiction Contest? The idea is to come up with the worst opening sentence for a fictional novel (I've still not heard anything from My Entry to the 2008 Competition ... I hope my prize didn't get lost in the post).
Well, I will send these two DVDs to whoever sends me the worst opening sentence for a technical book on FPGAs. Note that is shouldn't be from a real book – this is one you have to make up for yourself. Also, we're looking for a single sentence (not a paragraph) whose prose is dreadful but which is grammatically correct (or at least as grammatically correct as we care about considering we're engineers).
The deadline is to have all email entries sitting in my "Inbox" by the time my bum lands on my seat in my office by next Tuesday morning (4th November). At that time I'll post a blog containing the winning entry and any others that particularly tickled my fancy!
NOTE: Don't forget our Design Competition. In addition to fame beyond the ken of mortal man, the prize of an Altium Innovation Station and a perpetual software license is worth US$9000+!
Questions? Comments? Feel free to email me – Clive "Max" Maxfield – at max@techbites.com). And, of course, if you haven't already done so, don't forget to Sign Up for our weekly Programmable Logic DesignLine Newsletter.
