Two of Bocko's doctoral students, Xiaoxiao Dong and Mark Sterling, worked with Bocko to measure every aspect of a clarinet that affects its sound—from the backpressure in the mouthpiece for every different fingering, to the way sound radiates from the instrument. They then built a computer model of the clarinet, and the result is a virtual instrument built entirely from the real-world acoustical measurements.
I wonder how many variables it would need to capture differences in human performance. Will it faithfully capture the differences between two world-class players? Glenn Gould and Evgeny Kissin? How?
If it manages to do that, it may incidentally help us understand feelings, or personality. The captured data could be mined to search for the pattern of joy or sadness, for example. Furthermore, given enough data, software could synthethize new pieces. Kissin never blayed Bartok? No problem, just ask the software to 'play' Bartok in his style.
..so in other words, the real implications of this isn't in music compression, it's in simulation/synthesis of musical instruments. This is an exciting field to be sure, with enough unsolved mysteries for anyone. I am still hungering for the abolishment of the guitar amplifier.
Actually, there's this interesting software package written in Common LISP quite recently for performing versatile (audio) signal synthesis/simulation using different physical or abstract models. I haven't had a chance to look at it (and, come to think of it, I probably suck enough in LISP to turn it into a real chore), but it's at http://www.acoustics.hut.fi/software/BlockCompiler/ if anyone is interested.
Exactly. Don't look for this to reduce the size of your music collection until we can reverse-engineer physical models of entire bands and recording studios from the sounds they make.
Well, there was a submitted story some time ago that showed new technology that can recognize the individual notes of instruments (at some degree of accuracy) and they could then make adjustments... So, it doesn't sound completely impossible.
Synthesis is basically just a form of lossy compression (or lossless, if the original was synthesized as well, with a lesser or equivalent amount of entropy.) For one practical example, think of fractal compression of images--a series of mathematical equations are synthesized that just happen to best approximate the original image.
"Humans can manipulate their tongue, breath, and fingers only so fast, so in theory we shouldn't really have to measure the music many thousands of times a second like we do on a CD."
This seems widely flawed. Changes in play are limited by the dexterity of the player (and upper bounded by the dexterity of a body), but choices and timings aren't so mechanical.
The exact timing (feel, even) of, say, a swung eighth requires quite a lot of resolution before it's done right. I'd love to see them, even after they get tonguing "down", do a double blind study in Jazz.
I think their point is not that the resolution of human data is lower (although I expect it is quite a bit lower than 44k Hz). Instead, I think they are pointing out that fewer data points are necessary.
If your jazz drummer plays eight notes in a second, the timing alone can be represented in eight floating point numbers (a higher resolution than CD audio); the data is 8 * 8 bytes -- much smaller than the 44k * 4 bytes of raw audio samples that would be needed for a second of stereo CD audio. (I realize a drummer's performance encompasses several other variables, but these data points are similarly sparse relative to CD audio.)
Wouldn't the computer have to solve the cocktail party problem and have models for each identified channel for this to work for normal music? And, whooptee doo, it'll still probably sound like shit underwater in a plastic bag.
It's really just MIDI with more variables.