Music Similarity Measurement Using Lossy Compression
information processing algorithm
There are two sound clips.
It is very similar. I want to measure how similar they are perceptually, i.e. how similar humans perceive them to be. Is there a way to compare two clips using a lossy compression algorithm like MP3, AAC, or the Ogg Vorbis encoder?
It occurred to me that audio compressors already have a great deal of knowledge about psychoacoustics and human perception of sound built into them. Is there a good way to use them to measure how similar two clips are?
maybe something like L(C1||C2)/(L(C1)+L(C2))L(C1||C2)/(L(C1)+L(C2)), where L(X)L(X) is the compressed duration of the sound clipXX, and C1||C2C1||C2Is the result of concatenating the two clips? Or maybe find the highest bitrate, like this F(C1)F(C1)nearF(C2))F(C2))By a simple metric (for example, the L2 norm applied to the FFT spectrum), whereF( C)F(C)is the result of compressing CCat that bitrate and then decompressing it? Or something like that? Has anyone investigated this?
If it matters, the two clips are very similar: one is obtained by transforming the other. They are aligned in time and have the same length. Each one is relatively short (a few seconds at most). I searched around a bit, but didn’t find any references or research papers that discussed this approach, although maybe I haven’t.
Is there a good way to use them to measure how similar two clips are?
It’s a pretty sensible intuition, but one that I rarely see in the literature, aside from the peripheral idea of feature extraction from a compressed stream (the motivation here is simply to reduce the computational load).
I think the main reason is that audio codecs don’t store perceptual representations of audio data. Instead, they store a very code-centric representation of the audio (entropy coding, quantization, transform coefficients) chosen so that the distortion between the original and encoded signals is below a perceptual threshold. In a sense, looking at a compressed audio stream doesn’t tell you how a human would perceive an audio signal. It simply tells you that something below the perception threshold has been taken. The codec “sculpts” a meaningful audio signal, but never touches it.
where L(x) is the compressed duration of sound clip x and C1||C2 is the result of concatenating the two clips.
This makes sense on strings, I think the idea is found in some Cilibrasi or Vitanyi articles, but that’s not how audio codecs work. The audio codecs you mention are designed to handle streaming audio with low latency, so they only “see” a short window of the signal at a time and don’t try to eliminate redundancy in the long run. As a JPEG of an image. Copying 10 instances of the same note one after another will result in a file that is 10 times larger. Audio codecs have either a fixed bitrate or an adaptive bitrate that adjusts based on what the codec sees in a narrow signal window.
The redundancy that audio codecs eliminate is very short-term.
The idea of it will require a more complex “object-oriented” codec that tries to break down audio into a hierarchy of objects, like musical notes. The idea is still in its infancy in the academic world.
Another way to look at it is to check if the decoding process is similar to the generative model of the signal. This is the case for some string compression algorithms, where the compression process restores a string-like generative grammar (Sequitur, factor oracles…). This is the case for LPC-based speech codecs, where the LP coefficients incorporate knowledge about pronunciation and the remaining knowledge about prosody and vocalization/absence of speech: the compressed stream can be considered as data for a speech synthesizer. However, this is not the case for the generic audio codecs you mentioned: codecs don’t resemble the music production process.
