Audio Compression Encoder Testing
Jussi Laako
If you've ever encoded a piece of music with two encoders to see if there was any difference, you might have been surprised to find that the differences could be way more than subtle.
Jussi Laako has been testing various mp3 encoders plus Ogg, WMA, and
RA8. The results put the order as mp3, Ogg, and WMA and RA8 in equal third.
Of the mp3 encoders, Fraunhofer might be best sometimes and the others at
other times: There are plenty of gotcha's. Have a look at the graphs
yourself...
related links:
Test Graphs and Commentary
Libdsp
What are you testing in addition to Ogg?
mp3: Blade and lame are both mp3 encoders. Lame is the most commonly used encoder backend of free audio rip/encoder programs. Test results for Fraunhofer's mp3 encoder along with wma and realaudio are coming (Ed: And are now up).
I don't imagine you'll try it but it would be interesting if you could. The mp3 encoder plugin that you can get for pro tools was subjectively so much nicer than Blade that I'd be interested to see an official plot. I'm not sure but I think this plugin might have some relationship to the Fraunhofer encoder.
I don't have Pro Tools available, but the Fraunhofer codec used for the result now available at the updated web page (along with WMA, RA8 and RM) is the one that comes with Cakewalk SONAR 1.x.
I haven't done any listening tests comparing Ogg, Lame, and Blade. Do your graphs bear out the listening experience (especially with Ogg) or were you a little surprised with any of the results? I suppose that a lot of people would listen to compressed audio on inferior speakers with truncated top ends, which would account for a lack of anecdotal "bad mouthing" of Ogg.
I couldn't make my mind up on subjective listening tests of ogg. It doesn't cut the higher end as most mp3 encoders do (at lower bitrates), but it sounds somewhat blurred. Measured results also show this. It removes significant amount of high frequencies around 10 kHz while also boosting the remaining frequencies. However, it performs better at 100-1000 Hz range.
Another important aspect of sound quality is transient performance. I'm currently developing methods to test this.
That will be interesting. Broadly, what method are you thinking of using?
Spectrogram of the difference signal probably, along with ordinary spectrograms of the signal. I'll have to find out how to visualize this properly. And possibly some wavelet-based analysis.
In comparing a reference signal to the ones which went through codecs, how did you go about this? Did you record a frequency sweep, encode it, and then compare the results?
I used carefully chosen music samples to do the tests. Artificial test signals are not suitable for testing lossy audio codecs.
The samples in this test were recorded from analog source.
A sample (in WAV format) is first encoded with specific parameters and then decoded back to WAV using a reference decoder. Then it is compared against the original sample with my test software.
I have also tested most available mp3 decoder/players but there are no significant differences between them (compared to encoders).
The Libdsp page is "closed" at the moment to protest software patents in Europe. Would you like to tell us about Libdsp?
I have now reopened it. It was closed for a few days for this protest.
libDSP is a project to create a library of generic high performance digital signal processing functions for programmers to use. It is distributed under GPL license so it can be used by other GPL (or compatible) licensed software.
Thanks Jussi.
Bookmark:
post to Delicious Digg Reddit Facebook StumbleUpon
Recent on Mstation: music: Vivian Girls, America's Cup, music: Too Young to Fall..., music: Pains of Being Pure At Heart, Berlin Lakes, music: Atarah Valentine, Travel - Copenhagen, House in the Desert
front page / music / software / games / hardware /wetware / guides / books / art / search / travel /rss / podcasts / contact us