Hanno's blog

I've promised that I'll dig into some old file formats and check how well they can be accessed on today's systems with free software.

Today, I'll start with audio formats. To begin, in general there are two kinds of audio formats. Streamed audio formats start with a more or less raw audio stream, apply some encoding and sometimes (lossless or lossy) compression. There are also tracker audio formats. They have internal information on tone pitches and instruments. Most really old computer audio files are tracker formats (like the popular C64 SID format). This blog post will be about streamed audio formats and I'll save the tracked ones for a later one.

The file formats I've chosen are more or less random, the main criteria being that I once stepped over them and still remember that. There's a hughe collection of all kinds of media file samples on the mplayer server.

The single most important project regarding exotic audio or video formats is ffmpeg, a library that does despite its name much more than decoding mpeg. All major free software media players use ffmpeg.

The file formats I've investigated:

• Some of the very first files distributing music through the Internet I remember were real audio files (extension .ra or .rm) from the german punk band WIZO. Real audio has a whole bunch of variants, scanning through some of my old backups, most of them used either AC-3 or Real Audio 2.0 as their codec. Thanks to Waybach Machine, you can still find the WIZO downloads (Raum der Zeit - Techno is AC-3, the others are RealAudio 2.0).
• vqf (or TwinVQ) was once announced having better quality than MP3 and was discussed as its successor. However, it seems it is almost completely distinct today, I didn't find anything at all (except in the above mentioned sample collection) in vqf format for download.
• Monkey's audio, extension .ape, is a lossless audio codec, which is itself licensed under some kind of noncommercial-use-only license that doesn't qualify as free software. It's not really old, as it's still being developed, but I added it as another example of an uncommon format.
• Shorten (extension shn) is an old lossless audio format, which was often used by the etree project that collects recordings of concerts. Today, it is mostly deprecated by flac, but the old recordings are still available.
• voc: The popular dos floppy copying program vgacopy had sound before I had a soundcard - it used the pc speaker to play .voc files it had shipped. It's a format used by some Creative software for their SoundBlaster. It's a more-or-less raw audio format like wav.

ffmpeg is able to decode and play all of these audio codecs. But what I found out was that this doesn't necessarily mean every application using ffmpeg can do this as well. I've tested mplayer, xine, vlc, audacious and totem (based on gstreamer). Although there are quite many free software media players - both for audio and video - out there, this should cover pretty much everything. Most media players use xine, vlc or gstreamer indirectly.

	mplayer	xine	vlc	audacious	totem/gstreamer
ra AC3	Yes	No Yes	Yes	Yes	No
ra 2.0	Yes	No	No	Yes	No
vqf	Yes	No	Yes	Yes	No
ape	Yes	No	No	Yes	Yes
shn	Yes	Yes	No	Yes	Yes
voc	Yes	Scratchy	Scratchy	No	No

Shorten playback has some problems, seeking often does not work, but this seems to be a limitation of the format itself. If I found feature requests for those formats, I've linked them, I also opened a bunch of them myself.

Conclusion: ffmpeg does a really fine job in playing all the obscure audio streaming formats. However, not every player that's based on ffmpeg plays every format ffmpeg can play. mplayer is the only player that succeeds with everything, probably because mplayer's devleopment is very tightly related to ffmpeg's development.

Update: I forgot to mention libav. It is a fork of ffmpeg. However, there's not that much to say, as ffmpeg and libav are still quite similar in their codec support. audacious does not support libav yet, all other apps just produce the same result.

About one year ago, Sam Hocevar posted some results on tests with his fuzzing tool zzuf, which showed a large number of crashes in various applications, especially multimedia apps.
Crash bugs on invalid input very often lead to security issues, thus this should be taken seriously.

Now, I took the freedom to have a look how many of the issues found back then were fixed. I used the most current versions in gentoo linux (testing/~x86-system), which tend to be quite up-to-date. I also cross-checked the crashes for other apps, as they often use the same or similar code.
Seems only vlc devs did their homework (Sam Hocevar is part of the vlc team). Interesting enough, even firefox seems to have a gif-crasher since a year.

gstreamer crash by lol-ffplay.mpg lol-gstreamer.m2v lol-mplayer.m2v lol-mplayer.mpg lol-vlc.m2v lol-vlc.mpg
endless loop by lol-ffplay.m2v lol-xine.mpg

mplayer hang by lol-mplayer.wmv,
crash by lol-ffplay.flac lol-mplayer.aac lol-mplayer.mpg lol-mplayer.ogg lol-ogg123.flac lol-vlc.aac lol-xine.aac

xine crash by lol-mplayer.wmv lol-ffplay.m2v lol-ffplay.ogg lol-ffplay.wmv lol-gstreamer.avi lol-ogg123.flac lol-vlc.aac lol-xine.mpg

firefox crash by lol-firefox.gif

Thanks to Kostya, ffmpeg now supports the VC-1 codec, which is also used in Microsofts WMV9-format.

In the past, for Linux-users it was only possible to play those videos with win32codecs, which use proprietary windows-dlls and don't work on non-x86-platforms. With this improvement, one of the most common multimedia-formats unsupported by free software can now be played with all major free players (as ffmpeg is used by vlc, xine, mplayer, totem and many others).

Congrats to the ffmpeg-guys. The easiest way to get stuff playing is getting mplayer from svn (I used the mplayer-svn-ebuild from this Multimedia Overlay with some small modifications).

(via Breaking Eggs and making Omelettes)