English - 6

Just recently, Microsoft research has made some progress in developing a device to do live translations from English into Mandarin. I'd like to share some thoughts with you about that.

If you read my blog on a regular basis, you will know that I traveled through Russia, Mongolia and China last year. If there's one big thing I learned on this trip, it's this: English language is - on a worldwide scale - much less prevalent than I thought. Call me a fool, but I just wasn't aware of that. I thought, okay, maybe many people won't understand English, but at least I'll always be able to find someone nearby who's able to translate. That just wasn't the case. I spent days in cities where I met nobody that shared any language knowledge with me.

I'm pretty sure that translation technologies will become really important in the not-so-distant future. For many people, they already are. I've learned about the opinions of swedish initiatives without any knowledge of swedish just by using Google translate. Google Chrome and the free variant Chromium show directly the option to send something through Google translate if it detects that it's not in your language (although that wasn't working with Mongolian when I was there last year). I was in hotels where the staff pointed me to their PC with an instance of Yandex translate or Baidu translate where I should type in my questions in English (Yandex is something like the russian Google, Baidu is something like the chinese Google). Despite all the shortcomings of today's translation services, people use them to circumvent language barriers.

Young people in those countries are often learning English today, but it's a matter of fact that this will only very slowly translate into a real change. Lots of barriers exist. Many countries have their own language and another language that's used as the "international communication language" that's not English. For example, you'll probably get along pretty well in most post-soviet countries with Russian, no matter if the countries have their own native language or not. This also happens in single countries with more than one language. People have their native language and learn the countries language as their first foreign language.
Some people think their language is especially important and this stops the adoption of English (France is especially known for that). Some people have the strange idea that supporting English language knowledge is equivalent to supporting US politics and therefore oppose it.

Yes, one can try to learn more languages (I'm trying it with Mandarin myself and if I'll ever feel I can try a fourth language it'll probably be Russian), but if you look on the world scale, it's a loosing battle. To get along worldwide, you'd probably have to learn at least five languages. If you are fluent in English, Mandarin, Russian, Arabic and Spanish, you're probably quite good, but I doubt there are many people on this planet able to do that. If you're one of them, you have my deepest respect (please leave a comment if you are).

If you'd pick two completely random people of the world population, it's quite likely that they don't share a common language.

I see no reason in principle why technology can't solve that. We're probably far away from a StarTrek-alike universal translator and sadly evolution hasn't brought us the Babelfish yet, but I'm pretty confident that we will see rapid improvements in this area and that will change a lot. This may sound somewhat pathetic, but I think this could be a crucial issue in fixing some of the big problems of our world - hate, racism, war. It's just plain simple: If you have friends in China, you're less likely to think that "the chinese people are bad" (I'm using this example because I feel this thought is especially prevalent amongst the left-alternative people who would never admit any racist thoughts - but that's probably a topic for a blog entry on its own). If you have friends in Iran, you're less likely to support your country fighting a war against Iran. But having friends requires being able to communicate with them. Being able to have friends without the necessity of a common language is a fascinating thought to me.

Several years ago I bought a kind of very simple wardrobe from IKEA. It's called Bardu and is made out of steel rods and a plastic covering. It stands on wheels.

There are small plastic piece that connects the plastic rods with the wheels. And one of them broke a while back. I went to IKEA and asked for a replacement part. They told me that they don't ship parts for such old items - but they have an offering quite similar to the Bardu that I could buy. Sadly, the design has changed and the wheels are directly connected, so no compatible replacement part. The E-Mail service from IKEA told me the same: No replacement parts for old products.

At this point I could've complained about the fact that we live in a crazy world where someone suggests to you buying a new piece of furniture because a small plastic part of the old one is broken.

I posted a message in the RepRap-forum asking for help. If you don't know the RepRap: It's a 3D-printer, creating objects based on computer models out of simple plastic. The RepRap is an Open Source project built partly out of parts printed on other 3D printers. The idea is: Everyone can (with enough time and passion) built his own RepRap, all the documentation is available online.

I quickly got a response from someone from France who was willing to give it a try and re-create the needed plastic part on his 3D printer. Some message exchange later I sent him the broken and a non-broken part. Today, I got my RepRap-printed replacement part. It fits in perfectly. I'm seriously impressed.

Object on Thingiverse

Update (2012/12/11): I don't want to hide the fact that the whole issue turned out to be much trickier than thought. The original piece broke after a while. DeuxVis was so nice to experiment with likely more stable designs and sent me some more printed parts, but the first one already broke again. You can read the details in the RepRap-forum.

Continuing my blog series about ancient file formats, today I want to have a look at tracker and other non-streamed audio formats. On my last blog entry, someone pointed me to the fact that calling the C64 SID format a tracker format is quite inaccurate, so I mention the "other" formats for clarity. (And I apologize and ask for comments if I write anything wrong here, most of it is from a time before I was into those things)

Last time I talked about streamed audio formats. The basic idea with streamed audio formats is that you have a data representation of the audio waves. This is how most modern audio formats like mp3 or ogg work. This approach works well today, but it was not feasible on older computers, because it takes lots of memory. So old audio formats store some kind of meta information that a computer then uses to "compose" an audio track live.

For old audio formats, it seems the Audacious media player is the free software solution that you should look for.

C64 SID files

Probably the most famous ancient audio format is the C64 SID format. The Commodore 64 had a special chip - the SID chip - for audio which has a very distinctive kind of sound. The High Voltage SID Collection (HVSC) contains around 40.000 SID tunes (though only 60 MB in size).

To play SID files, the SID chip needs to be emulated, as they basically contain code for the chip itself. The sidplay project seems to be abandoned since a few years, but there is sidplayfp, a fork trying to keep it alive. It has its own command line player. Worth mentioning may be that the SID files you find today didn't exist on a C64, they were invented for SID players on systems other than the C64. The concept of "files" with headers as we know today wasn't really existing there.

The Audacious player contains a plugin for libsidplay. There is also a plugin for the gstreamer framework, but it depends on a very old sidplay version.

ATARI SAP files

For ATARI SAP files, the story is quite similar to the SID one, although they're less popular. They contain code for the POKEY chip, which was used on 8-Bit-Atari-Computers. The Atari SAP Music Archive (ASMA) collects such music. Audacious plays the format out of the box.

Tracker formats (MOD, XM, IT)

Tracker or module formats became popular with the Amiga, which used the MOD format. It contains a couple of instruments and information how they should be played. Later, popular tracker software introduced new formats like XM (Fast Tracker 2), IT (Impulse Tracker) or S3M (Scream Tracker). Tracker formats are still popular within the Demoscene.

There are two free software solutions to play module formats - MikMod and ModPlug. MikMod was dead for a couple of years, but just recently it seems development has revived. Both played every module format I ever happened to see, so I don't make a specific recommendation. Audacious has a plugin for ModPlug.

Adlib and other small tracker formats

I'll start with a personal story here. Once I tried getting into assembler programming and created so-called 4K-Intros. I started this under DOS in a time when this was already fading out. For sound, I found a nice piece of software called the RAD tracker, which came with some assembler source code to play their own mini audio format.

There existed a couple of similar mini-tracker-formats, mostly for the Adlib sound chip. Pretty much every PC sound card in that time had the functionality of an Adlib sound chip. melcom's ChipTune Archive contains a collection of Adlib files.

AdPlug is our free software solution here and - you guess it - Audacious has a plugin for it. However, AdPlug was not able to play all the files in melcom's archive.

For example, there is a collection of files created with the Edlib Tracker. While there seems to be a Winamp Plugin and a Player for MacOS X, I found no free software to play the files. However, there's still the backup solution of emulation. The DOSBox emulator does a fine job and you can run the original Edlib player in it.

Video Game Music

There are vast collections of sound archives extracted from old console video games. As systems like a GameBoy or a NES don't have "files", these formats have been created afterwards by enthusiasts to collect the music. There's a whole number of them, you can find samples of files like NSF (Nintendo Sound Format for NES games) or GBS (Gameboy Sound) (you'll find many samples behind the links). There's something special about these: They usually don't have one file for a track, but one file for a game. So e. g. an NSF file contains a whole buch of sound tracks.

The solution to play them is called Game Music Emu. And - you already guess it - Audacious has a plugin for that.

Conclusion

Free software plays pretty well with all kinds of strange audio formats, although the situation is a bit more scattered than with streaming music, where you pretty much have one solution for everything (ffmpeg). I stumbled at least across one case where no free software solution was available - but emulation helps out.
Audacious seems to play a central role in supporting all kinds of audio files. It's available for Linux and for Windows (although I haven't tested if it plays everything under Windows as well - comments welcome).

I've promised that I'll dig into some old file formats and check how well they can be accessed on today's systems with free software.

Today, I'll start with audio formats. To begin, in general there are two kinds of audio formats. Streamed audio formats start with a more or less raw audio stream, apply some encoding and sometimes (lossless or lossy) compression. There are also tracker audio formats. They have internal information on tone pitches and instruments. Most really old computer audio files are tracker formats (like the popular C64 SID format). This blog post will be about streamed audio formats and I'll save the tracked ones for a later one.

The file formats I've chosen are more or less random, the main criteria being that I once stepped over them and still remember that. There's a hughe collection of all kinds of media file samples on the mplayer server.

The single most important project regarding exotic audio or video formats is ffmpeg, a library that does despite its name much more than decoding mpeg. All major free software media players use ffmpeg.

The file formats I've investigated:

• Some of the very first files distributing music through the Internet I remember were real audio files (extension .ra or .rm) from the german punk band WIZO. Real audio has a whole bunch of variants, scanning through some of my old backups, most of them used either AC-3 or Real Audio 2.0 as their codec. Thanks to Waybach Machine, you can still find the WIZO downloads (Raum der Zeit - Techno is AC-3, the others are RealAudio 2.0).
• vqf (or TwinVQ) was once announced having better quality than MP3 and was discussed as its successor. However, it seems it is almost completely distinct today, I didn't find anything at all (except in the above mentioned sample collection) in vqf format for download.
• Monkey's audio, extension .ape, is a lossless audio codec, which is itself licensed under some kind of noncommercial-use-only license that doesn't qualify as free software. It's not really old, as it's still being developed, but I added it as another example of an uncommon format.
• Shorten (extension shn) is an old lossless audio format, which was often used by the etree project that collects recordings of concerts. Today, it is mostly deprecated by flac, but the old recordings are still available.
• voc: The popular dos floppy copying program vgacopy had sound before I had a soundcard - it used the pc speaker to play .voc files it had shipped. It's a format used by some Creative software for their SoundBlaster. It's a more-or-less raw audio format like wav.

ffmpeg is able to decode and play all of these audio codecs. But what I found out was that this doesn't necessarily mean every application using ffmpeg can do this as well. I've tested mplayer, xine, vlc, audacious and totem (based on gstreamer). Although there are quite many free software media players - both for audio and video - out there, this should cover pretty much everything. Most media players use xine, vlc or gstreamer indirectly.

	mplayer	xine	vlc	audacious	totem/gstreamer
ra AC3	Yes	No Yes	Yes	Yes	No
ra 2.0	Yes	No	No	Yes	No
vqf	Yes	No	Yes	Yes	No
ape	Yes	No	No	Yes	Yes
shn	Yes	Yes	No	Yes	Yes
voc	Yes	Scratchy	Scratchy	No	No

Shorten playback has some problems, seeking often does not work, but this seems to be a limitation of the format itself. If I found feature requests for those formats, I've linked them, I also opened a bunch of them myself.

Conclusion: ffmpeg does a really fine job in playing all the obscure audio streaming formats. However, not every player that's based on ffmpeg plays every format ffmpeg can play. mplayer is the only player that succeeds with everything, probably because mplayer's devleopment is very tightly related to ffmpeg's development.

Update: I forgot to mention libav. It is a fork of ffmpeg. However, there's not that much to say, as ffmpeg and libav are still quite similar in their codec support. audacious does not support libav yet, all other apps just produce the same result.

I recently had a discussion about the accessibility of today's computer content in the future. We started asking ourselves how well the support in current software is to read and use old legacy data formats - graphics, videos, text, layout documents, whatever may still be interesting today.

I remembered having such a discussion some years ago and back then, Works documents were mentioned by someone as a somewhat difficult format. Back then, libwps existed with some command line tools to convert to staroffice format (which could then be opened by openoffice) and experimental patches existed for openoffice itself. Seems at least here the situation has improved. The current version of libreoffice reads Works documents out of the box.

Free software projects play an important role in keeping old data accessible. Just to name two, ffmpeg does a great job in supporting a large number of old and exotic video formats. It's used by a bunch of popular video players like mplayer and vlc. For graphics files, there is imagemagick, which provides a conversion tool to up-to-date formats like PNG.

In some upcoming blog entries, I'll try to explore things, will look for old files and see if I am able to use them.

A call to my readers: Do you have any old stuff laying around that you'd find interesting to access today? Which file formats are difficult to access? Are you searching for tools to open / convert them? Do you have something old that might be worth publishing to others as well? Send me your stuff, I'm very interested.

Two days ago, the german Chaos Computer Club (CCC) published a sample that's supposedly a variant of a german state spy software (the so-called "Bundestrojaner").

You might wonder if your anti virus software is protecting you. The webpage Virus Total lets you upload suspicious files, scans them with 43 different anti virus applications and presents you the result. Currently, 24 of 43 scanners detect the Bundestrojaner.

The CCC provides some further information where they state that the file they released is not the original one - they had several samples that differed and to avoid detection of the potential source, they changed the differing parts to something completely else. You might wonder if your anti virus app also detects the "original" Bundestrojaner and not just the modified file the CCC released.

We can easily check this if we change the modified pieces again to something else. A modified variant lowered the detection rate to 14 of 43 - amongst them the popular McAffee software. Now, it's pretty useless to only detect the exact published sample of a malware if we know that the original malware is different.

Application	Version	Sig date	Modified sample	Original CCC sample
AhnLab-V3	2011.10.08.01	2011-Okt-09	Trojan/Win32.R2d2	Trojan/Win32.R2d2
AntiVir	7.11.15.175	2011-Okt-09	TR/GruenFink.1	TR/GruenFink.1
Antiy-AVL	2.0.3.7	2011-Okt-09	-	-
Avast	6.0.1289.0	2011-Okt-09	Win32:Trojan-gen	Win32:Trojan-gen
AVG	10.0.0.1190	2011-Okt-07	-	-
BitDefender	7.2	2011-Okt-10	Backdoor.R2D2.A	Backdoor.R2D2.A
ByteHero	1.0.0.1	2011-Sep-23	-	-
CAT-QuickHeal	11.00	2011-Okt-07	-	-
ClamAV	0.97.0.0	2011-Okt-10	Trojan.BTroj-1	Trojan.BTroj-1
Commtouch	5.3.2.6	2011-Okt-10	-	W32/R2D2.A
Comodo	10407	2011-Okt-10	-	Backdoor.Win32.R2D2.A
DrWeb	5.0.2.03300	2011-Okt-10	-	-
Emsisoft	5.1.0.11	2011-Okt-10	Trojan.Win32.Bundestrojaner!A2	Backdoor.Win32.R2D2!IK
eSafe	7.0.17.0	2011-Okt-06	-	-
eTrust-Vet	36.1.8605	2011-Okt-07	-	-
F-Prot	4.6.2.117	2011-Okt-09	-	W32/R2D2.A
F-Secure	9.0.16440.0	2011-Okt-10	Backdoor:W32/R2D2.A	Backdoor:W32/R2D2.A
Fortinet	4.3.370.0	2011-Okt-10	-	W32/R2D2.A!tr.bdr
GData	22	2011-Okt-10	Backdoor.R2D2.A	Backdoor.R2D2.A
Ikarus	T3.1.1.107.0	2011-Okt-10	-	Backdoor.Win32.R2D2
Jiangmin	13.0.900	2011-Okt-09	-	-
K7AntiVirus	91155258	2011-Okt-08	-	-
Kaspersky	9.0.0.837	2011-Okt-09	Backdoor.Win32.R2D2.a	Backdoor.Win32.R2D2.a
McAfee	5.400.0.1158	2011-Okt-10	-	Artemis!930712416770
McAfee-GW-Edition	2010.1D	2011-Okt-09	-	Artemis!930712416770
Microsoft	17702	2011-Okt-10	Backdoor:Win32/R2d2.A	Backdoor:Win32/R2d2.A
NOD32	6529	2011-Okt-10	Win32/R2D2.A	Win32/R2D2.A
Norman	6.7.2011	2011-Okt-09	-	-
nProtect	2011-10-10.01	2011-Okt-10	-	-
Panda	10.0.3.5	2011-Okt-09	-	Suspiciousfile
PCTools	8.0.0.5	2011-Okt-10	Backdoor.R2D2	Backdoor.R2D2
Prevx	3.0	2011-Okt-10	-	-
Rising	23.78.06.02	2011-Okt-09	-	-
Sophos	4.70.0	2011-Okt-10	Troj/BckR2D2-A	Troj/BckR2D2-A
SUPERAntiSpyware	4.40.0.1006	2011-Okt-08	-	-
Symantec	20111.2.0.82	2011-Okt-10	Backdoor.R2D2	Backdoor.R2D2
TheHacker	6.7.0.1.318	2011-Okt-09	-	-
TrendMicro	9.500.0.1008	2011-Okt-09	-	-
TrendMicro-HouseCall	9.500.0.1008	2011-Okt-10	-	BKDR_R2D2.A
VBA32	3.12.16.4	2011-Okt-07	-	-
VIPRE	10718	2011-Okt-10	-	Trojan.Win32.Generic!BT
ViRobot	2011.10.10.4710	2011-Okt-10	-	-
VirusBuster	14.1.3.0	2011-Okt-09	-	-

Scans done Monday morning around 8:00.

One of the few pieces of non-free software I always needed on my system is a rar unpacker. Despite that there are very good free alternatives for high-compression archivers like 7-zip or tar.xz, many people seem to like relying on a proprietary format like rar and it's in widespread use.

Years ago, someone came up with a GPLed rar unpacker, but sadly, that was never updated to support the rar version 3 format. Its development is stalled.

For that reason, some time back I suggested to the Free Software Foundation to add a free rar unpacking tool to their list of high priority projects - they did so. Happily I recently read that they've removed it. There's The Unarchiver now, based on an old amiga library. It supports a whole bunch of formats - including rar v3. It's mainly a MacOS application, but it also provides a command line tool that can be compiled in Linux.

It needs objective C, the gnustep-base libraries and it took me some time to get it to compile properly. For the Gentoo-users: I already committed an ebuild, just run "emerge unar".emerge TheUnarchiver

Update: Changed ebuild name to unar, as that's the name upstream uses for the command line version now.

Michael S. Hart, the founder of Project Gutenberg, died some days ago.

Project Gutenberg, if you don't know, is a webpage collecting electronic books online. It was founded in 1971 (yes, long before the Internet as we know it today existed), when Hart typed the Declaration of Independency on a Xerox mainframe. Hart can be seen as the inventor of electronic books - 40 years ago.

We're still waiting for ebooks to get into mainstream. Currently, ebook reading devices are available, but their usage is not widespread yet. But I'm almost certain that ebooks will become very important within the next years. Hart had that opinion 40 years ago.

Today, Project Gutenberg has about 36.000 books. Most of them are public domain, because their copyright expired. There are other similar projects today: Wikisource is a sister project of Wikipedia and archive.org has a lot of scanned books, including most of the public domain books digitalized by Google.

Some mission statements for Project Gutenberg from Michael S. Hart (taken from Wikipedia) I find that sum up things very well:
"Encourage the Creation and Distribution of eBooks"
"Help Break Down the Bars of Ignorance and Illiteracy"
"Give As Many eBooks to As Many People As Possible"

Some days ago it was reported that Microsoft declared it considers Linux on the desktop no longer a threat for its business. Now I usually wouldn't care that much what Microsoft is saying, but in this case, I think, they're very right – and thererfore I wonder why this hasn't raised any discussions in the free software community (at least I haven't seen one – if it has and I missed it, please provide links in the comments). So I'd like to make a start.

A few years ago, I can remember that I was pretty optimistic about a Linux-based Desktop (and I think many shared my views). It seemed with advantages like being able to provide a large number of high quality applications for free and having proven to be much more resilient against security threats it was just a matter of time. I had the impression that development was often going into the right direction, just to name one example freedesktop.org was just starting to try to unify the different Linux desktop environments and make standards so KDE applications work better under GNOME and vice versa.

Today, my impression is that everything is in a pretty sad state. Don't get me wrong: Free software plays an important role on Desktops – and that's really good. Major web browsers are based on free software, applications like VLC are very successful. But the basis – the operating system – is usually a non-free one.

I recently was looking for netbooks. Some years ago, Asus came out with the Eee PC, a small and cheap laptop which ran Linux by default – one year later they provided a version with Windows as an alternative. Today, you won't find a single Netbook with Linux as the default OS. I read more often than not in recent years that public authorities trying to get along with Linux have failed.

I think I made my point; the Linux Desktop is in a sad state – I'd like to discuss why this is the case and how we (the free software community) can change it. I won't claim that I have the definite answer for the cause. I think it's a mix of things, I'd like to start with some points:

• Some people seem to see Desktop environments more as a playground for creative ideas than something other people want to use on a daily basis in a stable way. This is pretty much true for KDE 4 – the KDE team abandoned a well-working Desktop environment KDE 3.5 for something that isn't stable even today and suffers from a lot of regressions. They permanently invent new things like Akonadi and make them mandatory even for people who don't care about them – I seriously don't have an idea what it does, except throwing strange error messages at me. I switched to GNOME, but what I heard about GNOME 3 doesn't make me feel that it's much better there (I haven't tested it yet and I hope that, unlike the KDE-team, GNOME learns from that and supports 2.x until version 3 is in a state working equally well). I think Ubuntu's playing with the Unity Desktop go in the same direction: We found something cool, we'll use it, we don't care that we'll piss of a bunch of our users. In contrast to that, I have the impression that what I named above – the idea that we can integrate different desktop environments better by standards – isn't seen as important as it used to be. (I know this part may provoke flames, I hope this won't hide the other points I made)
• The driver problem. I still encounter it to be one of the biggest obstacles and it hasn't changed a bit for years. You just can't buy a piece of hardware and use it. It usually is “somehow possible”, but the default is that it requires a lot of extra geeky work that the average user will never manage. I think there's no easy solution to that, as it would require cooperation from hardware vendors (and with diminishing importance of the Linux Desktop this is likely getting harder). But a lot of things are also self-made. In 2006, Eric Raymond wrote an essay how crappy CUPS is – I think it hasn't improved since then. How often have I read Ubuntu bug reports that go like this: “My printer worked in version [last version], but it doesn't work in [current version]” - “Me too.” - “Me too.” - “Me too” - no reply from any developer. One point that this shares with the one above is the caring about regressions, which I think should be a top priority, but obviously, many in the free software community don't seem to think so. (if you don't know the word: something is called a regression if something worked in an older version of a software, but no longer works in the current version)
• The market around us has changed. Back then, we were faced with a “Windows or nothing” situation we wanted to change to a “Windows or Linux” situation. Today, we're faced with “Windows or MacOS X”. Sure, MacOS existed back then, but it only got a relevant market share in recent years (and many current or former free software developers use MacOS X now). Competition makes products better, so Windows today is not Windows back then. Our competitors just got better.
• The desktop is loosing share. This is a point often made, with mobile phones, tablets, gaming consoles and other devices taking over tasks that were done with desktop computers in the past. This is certainly true for some degree, but I think it's also often overestimated. Desktop computers still play an important role and I'm sure they will continue to do so for a long time. The discussion how free software performs on other devices (and how free Android is) is an interesting one, too, but I won't go into it for now, as I want to talk about the Desktop here.

Okay, I've started the discussion, I'd like others to join. Please remember: It's not my goal to flame or to blame anyone – my goal is to discuss how we can make the Linux desktop successful again.

OpenLeaks is a planned platform like WikiLeaks, founded by ex-Wikileaks member Daniel Domscheit-Berg. It's been announced a while back and a beta is currently presented in cooperation with the newspaper taz during the Chaos Communication Camp (where I am right now).

I had a short look and found some things noteworthy:
The page is SSL-only, any connection attempt with http will be forwarded to https. When I opened the page in firefox, I got a message that the certificate is not valid. That's obviously bad, although most people probably won't see this message.

What is wrong here is that an intermediate certificate is missing - we have a so-called transvalid certificate (the term "transvalid" has been used for it by the EFF SSL Observatory project). Firefox includes the root certificate from Go Daddy, but the certificate is signed by another certificate which itself is signed by the root certificate. To make this work, one has to ship the so-called intermediate certificate when opening an SSL connection.

The reason why most people won't see this warning and why it probably went unnoticed is that browsers remember intermediate certificates. If someone ever was on a webpage which uses the Go Daddy intermediate certificate, he won't see this warning. I saw it because I usually don't use Firefox and it had a rather fresh configuration.

There was another thing that bothered me: On top of the page, there's a line "Before submitting anything verify that the fingerprints of the SSL certificate match!" followed by a SHA-1 certificate fingerprint. Beside the fact that it's english on a german page, this is a rather ridiculous suggestion. Checking a fingerprint of an SSL connection against one you got through exactly that SSL connection is bogus. Checking a certificate fingerprint doesn't make any sense if you got it through a connection that was secured with that certificate. If checking a fingerprint should make sense, it has to come through a different channel. Beside that, nowhere is explained how a user should do that and what a fingerprint is at all. I doubt that this is of any help for the targetted audience by a whistleblower platform - it will probably only confuse people.

Both issues give me the impression that the people who designed OpenLeaks don't really know how SSL works - and that's not a good sign.

I've written in the past about the EFF SSL Observatory. It's a great project that has scanned the whole IP space for SSL-certificates used in HTTPS. They provide a database with meta information and their project found a couple of issues where CAs have issued certificates with weak security settings and violation of their own policies. CAcert is a project which tries to be the "better SSL authority" - it issues certificates for free, based on a web-of-trust community. The CAcert root certificate is not part of any major web browser. The EFF has mainly analyzed the browser-accepted CAs - but they provide the data, so I could do it myself.

I did some checks on the all_certs table selecting the certificates from cacert. I found out that there were 143 valid certificates with 512 bit. That is completely insecure and breakable by a home computer today. I also found that the majority of certificates still has 1024 bit, which by today's standards should be considered harmful - there have been no public breaks yet, but it's expected that it's possible to build an RSA-1024 cracker for an attacker with enough money.

I did the following query on the database:
SELECT RSA_Modulus_Bits, count(*) FROM all_certs WHERE `Validity:Not After datetime` > '2010-03-08' AND ( `Issuer` like '%CAcert.org%' OR `Issuer` like '%cacert.org') GROUP BY `RSA_Modulus_Bits` ORDER BY count(*);

+------------------+----------+
| RSA_Modulus_Bits | count(*) |
+------------------+----------+
[...]
| 512              |      143 |
| 4096             |      632 |
| 2048             |     3716 |
| 1024             |     5790 |
+------------------+----------+

Now, what further checks can we do? I checked for the RSA exponent. I found two certificates in the database with exponent 3. RSA with low exponent is also considered insecure, although one has to state that this is not a serious issue. RSA with low exponents is not insecure by itself, but it can create vulnerabilities in combination with other issues (if you're interested in details, read my diploma thesis).

I have not checked the CAcert database for the Debian SSL vulnerability, as that would've been non-trivial. There were scripts shipped with the SSL Observatory data, but I found them not easy to use, so I skipped that part.

My suggestions to cacert were to revoke all certificates with serious issues (like the 512 bit certificates). Also, I suggested that new certificates with insecure settings like RSA below 2048 bits or a low exponent should not be allowed. CAcert did most of this. By now, all 512 bit certificates should be revoked and it is impossible to create new ones below 1024 bit or with low exponents. It is however still possible to create 1024 bit certificates, which is due to a limitation in the client certificate creation script for the Internet Explorer. They say they're working on this and plan to prevent 1024 bit certificates in the future. They also told me that they've checked for the Debian SSL bug.

I've reported the issue on the 11th March and got a reply on the same day - that's pretty okay, one slight thing still: There was no security contact with a PGP key listed on the webpage (but I got a PGP-encrypted contact once I asked for it). That's not good, I expect especially from a security project that I can contact them for security issues with encrypted mail. One can also argue if four months is a bit long to fix such an issue, but as it was far away from being trivial, this can be apologized.

I'd say that I'm quite satisfied with the reactions of CAcert. I always got fast replies to questions I had and the issues were resolved in a proper way. I have other points of criticism on the security of CAcert, the issue that bothers me most is that they still use SHA-1 and refuse to switch to a more secure hashing algorithm like SHA-512, although all major browsers have support for this since a long time.

I want to encourage others to do further tests on CAcert. I'd like to see CAcert being an authority that does better than the commercial ones. The database from the observatory is a treasure and should be used by projects like CAcert to improve their security.

When thinking about China, probably many people associate this with censorship.

On my trip, I had the chance to see the infamous great firewall from the inside. I haven't done any deeper analysis, but I'll share some thinkgs I've observed. A couple of famous sites (for example Twitter, Flickr) are blocked. Contrary to what many people may believe, webpages that are often associated with Warez (Rapidshare, Pirate Bay) were also blocked. The situation with Wikipedia was mixed. Most of the time, I could read the texts on Wikipedia, but access to the image servers was blocked. At the end of our trip, I couldn't access Wikipedia any more.

I encountered no blocks on less famous sites, although I regularly surf sites that could be labelled politically controversial. Though this probably doesn't tell much, except that the chinese authorities are not very interested in blocking european websites.

Interesting may be that the blocking works on an IP level. DNS resolution of blocked sites still works, but you cannot ping the IPs. I haven't extensively tried to circumvent the censorship, as I had no pressing need for it. The only thing I tried was an SSH tunnel, but that usually wasn't possible as the connection never was fast and reliable enough for a stable SSH session.

Most Hotels and Hostels provide Internet access - but most of them by cable. Usually, in other countries today this is done via wireless lan. My theory on that is that a cable-based Internet access makes it easier to log activity associated to a specific person (you always have to show your passport when you check into a Hotel). But still, we had anonymous Internet access (both wireless and cable) at a few places.

Another thing I'd like to mention is what the (non-technical) censorship did with me. I knew that in China people cannot just write a blog, they need some kind of license for it. I was very unsure what this means for me as a forein traveller. I came to the conclusion that I likely won't get any trouble if I just write about my trip without touching any controversial topics. Although I hadn't planned to write anything, this was always in my mind and probably influenced my writings. There was one time where I self-censored myself. In the entry about Hong Kong, I originally had this part, which I removed before publishing:

Most notably it is a place where free speech is possible to a much higher degree than in mainland China. This makes it a very important place for political discussion about China in general. We saw chinese dissident groups that had their information tables and spread leaflets around the Kowloon harbour.

Not much and I luckily have the opportunity to publish it now.

Recently, a new X-Men movie made it to the cinemas. I quite liked the first three X-Men movies. I wasn't very impressed by the prequel-movie Wolverine, which tells the story of one of the characters.

Now there's X-Men first class. Again a prequel, but from a different perspective. Wolverine only has a very short cameo appearance, this time the movie telling the story of many other X-Men characters. Prequels are difficult. They must tell a story that already has some fixed points and a fixed outcome that's already known my most viewers. The new X-Men film suffers exactly from that problems.

The movie does a very brave task - it tries not only to tell a prequel story to the other movies, it also involves a whole bunch of historic events. From the nazi concentration camps to nuclear weapons and the cuba crisis. This sounds like too much for a science fiction movie, but surprisingly it handles this part not so bad. It even motivated me to improve my history knowledge and read something about Argentinia's role in hiding nazis after the second world war (some german information here). It seems the movie is pretty close to real history in that respect.

But there's another problem in this movie. I found the main plot just doesn't fit. And that's due to the fact its a prequel where the outcome is already decided:

(please stop reading here if you haven't seen the movie and still want to see it - spoilers ahead)

The development of the charakter Erik Lensherr alias Magneto. As the victim of some cruel experiments by a mad Nazi scientist, his life is one of anger and revenge. But his opponent - and all viewers of the other X-Men movies already know that - has a role which is exactly what Magneto's role will be later. So in the end, he kills him - and then just replaces him. All allies of his biggest enemy just become allies of him. That just doesn't make sense.

(now you can read again)

All in all, it's still a pretty good movie. If you like such movies, it has its moments. But the in my opinion illogical story destroys a lot.

Final word: We had two X-Men prequels now. But X-Men 3 ended with a perfect cliffhanger. It should be resolved at some point - I'd like to see X-Men 4.

I have promised to write something about the route we had planned to take for the way back from China to Europe. We had several variants in mind, I'll list them all. All of them, however, have in common that they start in Ürümqi (乌鲁木齐, ئۈرۈمچی) - and as I already wrote, the train line to Ürümqi seems to be a bottleneck - it was booked out for an unknown amount of time.

Ürümqi is a town in the Uyghur province of China in the north-west. It is north of the Taklamakan desert. China's population is not evenly spread through the country. Most of the population lives in the eastern part. The west is sparsely populated and Ürümqi is one of the very few big cities in the west.

From Ürümqi, there are several options to go to Kazakhstan - there exist trains and busses both to Astana (Астана, أستانا), the current capital of Kazakhstan, and Almaty (Алматы, الماتى), the former capital.

Variant a: Twice through russia (our preferred option).

From Astana, there is a train directly to Kiev (Київ) in Ukraine. The train goes twice through russia. Once it scratches it before Oral / Uralsk (Орал). I think it doesn't even stop there. The other time it goes through the Caucasus region.

It should've been possible to buy the train ticket in Astana and then get a transit express visa in the russian consulate. I read some reports suggesting that EU people were able to do this. However, I was not entirely sure about that: Usually, a russian transit visa only allows to pass the country a single time. I don't know if crossing the country twice would've posed any problems.

Astana to Kiev is quite long - stopping was a problem, because you can only get the transit visa once you have the ticket for the whole journey. So our plan was to take the train just to Kharkiv (Харків) in the east of Ukraine. This would've limited the train trip to a bit more than two days. Still a lot, but acceptable for me.

Variant b: Once through russia.

Oral/Uralsk (Орал) in western Kazakhstan has its own russian embassy. As stated above, the train from Astana to Oral already crosses russia, but there's a way round: One can first take the train to Atyrau (Атырау) and then to Oral. This way, you don't leave Kazakhstan. The advantage: Lots of options to make stops, no overly long train trips.

The problem with this variant was that I had almost no information about the consulate in Oral: I haven't read a single report online that any EU citizen tried or successfully applied for a transit visa there. I only found some people asking that question, but without answers. So it was quite unsure if this would work.

Variant c: Avoiding russia altogether (option we originally intended to take).

It is also possible to avoid passing russia altogether. One can go by train to Atyrau (like in variant b), but then take a train on to Aktau (Ақтау) at the caspian see. From Aktau, there is a ferry service to Baku in Azerbaijan.

Now, this "ferry" has its own problems: It has no regular schedule. In fact, from what I read its no real ferry at all, but a cargo ship. It starts when there's enough cargo. So you have to get there and ask every day if there will be a ship today. Waiting times rank between some days and two weeks. I had liked to take that option, because I like travelling by ship and I thought that sounded like an interesting experience.

From Azerbaijan, one could take a train to Tbilis in Georgia and continue by bus to Istanbul in Turkey. From there, there is a train to Austria (the orient express sadly doesn't exist any more).

We had our visas ready for Kazakhstan and Azerbaijan. Georgia and Turkey are visa free for EU citizens.

If you look at a map, you may notice that there's another option: Going from Kazakhstan to Turkmenistan and Iran. However, that would've imposed getting two more visa plus the feeling that travelling through Iran might be a risk. So I haven't really investigated that option very much.

For our trip, we needed a couple of visa. I haven't applied myself for a visa any time before, so this was quite new to me. This was the most troublesome part of our travel preparations.

What I learned about getting visa:
- Every country has different rules for visa.
- You cannot apply for several visa at once - they take your passport. That means you have to add all the waiting times and cannot apply for more than one at once (this may seem trivial if you know the procedure, but I didn't).
- The information on the consulates webpages is often incomplete or inaccurate. (For example, if you have a 30 day visa: Does that mean 30 days starting from your entry to the country? Or 30 days starting from a fixed date you have to know in advance? Pretty relevant if you plan your trip.)
- If you phone a consulate, they won't answer. If you email a consulate, they won't answer.
- You cannot expect that anyone in the consulate is able to speak to you in a language you understand.
- You cannot expect that information you got from people in the consulate is correct.
- Usually, the best way to get information is searching the internet for people who have done the same thing before. There are specialized companies that arrange your visa, but the information you get from them is also often inaccurate.

In the end, we applied for 6 different visa (Russia, Mongolia, Belarus, China, Kazakhstan, Azerbaijan), although we didn't use them all in the end (see previous blog entry).
The most difficult part was the russian one. That was, in the end, the reason we couldn't make the trip the way we wanted to (taking the transsiberian train for both directions with stops). They have a kind of bizzare regulation regarding invitations: You need an invitation to apply for a russian tourist visa. This has evolved a market for agencies that arrange invitations. That means you pay them that they do a fake booking in a hotel you will never see in reality and get an invitation from them.
Another anecdote: When asking for the "two-way"-problem in the embassy, they gave us a contact to a travel agency that will help us. This travel agency suggested we could get two passports and thus apply for two visa - that would've been illegal according to russian law. I had no intention in seeing a russian jail from inside, so I refused to choose that option.

You see, it's a pretty complex issue. But there's one thing one should mention, too: It's not the russian (or other countries) authorities that are to blame here. Russia is very willing to relax its visa rules. They even suggested several times to abbadon the visa requirement for EU citizens at all. They just have one requirement: The regulation should be relaxed for their citizens, too. Everything I've heared suggests that russians trying to get a visa for Germany and other EU countries face more difficulties than the other way round. It's the EU that is blocking here.

If you want visa regulations to be relaxed, you'd better not only blame other countries regulations. You should also ask how regulation is the other way round. Looking at the current political debate in the EU, I don't have much hope that the situation will improve soon.

(the pictures are from Wikimedia Commons here (Russia) and here (Belarus) and are public domain)