Gentoo - 2

One of the few pieces of non-free software I always needed on my system is a rar unpacker. Despite that there are very good free alternatives for high-compression archivers like 7-zip or tar.xz, many people seem to like relying on a proprietary format like rar and it's in widespread use.

Years ago, someone came up with a GPLed rar unpacker, but sadly, that was never updated to support the rar version 3 format. Its development is stalled.

For that reason, some time back I suggested to the Free Software Foundation to add a free rar unpacking tool to their list of high priority projects - they did so. Happily I recently read that they've removed it. There's The Unarchiver now, based on an old amiga library. It supports a whole bunch of formats - including rar v3. It's mainly a MacOS application, but it also provides a command line tool that can be compiled in Linux.

It needs objective C, the gnustep-base libraries and it took me some time to get it to compile properly. For the Gentoo-users: I already committed an ebuild, just run "emerge unar".emerge TheUnarchiver

Update: Changed ebuild name to unar, as that's the name upstream uses for the command line version now.

Some days ago it was reported that Microsoft declared it considers Linux on the desktop no longer a threat for its business. Now I usually wouldn't care that much what Microsoft is saying, but in this case, I think, they're very right – and thererfore I wonder why this hasn't raised any discussions in the free software community (at least I haven't seen one – if it has and I missed it, please provide links in the comments). So I'd like to make a start.

A few years ago, I can remember that I was pretty optimistic about a Linux-based Desktop (and I think many shared my views). It seemed with advantages like being able to provide a large number of high quality applications for free and having proven to be much more resilient against security threats it was just a matter of time. I had the impression that development was often going into the right direction, just to name one example freedesktop.org was just starting to try to unify the different Linux desktop environments and make standards so KDE applications work better under GNOME and vice versa.

Today, my impression is that everything is in a pretty sad state. Don't get me wrong: Free software plays an important role on Desktops – and that's really good. Major web browsers are based on free software, applications like VLC are very successful. But the basis – the operating system – is usually a non-free one.

I recently was looking for netbooks. Some years ago, Asus came out with the Eee PC, a small and cheap laptop which ran Linux by default – one year later they provided a version with Windows as an alternative. Today, you won't find a single Netbook with Linux as the default OS. I read more often than not in recent years that public authorities trying to get along with Linux have failed.

I think I made my point; the Linux Desktop is in a sad state – I'd like to discuss why this is the case and how we (the free software community) can change it. I won't claim that I have the definite answer for the cause. I think it's a mix of things, I'd like to start with some points:

• Some people seem to see Desktop environments more as a playground for creative ideas than something other people want to use on a daily basis in a stable way. This is pretty much true for KDE 4 – the KDE team abandoned a well-working Desktop environment KDE 3.5 for something that isn't stable even today and suffers from a lot of regressions. They permanently invent new things like Akonadi and make them mandatory even for people who don't care about them – I seriously don't have an idea what it does, except throwing strange error messages at me. I switched to GNOME, but what I heard about GNOME 3 doesn't make me feel that it's much better there (I haven't tested it yet and I hope that, unlike the KDE-team, GNOME learns from that and supports 2.x until version 3 is in a state working equally well). I think Ubuntu's playing with the Unity Desktop go in the same direction: We found something cool, we'll use it, we don't care that we'll piss of a bunch of our users. In contrast to that, I have the impression that what I named above – the idea that we can integrate different desktop environments better by standards – isn't seen as important as it used to be. (I know this part may provoke flames, I hope this won't hide the other points I made)
• The driver problem. I still encounter it to be one of the biggest obstacles and it hasn't changed a bit for years. You just can't buy a piece of hardware and use it. It usually is “somehow possible”, but the default is that it requires a lot of extra geeky work that the average user will never manage. I think there's no easy solution to that, as it would require cooperation from hardware vendors (and with diminishing importance of the Linux Desktop this is likely getting harder). But a lot of things are also self-made. In 2006, Eric Raymond wrote an essay how crappy CUPS is – I think it hasn't improved since then. How often have I read Ubuntu bug reports that go like this: “My printer worked in version [last version], but it doesn't work in [current version]” - “Me too.” - “Me too.” - “Me too” - no reply from any developer. One point that this shares with the one above is the caring about regressions, which I think should be a top priority, but obviously, many in the free software community don't seem to think so. (if you don't know the word: something is called a regression if something worked in an older version of a software, but no longer works in the current version)
• The market around us has changed. Back then, we were faced with a “Windows or nothing” situation we wanted to change to a “Windows or Linux” situation. Today, we're faced with “Windows or MacOS X”. Sure, MacOS existed back then, but it only got a relevant market share in recent years (and many current or former free software developers use MacOS X now). Competition makes products better, so Windows today is not Windows back then. Our competitors just got better.
• The desktop is loosing share. This is a point often made, with mobile phones, tablets, gaming consoles and other devices taking over tasks that were done with desktop computers in the past. This is certainly true for some degree, but I think it's also often overestimated. Desktop computers still play an important role and I'm sure they will continue to do so for a long time. The discussion how free software performs on other devices (and how free Android is) is an interesting one, too, but I won't go into it for now, as I want to talk about the Desktop here.

Okay, I've started the discussion, I'd like others to join. Please remember: It's not my goal to flame or to blame anyone – my goal is to discuss how we can make the Linux desktop successful again.

For our trip, we needed a couple of visa. I haven't applied myself for a visa any time before, so this was quite new to me. This was the most troublesome part of our travel preparations.

What I learned about getting visa:
- Every country has different rules for visa.
- You cannot apply for several visa at once - they take your passport. That means you have to add all the waiting times and cannot apply for more than one at once (this may seem trivial if you know the procedure, but I didn't).
- The information on the consulates webpages is often incomplete or inaccurate. (For example, if you have a 30 day visa: Does that mean 30 days starting from your entry to the country? Or 30 days starting from a fixed date you have to know in advance? Pretty relevant if you plan your trip.)
- If you phone a consulate, they won't answer. If you email a consulate, they won't answer.
- You cannot expect that anyone in the consulate is able to speak to you in a language you understand.
- You cannot expect that information you got from people in the consulate is correct.
- Usually, the best way to get information is searching the internet for people who have done the same thing before. There are specialized companies that arrange your visa, but the information you get from them is also often inaccurate.

In the end, we applied for 6 different visa (Russia, Mongolia, Belarus, China, Kazakhstan, Azerbaijan), although we didn't use them all in the end (see previous blog entry).
The most difficult part was the russian one. That was, in the end, the reason we couldn't make the trip the way we wanted to (taking the transsiberian train for both directions with stops). They have a kind of bizzare regulation regarding invitations: You need an invitation to apply for a russian tourist visa. This has evolved a market for agencies that arrange invitations. That means you pay them that they do a fake booking in a hotel you will never see in reality and get an invitation from them.
Another anecdote: When asking for the "two-way"-problem in the embassy, they gave us a contact to a travel agency that will help us. This travel agency suggested we could get two passports and thus apply for two visa - that would've been illegal according to russian law. I had no intention in seeing a russian jail from inside, so I refused to choose that option.

You see, it's a pretty complex issue. But there's one thing one should mention, too: It's not the russian (or other countries) authorities that are to blame here. Russia is very willing to relax its visa rules. They even suggested several times to abbadon the visa requirement for EU citizens at all. They just have one requirement: The regulation should be relaxed for their citizens, too. Everything I've heared suggests that russians trying to get a visa for Germany and other EU countries face more difficulties than the other way round. It's the EU that is blocking here.

If you want visa regulations to be relaxed, you'd better not only blame other countries regulations. You should also ask how regulation is the other way round. Looking at the current political debate in the EU, I don't have much hope that the situation will improve soon.

(the pictures are from Wikimedia Commons here (Russia) and here (Belarus) and are public domain)

Having used my PGP key 3DBD3B20 for almost eight years, it's finally time for a new one: 4F9F43A9. The old primary key was a 1024 bit DSA key, which had two drawbacks:
1. 1024 bit keys for DLP or factoring based algorithms are considered insecure.
2. It's impossible to set the used hash algorithm to anything beyond SHA-1.

My new key has 4096 bits key size (2048 bit is the default of GnuPG since 2.0.13 and should be fairly enough, but I wanted some extra security) and the default hash algorithm preference is SHA-256. I had to make a couple of decisions for my name in the key:
1. I'm usually called Hanno, but my real/official name is Johannes.
2. My surname has a special character (ö) in it, which can be represented as oe.

In my previous keys, I've mixed this. I decided against this for the new key, because both my inofficial prename Hanno and my umlaut-converted surname Boeck are part of my mail adress, so people should still be able to find my key if they're searching for that.

Another decision was the time I wanted my key to be valid. I've decided to give it an expiration date, but a fairly long one: 10 years from now.

I've signed my new key with my old key, so if you've signed my old one, you should be able to verify the new one. I leave it up to you if you decide to sign my new key or if you want to re-new the signing procedure. I'll start from scratch and won't sign any keys I've signed with the old key automatically with the new one. If you want to key-sign with me, you may find me on the 27C3 within the next days.

My old key will be valid for a while, at some time in the future I'll probably revoke it.

Update: I just found out that having a key without SHA-1 is trickier than I thought. The self-signatures were still SHA-1. I could re-do the self-signatures and revoke the old ones, but that'd clutter the key with a lot of useless cruft and as the new key wasn't around long and didn't get any signatures I couldn't get easily again, I decided to start over again: The new key is BBB51E42 and the other one will be revoked.
I'll write another blog entry to document how you can create your own SHA-256 only key.

Update: I got some nice hints in the comments. cpufreqd also includes this functionality and is probably the much more advanced solution. Also, I got a hint to linux-PHC, which allows undervolting a CPU and thus also saves energy.

I recently quite often had the problem that my system suddenly was shutting down. The reason was that when my processor got beyond 100 °C, my kernel decided that it's better to do so. I don't really know what caused this, but anyway, I needed a solution.

So i hacked together overheatd. A very effective way of cooling down a CPU is reducing its speed / frequency. Pretty much any modern CPU can do that and on Linux this can be controlled via the cpufreq interface. I wrote a little daemon that simply checks every 5 seconds (adjustable) if the temperature is over a certain treshold (90 °C default, also adjustable) and if yes, it sets cpufreq to the powersave governor (which means lowest speed possible). When the temperature is below or at 90 °C again, it's set back to the (default) ondemand governor. It also works for more than one CPU (I have a dual core), though it's very likely that it has bugs as soon as one goes beyond 10 CPUs - but I have no way to test this. Feel free to report bugs.

This could be made more sophisticated (not going to the lowest frequency but step by step to lower frequencies), but it does its job quite well for now. It might be a good idea to support something like this directly in the kernel (I wonder why that isn't the case already - it's pretty obvious), but that would probably involve a skilled kernel-hacker.

After reading a lot about interesting stuff happening at this years FOSDEM, I decided very short term to go there. The FOSDEM in Brussels is probably one of the biggest (if not the biggest at all) meetings of free software developers. Unlike similar events (like several Linuxtag-events in Germany), it's focus is mainly on developers, so the talks are more high level.

My impressions from FOSDEM so far: There are much more people compared when I was here a few years ago, so it seems the number of free software developers is inceasing (which is great). The interest focus seems to be to extend free software to other areas. Embedded devices, the BIOS, open hardware (lot's of interest in 3D-printers).

Yesterday morning, there was a quite interesting talk by Richard Clayton about Phishing, Scam etc. with lots of statistics and info about the supposed business models behind it. Afterwards I had a nice chat with some developers from OpenInkpot. There was a big interest in the Coreboot-talk, so I (and many others) just didn't get in because it was full.

Later Gentoo-developer Petteri Räty gave a talk about "How to be a good upstream" and I'd suggest every free software developer to have a look on that (I'll put the link here later).

I've just attended a rather interesting talk about 3D-printers like RepRap and MakerBot.

At least since 2005 it's well known that the cryptographic hash function SHA1 is seriously flawed and it's only a matter of time until it will be broken. However, it's still widely used and it can be expected that it'll be used long enough to allow real world attacks (as it happened with MD5 before). The NIST (the US National Institute of Standards and Technology) suggests not to use SHA1 after 2010, the german BSI (Bundesamt für Sicherheit in der Informationstechnik) says they should've been fadet out by the end of 2009.

The probably most widely used encryption protocol is SSL. It is a protocol that can operate on top of many other internet protocols and is for example widely used for banking accounts.

As SSL is a pretty complex protocol, it needs hash functions at various places, here I'm just looking at one of them. The signatures created by the certificate authorities. Every SSL certificate is signed by a CA, even if you generate SSL certificates yourself, they are self-signed, meaning that the certificate itself is it's own CA. From what I know, despite the suggestions mentioned above no big CA will give you certificates signed with anything better than SHA1. You can check this with:
openssl x509 -text -in [your ssl certificate]
Look for "Signature Algorithm". It'll most likely say sha1WithRSAEncryption. If your CA is good, it'll show sha256WithRSAEncryption. If your CA is really bad, it may show md5WithRSAEncryption.

When asking for SHA256 support, you often get the answer that the software still has problems, it's not ready yet. When asking for more information I never got answers. So I tried it myself. On an up-to-date apache webserver with mod_ssl, it was no problem to install a SHA256 signed certificate based on a SHA256 signed test CA. All browsers I've tried (Firefox 3.6, Konqueror 4.3.5, Opera 10.10, IE8 and even IE6) had no problem with it. You can check it out at https://sha2.hboeck.de/. You will get a certificate warning (obviously, as it's signed by my own test CA), but you'll be able to view the page. If you want to test it without warnings, you can also import the CA certificate.

I'd be interested if this causes any problems (on server or on client side), so please leave a comment if you are aware of any incompatibilities.

Update: By request in the comments, I've also created a SHA512 testcase.

Update 2: StartSSL wrote me that they tried providing SHA256-certificates about a year ago and had too many problems - it wasn't very specific but they mentioned that earlier Windows XP and Windows 2003 Server versions may have problems.

Today I faced an interesting Linux problem that made me learn a couple of things I'd like to share. At first, we found an issue on a Thinkpad X301 notebook that was fixed in a newer BIOS version. So we wanted to do a BIOS update. Lenovo provides BIOS updates either for Windows or as bootable ISO CD-images. But the device had no CD-drive and only Linux installed. First we tried unetbootin, a tool to create bootable USB sticks out of ISO-Images. That didn't work.
So I had a deeper look at the ISO. What puzzled me was that when mounting it as a loopback device, there were no files on it. After some research I learned that there are different ways to create bootable CDs and one of them is the El Torito extension. It places an image of a harddisk on the CD, when booting, the image is loaded into memory and an OS can be executed (this probably only works for quite simple OSes like DOS, the Lenovo BIOS Upgrade disk is based on PC-DOS). There's a small PERL-script called geteltorito that is able to extract such images from ISO files.
It's possible to boot such harddisk images with grub and memdisk (part of syslinux). Install syslinux, place the file memdisk into /boot (found in /usr/lib/syslinux/ or /usr/share/syslinux/) and add something like this to your grub config:

Or for grub2:

Now you can select bios update in your boot menu and it should boot the BIOS upgrade utility.

(Note that this does not work for all Lenovo BIOS updates, only for those using an El Torito harddisk image - you can mount your iso with mount -o loop [path_to_iso] [mount_path] to check, if there are any files, this method is not for you)

Tobias Scherbaum already blogged this, but only in german, so I'm writing this again for the Planet Gentoo readers.

A german webpage called jugendschutzprogramm.de provides filters for webpages potentially dangerous for children. Now some people noticed that this page considers quite a lot dangerous.

Both gentoo.de and gentoo.org are considered only suitable for people over 14. So if you ever thought about installing Gentoo on the PC of a kid, think again what you might do to that kid.

Beside, my blog is even more dangerous: It's blocked by default.

The page is supported by a couple of companies providing pornographic content. Interesting enough, it's also supported by a big german Newspaper (BILD) that regularly has pornographic images on their frontpage. However, their page is considered harmless.

But what's really frightening is that jugendschutzprogramm.de is part of ICRA, an international system by big content and internet providers. It's even supported by the european union.

Update: Page has XSS, maybe someone wants to play with it?

<form action="http://jugendschutzprogramm.de/webmaster/label-generator.php" method="post">
<input name="URL" value='"><script>alert(1)</script>' type="text">
<input name="submit" type="submit">
</form>

A common way to check the health state of a hard disk is SMART. It gives various informations about occuring errors. In Linux, there's the smartmontools package containing tools to read SMART data of hard drives (smartctl -a /dev/[hddevice] gives you a bunch of information).

I found it always frustrating that SMART didn't work with USB drives. It's a standard bound to IDE/ATA. Although common USB-drives are internally IDE/SATA, sending the SMART commands to the drive requires proprietary extensions. But now, the smartmontools-developers have included support for some USB drives. It worked with the USB HDs I had available for testing.

There's no release yet containing the USB-support. If you're on Gentoo, you can fetch a live-CVS ebuild here.

The free software projects for media playing did a good job in the past on supporting a wide variety of formats. From the common to many very obscure formats, current versions of the free software mediaplayers were usually able to play them. Today it's even common to suggest vlc for Windows users if they can't play unusual media formats.

Though there were a few exceptions, the most notable probably the long-time missing support for many of the Real formats. While these are rarely used today, many archived videos in the Internet still rely on it. For example, many german television stations provide real video files on their webpages.

Recently and without much public notion, ffmpeg first got support for RV40, some weeks later also for RV30. This fills a long time gap in free software support for video formats. ffmpeg is used by all major free software video players (vlc, xine, mplayer), so you should get the support within some time in all of them. For now, it's quite easy to checkout mplayer from subversion and build it on your own.

Want something to try out? Here's a video from Desert Planet in real format.

The only gap I know of a format that really got usage in the wild and that is not yet supported by free software is WMA3.

As an FSFE fellow, I got interviewed for their webpage.

You can read it here.

Recently there were some News that Lenovo does not like Linux any more. This was supported by comments like this at Lenovoblogs (by a Lenovo engineer):

»Again, what’s the incentive for us to start providing all of this intellectual property for free to the Linux community? You may say it drives support for Linux on ThinkPads and people would buy more ThinkPads as a result. I think that’s a dubious assertion at best.«
(the subject was driver support for switchable graphics on modern thinkpads and brings up some common urban legends about linux and driver support)

Sadly, I experienced one more place where Lenovo seems to shift away from a Linux friendly viewpoint: I tried to return the windows license of my new Thinkpad with a pre-made form by Lenovo itself (I got this from someone else by eMail, not from Lenovo directly). In the net, you can find tons of reports that it was easy for people to get money back for their windows licenses by Lenovo.

Though what I got was this:
»Leider können wir Ihrem Wunsch nach Rückerstattung der Kosten für das auf Ihrem Lenovo Produkt vorinstallierte Microsoft-Betriebssystem nicht entsprechen, da das Betriebssystem aus unserer Sicht einen integralen Bestandteil des jeweiligen Lenovo Produkts darstellt.«
(rough translation: We won't refund your windows-license, because we think it's an integral part of the product)

I find it hard to understand why Lenovo makes this shift. When running around on linux conferences in recent months, the number of thinkpads is hughe. While many other vendors shift to a much more free software friendly behaviour (think of AMD/ATI), Lenovo seems to go the different direction. It's especially strange because Lenovo is probably one of the few vendors that has a notable market share in the linux community.

By the way, I welcome any hints how I should continue with the windows refunding. I'd prefer not to capitulate yet (like I did with my last laptop by Samsung), and I assume the law is clearly on my side.

Update: As some of you asked, here is the form by Lenovo, though you'll probably just get the same reply I got.

Probably interesting, here you can find all EULAs from Microsoft. They are quite clear on the subject and say that you MUST return the windows license to the vendor if you don't agree to the EULA.

In the meantime, I wrote several messages about the issue to various people and instutitions. The FSFE is also working on the subject.

Today my new IBM/Lenovo Thinkpad T61 8895WFJ laptop arrived. While my P30 did a good job, it really was time to replace it.

I'm currently in the phase of installing Gentoo and getting used to the device, but I think it was a very good choice.

Beside the fact that Lenovos are probably popular for a reason, the 1400x1050-resolution, the well Linux-supported Intel-graphics and a quite acceptable weight (2,4 kg) were reasons for this model. I'm still in favour of 4:3 screens, because if you wanna have a 16:10 one with a decent resolution (e. g. > 1000 pixels height) they become either very expensive or very heavy. I still wonder why no vendor seems to produce 4:3 screens any more (from my research, not a single Montevina laptop has 4:3).

Some time soon you'll probably find some documentation about Linux on the T61 8895WFJ at http://www.int21.de/t61/.

Recently I was asked by a friend for a linux tool to extract ressources from windows exe files, especially icons. He used a windows tool in wine till then.

I said that this shouldn't be so hard and already started writing my own parser (I came to the point where I could extract headers and content separately), when I found that there already is an appropriate tool called wrestool. It's part of the icoutils package.

wrestool -o . -x filename.exe
will extract all ressources (icons, cursors etc.) to the current directory.