May 2009 Archives

Virtualisation is one of the big next things to become mainstream technology. But it's a complex technology as there are many different software solutions available that differ greatly in terms of usability, flexibility and performance. As always there is a trade-off between ease-of-use and the performance of the guest system on your hardware. This time we will try to get a virtual machine up and running with a minimum of possible obstacles on our way to success, and we will try to install a cutting-edge Linux OS, Fedora-11-Preview, which will finally be released in about two weeks, today!

After all, trying out a new Linux distribution without the risk of making a mess of your hard disk is one of the most attractive features of virtualisation. What makes creating a guest virtual machine easy is the graphical user interface of VirtualBox from Sun Microsystems, that will guide us through the process in an intuitive way. Nevertheless the downside of this software is that even though it's open source, it's not entirely free. According to Sun's "VirtualBox Personal Use and Evaluation License (PUEL)" the software is free to use for personal and educational purposes as well as for product evaluation:

Sun grants you a personal, non-exclusive, non-transferable, limited license without fees to reproduce, install, execute, and use internally the Product a Host Computer for your Personal Use, Educational Use, or Evaluation. "Personal Use" requires that you use the Product on the same Host Computer where you installed it yourself and that no more than one client connect to that Host Computer at a time for the purpose of displaying Guest Computers remotely. "Educational use" is any use in an academic institution (schools, colleges and universities, by teachers and students). "Evaluation" means testing the Product for a reasonable period (that is, normally for a few weeks); after expiry of that term, you are no longer permitted to evaluate the Product.

But even though you'll need a commercial license from SUN to use VBox after evaluation, the download is free of charge and the PUEL allows for intense testing and is practicable for a number of non-commercial environments.

Getting VirtualBox for Your Linux Distribution

As we are going to install a new Fedora 11 as a guest on top of your current Linux system, the host, the first step is obviously to get VirtualBox for your host system and install it there. Fortunately SUN has prepared binaries for a number of different Linux distributions for download, so go to Sun's download page here and choose your package. Binaries are available in rpm or deb format, so your system's native installation tool should do the job. Just bear in mind to install the kernel headers and the gcc compiler first, to enable VirtualBox to compile a kernel module that will be loaded on boot after installation.

Unlike XEN which changes the way your usual Linux system works fundamentally, VirtualBox is just another application that runs inside your normal environment without changing anything on your host, except that it relies on the new kernel module created on installation.

Preparing VirtualBox for Installation of Fedora

Do you have two hours to burn? Let's get going.

To get to the point where we can start to install Fedora, we basically have to prepare a virtual disk and attach a boot medium to the virtual machine.

By now you will have found the VirtualBox GUI in the System Tools menu and have started to create a new virtual machine. First, we have to give it a name and select the OS type. Adjust the memory slider to at least 512 MB or even more if you can afford it. The more memory you assign to the virtual machine the faster it will run eventually.

In a second step we need to create a virtual hard disk in a file. During installation of the guest system this file will be used as the device "/dev/sda". All formatting and partitioning is being performed inside this file although it seems to be a normal block device for the guest system to use. Later, when Fedora uses its hard disk you will be asked to erase all your data on the device "/dev/sda". I can see the sweat on your face when you click the OK button for the first time. Your real hard disk, called /dev/sda as well, will survive this procedure, believe me. You have to choose the final size of your filesystem for the Fedora installation here, so think twice how much space you will need.

As you can see I have only spent 4 Gigabytes for my Fedora test system, enough to finish a default installation. But your mileage may vary, you know, and it's not easy to expand your virtual disk after finishing the installation.

Initialising the file that represents your virtual disk will take a while. Check that the correct vdi-file is selected and finish this part of the job.

Before you can start the new virtual machine (to install Fedora) some minor changes to the default settings have to be made. First of all I would recommend to change the network configuration.

Usually the VirtualBox GUI selects NAT, but I think it's more straightforward to configure the network interface to share the physical host interface with the new guest system. I don't know whether or not this is really necessary in your situation, but it does not hurt anyway.

As a last step we need to tell VirtualBox where to find the installation media. If you have a download of the Fedora packages at hand stored in ISO format, you can use it here. For the sake of simplicity I would like to go a different route and would ask you to download the 166 MB netinstall.iso file from the Fedora downloads page. This file could be burned to a CD making up for a nice boot CD, but we can use the downloaded iso-file here as the requested "Image File". The only problem is, that we did not attach the file to the virtual machine yet.

To make it available we have to go back to the "Virtual Media Manager" in the File menu and choose the downloaded iso-file as a CD/DVD image.

I hope you have come this far and I'd like to encourage you to start your new virtual machine to go ahead with the installation. Once you've hit the start button again, you should be greeted with the Fedora installation window, and from this point onwards everything should work as if you had inserted a real CD and tried to install on bare hardware. In the course of installation you will inevitably reach the point where the hard disk has to be formatted, so keep an eye on the size of your install disk, it is exactly the size of your vdi-file. It does not matter that your real hard disk is also referenced by /dev/sda, your new /dev/sda is entirely separate, so dont't hesitate to repartition it to your hearts content. And you can safely install the GRUB boot loader in the master boot record, your original one will not be overwritten by the new installation. That's the magic of virtualisation.

After finishing your Fedora installation, there is one thing to remember, you have to remove the installation media before booting (the virtual machine) again. Well, you know how we had attached the iso-file to the machine, simply untick the box to detach it, then power off the virtual machine and start it again. Congratulations to your new Fedora 11 almost two weeks ahead of schedule.

Normally, a user is someone listed in the system's database file /etc/passwd. There is no need for a flesh-and-bone user equipped with a password and ready to log in. Many users listed in the system database are simply immaterial daemon users, like mysql, listed for the operating system to be able to assign a name (especially a UID in the form of a number) to a running process.

But at least these daemon users own processes, i.e. the mysql database server, and of course, real objects like files and directories are owned by those users. In a well run environment these users do not have a password, so their account cannot be abused by somebody else, and almost always the shell that would be started if someone could log into this account is the binary /sbin/nologin, which does exactly what's on the tin, denying login.

And then there is another breed of users that seem to be ordinary fellows in the system and still do not exist as a user in the normal sense. These virtual users do have the ability to run certain programs, they all have a mailbox in which their private messages arrive and they all need a password to access their home directory, in which all their assets are being stored, but strangely enough there is no trace of them in the system's user base and password file. One can argue that because the virtual users don't have a valid number (UID) they are no users, and in a strict sense this is certainly true. But how can they own files and use a mailbox, just like other users do?

Hunting down the virtual users

We all know that in order to have a mailbox, there has to be a mailbox file (usually in /var/spool/mail) that is owned by the user respectively. And if someone is attempting to read this mailbox, there has to be some kind of authentication, so the password must be stored somewhere. At this point a real (daemon) user comes into play, the postman Tom, who of course has a valid UID listed in the file /etc/passwd, which is in fact 555 on my system.

Tom works at the heart of the mail delivery process, managing the virtual user's home directories and their mailboxes, and he is the only real user necessary to serve hundreds of virtual users on the system.

Virtual home directories and mailboxes

For every virtual user Tom creates a home directory that is used by IMAP to store the virtual user's inbox and various index and logfiles individually.

All these files have been created by Tom for each of the virtual users. In order to provide this infrastructure, we have to make sure that Tom is able to use the virtual user's password when needed, and most importantly to handle the authentication process for them as well. The following lines of code show how the configuration of DOVECOT has to be changed for Virtual Domains.

All passwords are stored in the file /kx/horde/htpasswd in the usual way required by the apache web server. This is important for two reasons, because the dovecot process can use this file to authenticate virtual users by changing permissions to apache for authentication, and simultaneously this file allows other web-based software to access the virtual user's homes with the same password. We will have a look at this later.

Getting all email in the right direction

By now we have managed to install mailbox access for virtual users via IMAP without the need of registration of all these users in the system's database. What we do not have in place is a mechanism to fill up the virtual user's mailboxes with incoming mail.

As you may suspect another important part of the mail delivery process has to be adjusted to ensure that the virtual users who can have totally different email-addresses will receive their mail without hassle. This time we need to add a few lines to the POSTFIX configuration file and we have to create a mapping between the email addresses and the (real) virtual users, who are supposed to read the mail.

The pivotal point here is the text file "kx/dovecot/virtual_mailbox_map" in which each email address is followed by the virtual user's name. But before postfix can use this database to deliver mail it has to be hashed, i.e converted into a database file "kx/dovecot/virtual_mailbox_map.db" with the postmap command below.

Extending our infrastructure

The most important benefit of this system is not to have virtual users in the system's user database and being able to treat them as ordinary users at the same time. As some users tend to choose a weak password this will help to limit the damage that can be done to the system considerably. But once virtual users are established with an email address and password in the manner I have described, it seems to be an obvious idea to use these credentials for another purpose, to organise a file system-like structure for virtual users, to store their data as well. Fortunately such a file storage can be easily set up using the HORDE framework, and in particular the gollem module, that can be configured to use the file system or a MySQL database to store the virtual user's files while making them available for upload and download via a browser-enabled interface.