Most Popular Postings

Booting a CD Without a CD Drive

By Ralph on August 31, 2009 9:04 AM | No Comments | No TrackBacks

ISO Files

Most Linux distributions and a number of CD utilities like Clonezilla or GPARTED-Live come as iso-images, ready to be burned to a CD medium, to start an installation or running a live session for some useful purpose. But all these iso files are supposed to be booted inside an ordinary CD drive. What if you just don't have a CD burner at hand or if your CD drive is damaged or missing all together, because you are testing the shiny new motherboard without a CD drive?

Don't think you are stuck, there's help around the corner that will lead you into a habit of booting your iso files in a much more creative fashion.

"Stop for a moment, could I not simply put the iso file on a USB key using the dd command? Most modern BIOSes are perfectly capable of booting USB keys and surely there is a USB slot on your new motherboard you can use."
Of course you could do that, the iso file contains a valid iso9660 filesystem together with the bootloader for CDs. And the dd command will transfer the file to your USB key without adding any other structure. But, don't expect the new USB key to boot like a CD, because after you have changed your BIOS settings to allow booting from a USB hard disk, the BISO will expect the structure of a hard disk not that of an iso-filesystem. It will look for a MasterBootRecord (MBR) in sector 0 in the same way as if it would boot your ordinary hard disk. And an iso-filesystem doesn't have a MBR and a partition table where the BIOS expects it to be. So forget booting iso files directly.

"Ok, seems as if there is no way around putting a bootloader like grub on the USB key, and ... , wait, there is a file called iso9660_stage1_5 in the grub directory. It looks like grub has iso9660 support already. Can we use this to boot the iso file?"
Well, unfortunately no. The iso9660 support built into grub can be used to create a bootable CD with grub as the bootloader. Your file is being used when a new iso file containing your grub bootloader is created. What we need instead is the ability to read the kernel and the initramdisk inside the iso-filesystem at boot time when no filesystem is mounted.

"Sounds as if we need a loop device to get at the files inside the iso filesystem. Does grub support mounting a iso-filesystem using a loop device?"

The New Grub

Yes and no. The traditional grub (version 1) that everyone uses does not have this loopback support, but the grub development team has been working on grub2 for a number of years now, which has loopback support already. The new grub2 is still under development, so it's not yet ready to replace the legacy grub version 1, but grub2 is exactly what we need to boot our iso files directly.

A Word Of Caution

So we are about to prepare a USB key to boot iso files with the grub2 bootloader written into the key's master boot record. Writing MBRs on a Linux system usually raises blood pressure a bit, as we have to be extra cautious not to ruin our running Linux system. If you want to follow me from here, please make sure that you do not perform the grub2 installation on your company's production server and double-check everything before issuing commands, to be on the safe side. And for those of you, who hesitate to write MBRs at all, I have an offer for you at the end of this posting, read on.

Preparing The USB Boot Key

Like any other software grub2 can be installed using the distros repositories, at least with Fedora 11 that I use now. The installation will not replace our legacy grub software we are using to boot our computer, as the new files go into a directory "/boot/grub2" without changing anything of the current grub setup unless grub2 is deliberately used to overwrite the MBR. And this is the only thing we have to avoid. But for now, our objective is to get the grub2 software onto our USB key, writing the key's MBR will then be the crowning event before we start testing the key.

For preparing the boot key it is absolutely essential that you know how the Linux kernel recognizes your USB key. To find out, you can observe the log file "/var/log/messages" while you plug your key into the computer. For now I assume that you have one hard disk in your computer (/dev/sda) and your USB key will probably show up as /dev/sdb. First we re-partition the key and create two Linux partitions, a small one of 10 MBytes for the grub2 software and a larger one where all our iso files can be stored:

#> umount /dev/sdb1
#> fdisk /dev/sdb
#> mkfs -t ext3 -c /dev/sdb1 ; tune2fs -L usbboot /dev/sdb1
#> mkfs -t ext3 -c /dev/sdb2 ; tune2fs -L isos /dev/sdb2

You can now mount the second partition and copy all the iso files you wish to boot into this location. Writing the MBR now requires that the first partition is mounted on a well know mount point like /mnt, because we will use this directory to write the grub2 software to the key with the following command:

#> grub2-install --root-directory=/mnt /dev/sdb

Please double-check that your key is mounted and is called /dev/sdb before you use this command, which populates the directory /mnt/boot/grub2 with grub2 modules and writes the MBR into /dev/sdb.

As you may know there is one task still left before we can start testing our boot key, we need to create a menu list, which is called "/boot/grub2/grub.cfg" now. A number of changes have been made to the config file compared to the old grub, so let's have a look at the new configuration:

set timeout=5
set default=0

menuentry "ISOLINUX" {
loopback loop (hd0,2)/centos.iso
set root=(loop)
linux /isolinux/vmlinuz
initrd /isolinux/initrd.img
}

The most remarkable change is the numbering of partitions which has become more natural as the first partition now is (hd0,1), while hard disks still start with number 0. Then the old title entry is replaced by menuentry and loading the kernel is now done with the linux command. Apart from all these cosmetics we can now create a new device called (loop) that will replace a real partition like (hd0,2) to access files inside the iso-filesystem, once we have connected the iso file with this new device using the loopback command.

It is even possible to list the files that grub2 can see on the (loop) device using the grub2 command line:

grub2> set root (loop)
grub2> ls /

Now it's time to check if the ISOLINUX menuentry works. Reboot and see your new boot key fire up the CentOS netinstall process. Voila.

Download Your Bootable USB Key

I have prepared an image of my boot key for you that you can download, if you want to create a boot key without the hassle of following the steps above. The small 32 MByte image file contains all the magic and can be copied to your USB key with the dd command. Please send me an email and I will give you a download link with the necessary instructions. Once you have your key working, you can delete and re-create the second partition to make space for all your live CD iso files on the key. All you need to edit is the config file.

That way you can carry all your distros on one key and boot them all from the grub2 menu without having to change the Linux system on your hard disk.




Clone Your Hard Disk Today - With Clonezilla

By Ralph on June 6, 2009 5:35 PM | No Comments | No TrackBacks
I know, you are one of those people who know that it is absolutely essential to make regular backups of your system's hard disk. There is no excuse not to make regular backups, but as we all know, we don't really do that. Or is it just me?

No, you have to admit that even though the reasons to make backups are indisputable, something strange always - no, not always, but many times - keeps you from doing the necessary, updating your backups and securely storing them away. Making backups is not fun, but that's not the reason why we don't do it regularly. I suspect that you as well as I simply don't have the right tool yet, to initiate an automatic process that boils down to getting a reliable backup complete and safe to restore, which does not require much attention while assembling the data for us.

Fortunately this situation is going to change dramatically for you, today. I've run into a backup solution that is just that, an automatic process that does not require user intervention when started to do the right thing. It's called CLONECILLA and is essentially a Live Linux system, optimised for backup and recovery.

Clonezilla - Your bootable backup CD

Clonezilla can be downloaded here as an ISO-file of 106 megabytes.. This file contains everything that makes up a bootable Linux system, so be careful to write this downloaded iso-file to your CD in raw-mode. Because it's of no use to have the iso-file on a CD in another file system, the iso-file is the file system and it is the only thing that has to be on the CD.

Did I mention that Clonezilla is a backup solution that covers all sorts of partition types, so that our windows-using friends could use it, too? Clonezilla has the ability to clone a complete hard disk no matter what kind of partition your disk comprises of. It even backs up the master boot record and the partition table, you will get a complete set of data from which all or any part will be recoverable when you need it. And all data is created as zipped files of max. 2 gigabytes size so that everything can be burned to DVDs, if needed.

Setting Clonezilla on the right track

As we are now heading for a complete backup of your laptop's hard disk, one thing is to be considered before we start, where will all the backup data be stored? Fortunately Clonezilla offers a number of possible storage media ranging from an additional (plugable) local disk, a SSH server or a SAMBA server to the traditional NFS server. I assume that you have a portable USB hard disk on which you will store the backup. Please ensure that the free space on this disk is about half of the size of what you are going to backup, and add a little bit of a buffer, too.

I have found that a second thing is important before you start letting Clonezilla create the backup, make sure that all partitions of your hard disk can be mounted. Best practice would be to perform a file system check on all partitions before starting the Clonezilla boot CD.

The following few steps are all you need to initiate a full backup of your laptop's hard disk and you will see that once you've selected a few things the rest is going in a jiffy.

Step 1 Boot your Clonezilla backup CD, select language and keymap, then select "Start Clonezilla"

Step 2 After selecting the backup media (local_dev) power on your USB disk and allow a few seconds for the system to recognize your new hard disk on which the backup will be stored. Clonezilla creates a directory on the USB disk derived from the date and time of the backup. You can choose where to store this directory on the disk.

You can always switch to another terminal using <CTR><ALT>F2 to see that the USB disk is now mounted on /home/partimag. All files go into this directory.

Step 3 Select "savedisk" to ensure that the whole hard disk is backed up. When the partition information is read in Clonezilla will start the backup process automatically.

Step 4 Depending on the size of your hard disk, you can now relax and let Clonezilla compose your backup for you. Check the backup directory in the meantime to see it filling up with data.

The performance of creating the backup is very good, it took 47 minutes for a 30 gigabyte data partition on my three year old laptop. The whole disk (120 gigabyte) was ready after two and a half hours time leaving me a total of 66 gigabytes of compressed data.


Try Clonezilla today, it takes very little time to be secure. No excuses!











                                    
Improve Your VirtualBox Experience

                                    

                                        

                                            By Ralph on June 29, 2009  8:41 PM

                                        

                                        | No Comments


                                        | No TrackBacks

                                    

                                


                                


                                    

                                         Maybe you have  followed my guide to easy virtualisation with Sun's VirtualBox software and you are familiar with the basics of running a virtual machine inside your normal Windows or Linux system by now. I hope that you enjoy the many features of your new virtual machine as most devices used by the virtual machine just work out of the box. Reading CDs and using the audio device for playback should work right away and setting up a printer via the network connection is a matter of few clicks. 

But the one problem you have certainly run into by now is the low display resolution which defaults to 800x600 pixels, considerably impeding your freedom on the screen. Two simple steps will sort out this problem forever. First select a suitable "Monitor Type" in the Administration - Display dialog and save the setting. Then edit the file "/etc/X11/xorg.conf" and add something like the following in the SubSection "Display" part:





       SubSection "Display"
                Viewport   0 0
                Depth     24
                Modes "1024x768"  "800x600"
       EndSubSection





After logging out and restarting the X server again you should have a much bigger screen that almost covers the whole display if you switch into FULLSCREEN mod with <Ctl>F.




Guest Extensions pave the way for a seamless desktop experience


During the installation of VirtualBox a custom kernel module was being built that enables the host system to manage the guest, but the guest system runs without any consciousness of the fact that it is only a virtual machine. To ensure a better integration between the host and the guest, SUN have developed a package called "Guest Extensions" that must be installed inside the guest to make it VBox-aware. The installation of Guest Extensions culminate in compiling another kernel module that is being used by the guest to improve communications with the host and to make additional features like shared folders available.

Before we can use these desired features we still have to prepare the guest system to compile kernel modules, that means we have to install a full development stack inside the guest system for kernel modules first, and then install the Guest Extensions. Use the following yum command to prepare the CentOS/Fedora host:
  




yum update kernel kernel-headers kernel-devel dkms




The Guest Extensions come as an iso-file "VBoxGuestAdditions.iso" that could be found in the directory "/usr/share/virtualbox". Make sure that this iso file is available as a CD-iso-image to the guest system by adding it to the appropriate section in the Virtual Media Manager menu. Restart your virtual machine to make sure the new, updated kernel is actually running and start the setup script "VBox LinuxAdditions-x86.run" on the CD image after you have mounted the iso-file from within the guest system. After rebooting the virtual machine the new kernel module for the guest becomes active and the extensions are ready to use.

The first thing you'll notice ist that the mouse pointer is no longer confined to the guest window, you can click on everything that is visible on the screen, no matter if it is a host or a guest window. And, of course, you now can copy and paste text between the host and the guest as you like it. A much more important change took place behind the curtain, your access to the hard disk is faster now. Let's have a look at the following table
which gives an overview over transfer rates measured on my laptop.






 Action 
       CentOS 5.2 HOST  
        Fedora 10 GUEST 
WITH Guest Extensions  		

        Fedora 10 GUEST 
without Guest Extensions



 creating files with zeroes 
       27.1 MByte/sec  
        21.8 MByte/sec

        14.1 MByte/sec



 copying files 
        12.47 MByte/sec
         11.96 MByte/sec
        N/A




  writing to network shares
        12.26 MByte/sec
         5.6 MByte/sec
        5.4 MByte/sec




 writing to shared folders 
        18.0 MByte/sec
         6.7 MByte/sec
        not available







As you can see the performance has considerably improved for writing files to disk and is almost the same for writing normal files while there is still a huge difference when network attached storage is used to extend the usually poor disk space of the virtual machine. But the use of virtual folders at least enables the host system to write data to the guest's file system rather quickly, while the guest does not really gain any speed doing the reverse operation.

Anyway, installing SUN's Guest Extension truly polishes your virtual guest machine and lets it shine brighter than before.






                                    



                                










                                    
Getting Outahere - Explained

                                    

                                        

                                            By Ralph on April 22, 2009  7:23 PM

                                        

                                        | No Comments


                                        | No TrackBacks

                                    

                                


                                


                                    

                                        
How do I connect to the Internet?


By putting a network cable into the DSL router, well, most of the time, yes. But sometimes my laptop needs to establish the connection to the Internet using a wireless data modem. Obviously this will change a few things on the machine that are not entirely clear to someone starting on this track for the first time.




In this article we will look behind the curtain to understand the process of dialling into the Internet via a wireless modem.



Let's begin with the basic differences. In using the modem, we have decided that the laptop itself will become part of the Internet, while it was only part of the local area network up to now. First of all, that would require to run a firewall on the laptop if you did not already have set up a firewall before. Protecting your machine while connected to the Internet is indispensable, and you have to make sure, that your firewall is not disabled entirely. Check your firewall rules with the following command:





/sbin/iptables -L




Chain INPUT (policy ACCEPT)

target     prot opt source               destination
         


Chain FORWARD (policy DROP)

target     prot opt source               destination
         


Chain OUTPUT (policy ACCEPT)


target     prot opt source               destination





 
The output above shows, that there is no protection and you have to solve this problem first. Anyway, firewall configuration will be covered in a separate post.




Where do all these packets go?

As long as your machine had been using the router an IP number belonging to the router was listed in the laptop's routing table. More precisely, the ethernet adaptor would send every data packet destined for the Internet to the machine with this default address, and for the laptop to be able to establish its own connection, the old default route to the DSL router has to be removed from the network setting. Nowadays we simply click through the network dialog and delete the "default gateway address" there. But if you like to know where to check that setting, please examine the file "ifcfg-eth0" in a directory called "/etc/sysconfig/network-scripts", there should be no GATEWAY line.





It is one of the tasks performed by a process called "pppd" to set up a new default route when the connection to the Internet is successfully installed using the modem and a new network interface "ppp0". Getting the Point-To-Point-Protocol-Daemon (pppd) up and running will be the most important thing to get right, but then, fortunately it is the only thing to do. There was no need for a pppd as long as we had the router, but now this process working in the background is responsible for shovelling all packets back and forth into the Internet for us. 




Chatting up the PPP Daemon


Given that everything is well prepared, starting the pppd process is as easy as the following




/usr/sbin/pppd file three.options connect '/usr/sbin/chat -v -f three.chat ' /dev/ttyUSB0





As I am using a THREE data modem there are two files that determine the process of dialling into the Internet, an options file and the chat-script. The last parameter the pppd process depends on is the device file for the hardware used. In most cases running the pppd fails because this special file that represents the wireless modem does not exist.
Look around if your Linux system lists the device file, and  if not, we cannot proceed unless the hardware is actually accessible through the kernel. 

The kernel module that is responsible to make the wireless modem visible to the kernel is called "usbserial" and it has to be used with two pieces of information, the vendor-id and the product-id, which are different for every modem you use. For my wireless modem I have to use this command.




/sbin/modprobe usbserial vendor=0x19d2 product=0x0001



You have to check your actual hardware and find out which values are correct for your hardware, using the "lsusb" command for further information.



Now that the hardware is available through the device file "/dev/ttyUSB0" we can define a few options the pppd process should honour while starting up.



460800

noipdefault

defaultroute

persist

noauth

updetach

novj

novjccomp

nopcomp

nodeflate

holdoff 5





Apart from requesting the desired baud rate the process tries to get a remote IP address from the ISP's login server (noipdefault), does not require the ISP's end point to authenticate itself (noauth) and establishes the default route in the routing table after setting up the connection.



During this process the pppd process exchanges some information with the ISP's login server on the Internet to get the required IP address. The details of this conversation are lay down in the chat-script. 





ABORT   BUSY

ABORT   VOICE 

ABORT   "NO CARRIER"

ABORT   "NO DIALTONE"


ABORT   "NO DIAL TONE"

""      ATZ

OK ATE0V1&D2&C1S0=0+IFC=2,2

OK AT+CGDCONT=1,"IP","3ireland.ie"

OK ATDT*99#

CONNECT "" 





This file lists two corresponding pieces of information each line. While reading the modem input the process waits for the appearance of the first part being sent by the server, responding with the second part which is, in most cases of course, an old-fashioned modem AT command. You can watch this conversation between the modem and the ISP's access machine in detail if you display the tail of the logfile "/var/log/messages".
 




/tail -f /var/log/messages




Finally, when the remote IP address is obtained, the interface "ppp0" can be used.


Get a decent Name Service - And Enjoy!


Of course you should be able to surf the web using the new network connection as the default route should be created by the now running pppd process. If you still don't get your Internet applications running there is a last hurdle to overcome. The domain name service (or DNS for short) is always been used by our applications. No matter what kind of program you use, the first step is always the substitution of a domain name by the correct IP address. Because of the fact that this substitution has to be done many times, one or more name servers must be in easy reach for your laptop. I have found some nameservers that work for me and are reliable and fast, but depending on your location, different ones may be better. Try to find the ones that perform best for you and finally record their IP addresses in the file "/etc/resolv.conf" to make sure, that your internet connection is performing well.
 

Now, this is how I connect to the Internet!  The following excerpt is taken from the usual suspect file  "/var/log/messages"






                                    



                                


                                    



                                

                                

                            



                    
                    


                        

                    



                    

    



    
    

    
Search

    

        

            

            

            
            
        

    






    

        
        
    





    
Recent Entries

    

        
    



        
    



    
Categories

    

    
        
        
        
    
    






    
Monthly Archives

    

        
    



        
    






    

        
    





    

        OpenID accepted here
        Learn more about OpenID