Help, my RAID array does not complete synchronization!

Let us suppose the following situation: You have a Linux server with a software RAID1 array (md) and, for one reason or another (mostly because your are a lazy admin, admit it!), both disks are reporting unreadable sectors, either through SMART or through actual failed readout attempts.

So you installed a 3rd good disk, set it as a spare, then failed one of the 2 bad ones to initiate synchronisation onto the good new disk. However, all hell breaks lose as you find out your synchronisation doesn't complete (/proc/mdstat reports U_ or _U) and instead of ignoring the unreadable sectors as it should, md decides that it cannot continue.

Worse, if you look at your dmesg, you find out that it is being polluted by a continuous stream of:

RAID1 conf printout:
--- wd:1 rd:2
disk 0, wo:0, o:1, dev:sda1
disk 1, wo:1, o:1, dev:sdb1

Help!!!!

OK, first of all, since this information is quite hard to find, especially if you are in a hurry, here are what the abbreviations above mean:

• wd: working disks
• rd: raid disks
• wo: write-only (if set to 1, this usually indicates a problem, and that data duplication doe not occur for this device)
• o: online

Obviously, wd:1 as well as wo:1 for the second disk is not something we want to see. Why can't our good spare disk be added as R/W to the gorram array? Heck, if the problematic disk fails, that single-handedly contains our up-to-date data now, we will be in big trouble. What's the point of providing redundancy, really, if md fails to synchronize as soon as there's one measly sector it cannot read!

It's a bird! It's a plane! No, it's hdparm!

Well, the sad truth of md on Linux (which may have improved with newer versions) is that it isn't resilient at all when it comes to unreadable sectors during sync. I guess the developers decided that, since the point of redundancy is to always have at least one good set of data, they didn't need to focus on situation where the "good" set of data may also have some corruption, and therefore never planned for anything but try and re-read an unreadable sector forever, until the disk magically repairs itself (right... fat chance!).

Now (and for the rest of this post I will mostly be following the excellent information provided by Bas on his blog) to compensate for that oversight, the trick is to have md read the problematic sectors one way or another, so that the synchronisation can complete. May sound easier said than done but most of the time it shouldn't be an issue, as recent disks with SMART are engineered with a set of spare sectors, to be allocated in replacement of unreadable or unwritable ones for exactly this kind of situation. The issue however is that reallocation of sectors only occurs on write access.

What this means then is that, while the disk has the technology to "fix" itself, as long as you are only attempting to read the problematic sectors, reallocation will not be triggered and you will continue to get read errors. Thus, you must manually issue a write to the problematic sector(s) to trigger the "recovery" mechanism (NB: I'm using "fix" and "recovery" loosely, as you can of course not recover data from these sectors if they are reallocated, therefore will end up with some corrupted data).

This can be confirmed by checking the Offline_Uncorrectable (#198) and Reallocated_Sector_Ct (#5) reports from SMART:

# smartctl -A /dev/sda
smartctl version 5.38 [x86_64-redhat-linux-gnu] Copyright (C) 2002-8 Bruce Allen
Home page is http://smartmontools.sourceforge.net/

=== START OF READ SMART DATA SECTION ===
SMART Attributes Data Structure revision number: 16
Vendor Specific SMART Attributes with Thresholds:
ID# ATTRIBUTE_NAME          FLAG     VALUE WORST THRESH TYPE      UPDATED  WHEN_FAILED RAW_VALUE
  1 Raw_Read_Error_Rate     0x002f   100   100   051    Pre-fail  Always       -       105
  2 Throughput_Performance  0x0026   054   054   000    Old_age   Always       -       2759
  3 Spin_Up_Time            0x0023   084   084   025    Pre-fail  Always       -       4989
  4 Start_Stop_Count        0x0032   100   100   000    Old_age   Always       -       10
  5 Reallocated_Sector_Ct   0x0033   252   252   010    Pre-fail  Always       -       0
  7 Seek_Error_Rate         0x002e   252   252   051    Old_age   Always       -       0
  8 Seek_Time_Performance   0x0024   252   252   015    Old_age   Offline      -       0
  9 Power_On_Hours          0x0032   100   100   000    Old_age   Always       -       11496
 10 Spin_Retry_Count        0x0032   252   252   051    Old_age   Always       -       0
 11 Calibration_Retry_Count 0x0032   252   252   000    Old_age   Always       -       0
 12 Power_Cycle_Count       0x0032   100   100   000    Old_age   Always       -       10
191 G-Sense_Error_Rate      0x0022   252   252   000    Old_age   Always       -       0
192 Power-Off_Retract_Count 0x0022   252   252   000    Old_age   Always       -       0
194 Temperature_Celsius     0x0002   064   060   000    Old_age   Always       -       32 (Lifetime Min/Max 20/40)
195 Hardware_ECC_Recovered  0x003a   100   100   000    Old_age   Always       -       0
196 Reallocated_Event_Count 0x0032   252   252   000    Old_age   Always       -       0
197 Current_Pending_Sector  0x0032   252   100   000    Old_age   Always       -       0
198 Offline_Uncorrectable   0x0030   252   100   000    Old_age   Offline      -       0
199 UDMA_CRC_Error_Count    0x0036   200   200   000    Old_age   Always       -       0
200 Multi_Zone_Error_Rate   0x002a   100   100   000    Old_age   Always       -       2
223 Load_Retry_Count        0x0032   252   252   000    Old_age   Always       -       0
225 Load_Cycle_Count        0x0032   100   100   000    Old_age   Always       -       10

If you see a zero at the end of these attributes but the disk still reports that it has trouble reading sectors, it indicates that the sector reallocation process hasn't kicked in yet, and needs to be triggered manually.

The first order of the day then is to find the address of the sector(s) we should trigger a write to. This is fairly easy, as all you need to do is run a SMART test, with something like smartctl -t long /dev/sda and write down the first sector address where a read error is reported:

# smartctl -a /dev/sda
(...)
SMART Self-test log structure revision number 1
Num  Test_Description    Status                  Remaining  LifeTime(hours)  LBA_of_first_error
# 1  Extended offline    Completed: read failure       60%     10864         293039329
(...)

Once we have that address, we could of course use dd, but an even simpler approach is to use a recent version of hdparm, as it adds easy support for reading/writing a single sector.

First thing to try with hdparm then, is confirm that we have a problem accessing that sector:

# hdparm --read-sector 293039329 /dev/sda

/dev/sda: Input/Output error

This confirms what the SMART test reported. You can try a few more read attempts, to validate that the sector is busted, and then, you can issue a write so that the disk finally realizes it should reallocate that sector. Note that, because the operation obviously means destroying existing data, hdparm requires you to add a --yes-i-know-what-i-am-doing flag to issue the write, hence:

# hdparm --yes-i-know-what-i-am-doing --write-sector 293039329 /dev/sda

/dev/sda: re-writing sector 293039329: succeeded

You can then issue a read again, which will confirm that the sector has been reallocated:

# hdparm --read-sector 293039329 /dev/sda

/dev/sda:
reading sector 293039329: succeeded
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000
0000 0000 0000 0000 0000 0000 0000 0000

If you issue smartctl -A again, you should also see that the sector has been reallocated:

198 Offline_Uncorrectable   0x0030   100   100   000    Old_age   Offline      -        1

It's usually a good idea to use hdparm to read adjacent sectors as well, and correct them as needed, then repeat the operations above until the SMART self test completes without error and you have smoked out all the problematic sectors. At this stage, if you issue a resync of the array with the new disk, it should complete successfully and redundancy will be restored. Time to order another replacement and check your data for corruption. But at least, you are redundant again.

Addons

• To get details of your md array, you can use mdadm --detail. Eg.

  # mdadm --detail /dev/md2
  /dev/md2:
          Version : 0.90
    Creation Time : Tue May  6 18:43:16 2008
       Raid Level : raid1
       Array Size : 130030016 (124.01 GiB 133.15 GB)
    Used Dev Size : 130030016 (124.01 GiB 133.15 GB)
     Raid Devices : 2
    Total Devices : 3
  Preferred Minor : 2
      Persistence : Superblock is persistent
  
      Update Time : Tue Jan 10 13:42:29 2012
            State : clean
   Active Devices : 2
  Working Devices : 3
   Failed Devices : 0
    Spare Devices : 1
  
             UUID : 0be47c81:ede086ae:0c460403:d81de298
           Events : 0.3658859
  
      Number   Major   Minor   RaidDevice State
         0       8        3        0      active sync   /dev/sda3
         1       8       19        1      active sync   /dev/sdb3
  
         2       8       35        -      spare   /dev/sdc3

• You are strongly encouraged to check your syslog or messages for reports of I/O issues, especially if you want to locate the data that may have been affected.
• This method is not guaranteed to work! Sometimes a SMART test will report a read error but a readout of the sector using hdparam will work fine, so you won't be able to get the disk to reallocate it. However, tis shouldn't matter too much for md resync which is what we are interested in here.
• If your disk has a lot of unreadable sectors, it is possible that you may run out of spare sectors for reallocation. It's hard to say how many spare sectors are made available by hard drive manufacturers, but I assume it isn't that many.
• You may have a problem recompiling a recent version of hdparm on some older Linux systems:

  fallocate.c: In function ‘do_fallocate_syscall’:
  fallocate.c:39: error: ‘__NR_fallocate’ undeclared (first use in this function)
  fallocate.c:39: error: (Each undeclared identifier is reported only once
  fallocate.c:39: error: for each function it appears in.)
  make: *** [fallocate.o] Error 1

  If that is the case, just add:

  #define __NR_fallocate 285

  in fallocate.c
• Some disks seem to be smart enough (no pun intended) to do further correction, once they have registered Offline_Uncorrectable sectors, so you may actually find out that, after a few hours, the value of Offline_Uncorrectable falls back to zero, and still the sectors can be read or written with extended SMART tests not reporting any issue. Pretty neat, but I still wouldn't entirely trust the disk...

Installing OSX (Snow Leopard) + Linux (Slackware 13.37 x86_64) on a GUID/GPT disk, with Software RAID enabled (ICH8R) - Part 1

Now that's quite a mouthful of a title. And considering that there is much to discuss, I will break this whole thing into a multipart post, so here is the first part of our series.

Introduction & Requirements

First of all, I gotta start with the disappointing news that, despite what you might anticipate from the title, this is not a guide to setup motherboard RAID (a.k.a. fakeRAID) support in either OSX or Linux (though the later is no biggie - see previous posts here for various options). The mention of RAID here is to indicate that we want to do an OSX and Linux that coexists nicely with a previous fakeRAID installation, by NOT requiring you to disable RAID settings in BIOS if they are already enabled.

In practice here, the goal is to add OSX and Linux on a separate non-RAID disk to an existing Windows 7 installation that was set to use fakeRAID (RAID 1 from ICH8R), without touching the BIOS settings.

Basically, what happened is that I recently upgraded my good old Asus P5B Deluxe based rig to a dual Samsung F3 1TB, installed Windows 7 ontop of the ICH8R RAID 1 array which I created in BIOS as a result, and since that left me with a bunch of unused SATA drives, I thought I would put one of these to good use by installing both OSX and Slackware on it (that Slackware 13.37 sure IS nice!), and use it to boot the whole lot. Sounds easy, but it actually presents a nice set of challenges, some of which include:

• Adding extra drivers to a vanilla OSX installation media, so that it recognizes our AHCI controller even in RAID mode
• Allow LILO (or GRUB) to boot OSes other than the main Linux system on a GPT/GUID partitioned disk (with a BIOS that is NOT EFI)
• Installing OSX Snow Leopard from scratch, including updates and necessary drivers for your rig, but WITHOUT having to spend one's life endlessly browsing the OSX86 forums. Yes, the effort is much appreciated guys, but I have better things to do then become an expert on kexts, Chameleon, DSDT, iBoot and whatnot.

My specs:

• Asus P5B Deluxe with 2x 1TB SATA HDDs in RAID mode (ICH8R) and 1x 320GB HDD (non RAID)
• nVidia GPU (7950GT)
• PS/2 keyboard, USB mouse + IDE DVD (the DVD is only going to be used for iBoot, so SATA or IDE won't matter one bit)

Requirements:

• OSX Snow Leopard official installation DVD or .dmg image (build 10A421A)
• One 8 GB (or bigger) USB key
• An existing Snow Leopard installation. Yeah, I know it sucks, but hey.

That last item means either you need to create a VMWare image from the DVD media, or borrow a friend's Mac, but we want to create a USB OSX installation medium, which we can customize so there's little way to bypass that requirement. If you really don't have access to a preexisting OSX system, it's probably possible to create an USB key from Linux from the OSX installation DVD, but I haven't tried it so I can't comment on that. Besides, a VMWare image with Snow Leopard shouldn't be that hard to find.

My kingdom for AHCI? Surely you are jesting!

Question: What do all the hackintosh installation guides you find on the internet have in common?
Answer: They all begin with "You MUST change your BIOS settings and set your SATA controller to AHCI before you begin the installation"

Of course, this is utter nonsense. If a guide begins with "You MUST change your BIOS settings to something potentially incompatible with what you want, and risk destroying existing data as a result" (by disabling RAID mode from ICHR), then it's a lousy guide and you should stop right there. For having been a fakeRAID/motherboard RAID user for some years now, with both Intel and nVidia solutions, I'll tell you frankly: all those who say "Don't use fakeRAID: buy a dedicated RAID controller instead" are idiots. First of all, money to buy a separate controller doesn't grow on tree and secondly, with all these cores lying around, the idea that one should spend extra, when all they want is add fault tolerance to consumer grade hardware is ludicrous. We're not building an enterprise grade server here dammit! Besides, most of these solutions are rather well supported in Linux now, which means that if you screw up your fakeRAID array, you may just be able to do something about it. With a hardware proprietary non reverse engineered RAID solution on the other hand, you might not be so lucky...

Therefore, the hell with changing your BIOS settings to AHCI! If Linux doesn't bat an eyelid about accessing SATA HDDs with an ICH controller in RAID mode, why should OSX be the exception? Therefore, we'll keep our ICH BIOS settings to RAID, thank you very much, and ignore all this AHCI brainwashing.

Now, since the OSX installation media has not been designed to support ICH8 in RAID mode, if you try something like iBoot and an unmodified OSX installation media, then unless your HD is plugged on a JMicron SATA controller (which is an actual possibility on the P5B Deluxe), the OSX installer will not see it. This basically means we're gonna have to modify the OSX installation system so that it sees our ICH8R controller as AHCI ICH8 and that's also where doing the install from an 8GB (or more) USB key, that we can modify on the fly, will come real handy.

But first, we gotta create the base vanilla installer on our key.

Creating a vanilla OSX installation USB key

To do that, you need to be in your existing OSX system (VMWare, soon-to-be-ex-friend's machine, etc.). I'm going to assume that you have the vanilla OSX disk as a .dmg. The steps to create the key then are as follow:

1. Plug in your USB key. Don't worry if it automounts.
2. Open the OSX disk utility (Applications -> Utilities ->Disk Utilities). You should see something like "8.02 GB Generic STORAGE DEVICE Media" or something, as your USB volume
3. Click on the "Partition" button on the left, to access the partition tab, then click on the "Current" Volume Scheme button so that you get a dropdown, and select 1 partition
4. In options, under the partition diagram, make sure you select "GUID Partition Table". For the Format, you select "Mac OS Extended" (though it shouldn't matter)
5. Click Apply to partition & format the USB key
6. Once formatting is complete, select the Restore button
7. Now double click on your OSX .dmg image to mount it (or insert your OSX DVD). A "Mac OS X Install DVD" icon should appear on your desktop
8. Drag that icon onto the Source field of the Restore parameters in Disk Utility until a green '+' sign appears, and then drop it there
9. For the Destination field, just drag and drop the "Untitled 1" partition, or whatever you called it, from your newly formatted USB stick
10. Click the "Restore" button. You will be prompted for the system's administrative password.

For those who want to see it for themselves, the whole process of creating an OSX installation USB key is demonstrated by the first part of the video below (credits go to stellarola - you can ignore the Stella Magic part that comes afterwards):



The restore process will take a few minutes (15-30 mins), but, once completed, will give you the ability to install OSX from USB using iBoot.

Installing a text editor

Because we're going to do it the hard way, a must have for our USB key is a text editor, so that we can tune the USB key to our needs during the installation process (more about this in the next post). As a UNIX veteran, I swear by vi, but I do understand that not everybody is confortable with vi sequences, so to alleviate the editing process, we'll go with the more intuitive GNU nano.

One might wonder why there is no default editor on the vanilla OSX installation media, but the answer is that Apple obviously considered an installer to be read-only affair, so it made little sense adding vi or nano to the installation utilities. On the other hand, we will very much require the ability to edit files on USB, so we need a text editor for OSX. Of course, since GNU nano is GPL and therefore, anybody can legally publish a nano OSX binary to copy onto an USB key, I'm not going think twice about putting the one I conveniently recompiled from source (using Xcode 3.2) and that I am putting online here.

Once downloaded, just drag and drop the "nano" file onto your USB stick folder (where you'll see the OSX installation paraphernalia). As long as it's on the key, we're good - no need to worry about placing it into bin/ or usr/bin/ just yet. Once it's copied, you can eject the USB, as at this stage, we won't need to access another OSX system: we'll just do everything we need from USB.

iBoot

Finally, the last piece we need for the installation is the latest iBoot CD (bugmenot). And while on the iBoot download page, you might as well download MultiBeast, as it will come handy to finalize our installation.

The iBoot zip basically contains an ISO which you should burn to a CD. Note that if you can't use a CD, you should be able to use that ISO image to produce an alternate iBoot loader (eg use a second USB key or TFTP), but I'm not going to detail that.

Resync'ing a RAID 1 array

Can come handy when attempting to correct unreadable sectors on a RAID disk. Eg. if your RAID1 array is built around /dev/sda3 and /dev/sdb3 and you had unreadable sectors reported by SMART on /dev/sda (Current_Pending_Sector), but got no error during the SMART extended self test, you might want to resync the /dev/sda partitions on /dev/sdb as follows:

mdadm --fail /dev/md2 /dev/sda3
mdadm --remove /dev/md2
/dev/sda3
mdadm --add /dev/md2 /dev/sda3

Why does windows find the need to modify the partition table at boot...

...or how I almost lost 1.5 TB of data today.

If you use software/fake RAID, this might happen to you too.

See, this morning I thought I would upgrade my HTPC's BIOS just for the heck of it. This is the PC that has a 2TB RAID5 array based on nVIDIA's MediaShield.
Now, despite what I had selected during the BIOS update, the BIOS settings and DMI data were reset after reboot, which means that the HDDs were back to individual IDE emulated drives, rather than members of the RAID array.

Normally, this wouldn't be a big deal, except that, before I cancelled the Windows boot, it was apparently able to look at the disks (using the MediaShield driver), find out that the capacity of the disk it was booting from (now a single 1TB IDE/AHCI HDD) was less than the capacity reported in the partition table, and re-write the partition table of HDD1 to reduce the dimensions of the last partition.

Of course, re-writting a partition table without anybody asking you to is the shortest way to screw up a disk or RAID array, and screw up it did: As soon as I restored the RAID settings in the BIOS and booted Windows, my 1.5 TB data partition was now identified as unformatted and gone! Talk about massive data loss...

No respectable O/S should ever modify a partition table without asking the user first. It's just common sense: The O/S is never, and I have to stress out that part, NOT EVER, smarter than its user (no matter what the O/S developers might think, or how smart they think they are themselves). You do not modify a partition table without asking, EVER, it's really simple as that!

Now, after much cursing, and some accidental good luck, I found that if the first drive was disconnected from the RAID5 array (which happened accidentally as I was trying to invert HDD#2 and HDD#3, since it originally looked like the BIOS upgrade has modified the SATA IDs), the rest of the array booted fine, albeit in degraded mode, and saw the old 1.5 TB data partition alright. Definitely makes sense with the fact that Windows would of course only have modified the partition table of the boot drive while the HDDs were in IDE mode.
But of course, as soon as you remove one drive from your RAID5 array, and boot in degraded mode, the array will flag that drive as failed on next reboot

From there on, the solution is to re-add the drive to the array to resync. Takes a while, but if you trust your other disks not to fail duing the super-lengthy re-sync, probably the safest solution.
Otherwise, it's probably a good idea to have a copy of the Master Boot Record (i.e. the first 512 bytes) of every single drive from your array, and restore it using a decent O/S like Linux. Plus, as experience will show you time and again, it's also always good practice to keep a copy of the MBRs of all your disks that contain important data, so that you can try to address any kind of partition formatting catastrophe.

A farewell to LILO (One bootloader to rule them all, part 2)

As mentioned previously, I set up a bunch of disks in a mixed RAID configuration on my new P5B Deluxe system, and merrily went on installing OS after OS.

The point of this exercise was to have my main OS (probably Vista 32) as well as a near half-a-terabyte partition set in RAID 0 mode, for speed, while some other stuff, for which disk performance is not that critical (OS-X, XP, Linux, more space), would reside quietly on a 3rd non raid disk. And while I'm at it, I would also setup a RAID 1 partition, as the Intel ICH8R "controller" of the P5B is now nice enough to allow you to create both a RAID0 and RAID1 drive out of the same 2 disks RAID array.

Which leaves us with the obvious problem of booting that lot, a capital problem indeed ( especially now that Vista is being such an ass about its boot loader). As an aside, I would like to point out that this is what happens when a company has a near monopoly with regards to installed OS base: They will obviously do their darnedest to prevent ANY other OS from eating even the smallest part of their market share. And I could comment on OS-X too. What is wrong with a standard partition scheme that you will recognize once you get your hand forced anyway (but not any sooner)? Why are you not happy with a big fat bootable primary partition with af type setup for you with Linux's fdisk?
And much much worse, how comes the partitions OS-X creates end up on the exact same block as the start of the next partition (have a look at your fdisk results in Linux - What the hell?)? I might talk about that later on, because this headache resulted in recently losing both a fresh install of XP along with a data partition and, what's worse, my Linux system. As the old proverb go, "Install XP, lose your data"...

So, considering that I have now done this operation twice, let me give you the long awaited heads up on the one bootloader to rule them all...


Accessing the ICH RAID disks under Linux

o Compile kernel with device mapper
o Pick up the latest devmapper package from ftp://sources.redhat.com/pub/dm/
o Pick and compile dmraid
o dmraid -ay (if dmraid complains about missing kernel options, add them!)
o dmraid -s (Yay!) or ls /dev/mapper

And while you're at it, you can add something similar to this at the end of your /etc/rc.d/rc.local file (Slackware)

echo mounting RAID Volumes
dmraid -ay
mount -v -t ntfs /dev/mapper/isw_fhbagdjeh_Twix1 /mnt/vista32
mount -v -t ntfs /dev/mapper/isw_fhbagdjeh_Twix2 /mnt/vista64
mount -v -t ntfs /dev/mapper/isw_fhbagdjeh_Twix3 /mnt/strider
mount -v -t ntfs /dev/mapper/isw_fhbagdjeh_Secure1 /mnt/secure

Becomes simple enough... when you've done that twice.


So long LILO, and thanks for all the flawless boots...

One word if you stubbornly try to get LILO to boot one of the RAID0 partition as I did: Agh!!!

Fatal: Sorry, don't know how to handle device 0xfd02 => compile with latest source, as it will compile with devmapper (device-mapper or LVM2) if library is present. Look for -DDEVMAPPER when compiling.
OK then, added devmapper option, make all, make install and...
"Fatal : device mapper : only linear boot device supported"

Enought with this. I'm sure the LILO devs will add it some day, but for now grub it is then.


Pub grub?

GNU's versions: WTF? Pick up 1.95 and... configure: error: LZO library version 1.02 or later is required
OK then, http://www.oberhumer.com/opensource/lzo/download/ to eventually realize: You should NOT grub 1.95. It's really grub 2.0 and too spanking new to be of any use. Instead, pick up grub 0.97 and live happily ever after!

Now, some of you might see some mention of a grub 0.97 dmraid patch, but because I'm actually using the 3rd non RAID disk (actually 1st one on my system) to install the bootloader, as well as ultimately boot, patching grub is unnecessary. I guess it would be necessary if we were to use our RAID0 as the boot disk. I'll have to try RAID5 one day, just for kicks :D

So, without further adieu to LILO, and with thanks to the Gentoo users:

grub --device-map=/dev/null
# My actual non-RAID boot disk
grub> device (hd0) /dev/sda
# The RAID0 array where the Vista's reside
grub> device (hd1) /dev/mapper/isw_fhbagdjeh_Twix
# Let's first add Linux, so that we can then edit the menu.lst
# My Linux is on /dev/sda4, hence hd0,3 with zero based indexes
grub> root (hd0,3)
Filesystem type is xfs, partition type 0x83
grub> setup (hd0,3)
Checking if "/boot/grub/stage1" exists... yes
Checking if "/boot/grub/stage2" exists... yes
Checking if "/boot/grub/xfs_stage1_5" exists... yes
Running "embed /boot/grub/xfs_stage1_5 (hd0,3)"... failed (this is not fatal)
Running "embed /boot/grub/xfs_stage1_5 (hd0,3)"... failed (this is not fatal)
Running "install /boot/grub/stage1 (hd0,3) /boot/grub/stage2 p /boot/grub/menu
.lst "... succeeded
Done.
grub> quit

At this stage, you wanna check your /boot/grub/device.map file. If you don't see the /dev/mapper entry for (hd1) as expected, you will have to add it manually.

From then on, you can go on creating your /boot/grub/menu.lst, which might eventually look something like this:

default 2
timeout 3

title  Slackware Linux 11.0
kernel (hd0,3)/boot/vmlinuz root=/dev/sda4

title  Slackware Linux 11.0 (Rescue)
kernel (hd0,3)/boot/vmlinuz.rescue root=/dev/sda4

title  Vista (32 bit)
rootnoverify (hd1,0)
chainloader +1

title  Vista (64 bit)
rootnoverify (hd1,1)
chainloader +1

title  Windows XP
rootnoverify (hd0,1)
chainloader +1

title  MacOS 10.4.8
rootnoverify (hd0,0)
chainloader +1

Parting words

And thus, I will be leaving with a "so long old Friend..." Since 1995, there haven't been much of a PC installed by yours truly which didn't display the unmistakable red greetings, and I had become such an expert at writing a /etc/lilo.conf, I could have done it blindfolded.
Still, I gotta give it to the GNU guys... Their solution to the booting problem is deceivingly elegant. I'm especially impressed with the "modify menu.lst on the fly" aspect of things and the fact that dmraid "just" works...