2009-02-05

And here we go again - RAID5 fakeraid + Linux

It's that time of the decade where I get a new machine, this time an nVidia MCP79 (a.k.a. GeForce 9400) based motherboard from Gigabyte. Oh, and yes, the MCP79 is the same all-inclusive nForce chipset nVidia uses in the new MacBook Pros, as well as their Atom "can of whoop-ass" for set top HDTV boxes.

As most people, I'm pleasantly surprised by the features of the chipset. My goal was to build an HTPC, and with one good durable mobo (with both HDMI and optical SPDIF), the adding of a 45 nm quad core and 4 GB dirt cheap DDR2 RAM, I should have a solution that will last for some time. Oh, and of course, since we're hopefully going to store a lot of media data there, we can't forget the 3x 1 TB Samsung SpinPoint HDDs in a RAID5 array, which is what brings us here today.

Yep, as you guessed from the title, it's ye olde getting Linux to work with fakeraid. And the first thing I gotta say is I'm sick and tired of all the posts you see on the net where people asking how to make Linux work with their fakeraid are met with a "why don't you use Linux's md instead".
Well, I'm sorry, but there are still people out there who want to dual boot with Vista and trust nVidia RAID5 driver (MediaShield) enough not to want to make a complete mess of their Windows installation. Hell, I'd rather spend time figuring out how to make my RAID5 work in Linux than have to spend time doing the same thing in Windows. Plus, I like the convenience of having my BIOS handle things like RAID.

Here we are then, with Vista installed and painfully updated at last (see below) and a blank partition on the nVidia RAID5 MediaShield array that's just dying to get Linux installed.

OK, it's not that I mind a challenge here, but I've got a blank HDD lying around, so this time, what we're going to do is install Linux on that standalone drive, conf it so that it sees the RAID5 partition, and then move over our system rather than have to create our own custom RAID5 installation CD.

As usual, I'm gonna use the latest Slackware (12.2) so ymmv.

1. Slackware install with ALL the options you really want in the end (we'll be using that partition as the source) and LILO booting the single HDD = piece of cake. Just make sure that you don't install LILO on anything else BUT the HDD that's not part of the array
2. First bad surprise - if you keep your 2.6 kernel as is, no matter what you try, dmraid will return:
ERROR: device-mapper target type "raid45" not in kernel
Don't waste your time changing the raid45 to raid456 identifiers in dmraid either - you'll still get the same error. The kernel actually needs to be patched for dmraid to be happy.

[UPDATED]One year later, and still a mess. As of 2.6.35, the dm-raid45 module still hadn't been integrated to the kernel, plus the evil people at Red Hat who were supposed to maintain that module have completely dropped the ball, the Gentoo people also have dropped the ball (after 2.6.31) and the last set of dm-raid45 patches I could only go to 2.6.33 (and don't work against 2.6.35). So I guess we're stuck with 2.6.33 then. This last set of dmraid patch can be found from Mandriva. Can't help but find it strange that, while the dm-raid45 module is commonly used in Ubuntu (except that dmraid itself is useless with large disks as it doesn't support GPT), nobody seems to care about trying to merge it with the official kernel tree. Oh well...
root@stella# cd /usr/src/linux-2.6.33/
root@stella# wget http://tmb.mine.nu/Mandriva/Cooker/dm-raid/dm-raid45_2.6.33-rc1-20091126.patch
root@stella# patch -p1 < dm-raid45_2.6.33-rc1-20091126.patch
root@stella# wget http://tmb.mine.nu/Mandriva/Cooker/dm-raid/dm-raid45-buildfix-for-2.6.33.patch
root@stella# patch -p1 < dm-raid45-buildfix-for-2.6.33.patch
OK, so you just go through your usual kernel recompile. Now, a couple of things I wanna point out:
- You do want to reduce the size of your kernel if you're planning to remove the standalone HD and still boot your RAID5 array Linux partition with a decent solution
- Of course you want to add Device mapper support (Device drivers -> Multiple devices driver support (RAID and LVM)) and the new experimental "RAID 4/5 target" that was added by the kernel patch
- Despite being a Gb adpater, the nForce Ethernet driver (forcedeth) is to be found in: Ethernet (10 or 100Mbit) -> EISA, VLB, PCI and on board controllers -> nForce Ethernet support
- if you're using ext4 for root, don't forget to include ext2 support as well, as your root partition cannot be mounted without it.

Once you have a kernel that boots, you can move on to the dmraid install. This time, no need for devmapper. Just download the current dmraid source from http://people.redhat.com/~heinzm/sw/dmraid/src/:
root@stella# cd /usr/src
root@stella# wget http://people.redhat.com/~heinzm/sw/dmraid/src/dmraid-current.tar.bz2
root@stella# tar -xjvf dmraid-current.tar.bz2
root@stella# cd dmraid/1.0.0.rc15
root@stella# ./configure; make; make install
root@stella# dmraid -ay
RAID set "nvidia_aeejfdbe" was activated
RAID set "nvidia_aeejfdbep1" was activated
RAID set "nvidia_aeejfdbep2" was activated
RAID set "nvidia_aeejfdbep3" was activated
Yay! it works!

If you only see see the RAW disk (nvidia_aeejfdbe) but not the partitions, that's probably because you're using a GPT disk. In that case, you need to also run kpartx as follows:
root@stella# kpartx -a -v /dev/mapper/nvidia_aeejfdbe
add map nvidia_aeejfdbe1 (252:1): 0 262144 linear /dev/mapper/nvidia_aeejfdbe 34
add map nvidia_aeejfdbe2 (252:2): 0 209715200 linear /dev/mapper/nvidia_aeejfdbe 264192
add map nvidia_aeejfdbe3 (252:3): 0 186478592 linear /dev/mapper/nvidia_aeejfdbe 209979392
add map nvidia_aeejfdbe4 (252:4): 0 11324626944 linear /dev/mapper/nvidia_aeejfdbe 396457984
kpartx itself can be obtained from the multipath-tools.

OK, so now we can get going. Be sure you format the RELEVANT partition. In my case, I'll be using nvidia_aeejfdbep2, the second partition on the RAID5 array for the GNU/Linux system.
root@stella# mkfs.ext4 /dev/mapper/nvidia_aeejfdbep2
root@stella# mount /dev/mapper/nvidia_aeejfdbep2 /mnt/hd
root@stella# mkdir /mnt/hd/proc
root@stella# mkdir /mnt/hd/sys
root@stella# cp -ax / /mnt/hd
This could take a while...
Note that the -x option ensures that we stay on a single file system

OK, in the next post we'll see the grub way to (kinda) boot this whole mess

2 comments:

  1. Thanks for the info; it got me going! It looks like this bug is at least 18 months old, and I still have to patch my kernel (gentoo) to get things working!

    ReplyDelete
  2. Oh wow, thanks for this. I am in a similar situation. The Fedora community is helping me figure out how to reassemble my raid from from a usb os. Enterprise set up.
    For all you computer scientists out there, if your in learning more about raid technology in Linux here is the link to a similar yet different raid environment.
    https://forums.fedoraforum.org/showthread.php?320473-dd-error-writing-dev-sdc2-No-space-left-on-device/page4

    ReplyDelete

Note: only a member of this blog may post a comment.