Setting up a cross-compiler for the SheevaPlug on Linux x86_64
While it is definitely possible to recompile a kernel on the SheevaPlug itself, let us just consider the raw compilation times observed when compiling a 2.6.31.2 kernel (exact same .config) on the Sheeva and on a more up to date PC:
- Quad Core PC (Slackware 13.0 x86_64 w/ ext4 fs on a RAID5 array) - 3:03 mins (183 secs)
- SheevaPlug (Ubuntu 9.04 w/ ext4 fs on a high speed SDIO card) - 57:42 mins (3462 secs)
That's one full order of magnitude right there (or about 20x slower)! And if you need a further case for cross-compilation, remember that extracting the kernel source takes ALOT of space. You will run out of space if you try to extract it on the internal flash NAND - it's just too big for what's left of the 512 MB space, and even on external media, you might want to think twice about using the limited space you have there.
If you're familiar with this blog, you know that there's no way we'll want to use any of the old, likely outdated, tools that came with the SheevaPlug SDK CD or internet binaries got compiled by people we don't know (and therefore don't trust!). Besides, the Cross compiler tgz I got on the CD has a big fat CRC error to start with, and it's 32 bit while our cross compiling rig is running Linux x86_64. If what we've seen with the 32 bit openocd binaries is any indication of what's in store for 64 bit users, we might as well cut to the chase and ignore anything that's not 64 bit.
What we do want, and always will want, is the most cutting edge and up to date technology we can lay our hands on. Therefore, we will of course recompile our own cross compiler for the SheevaPlug using the latest gcc, glibc and binutils.
Now, we could of course dive in straight ahead, and recompile, by hand, the whole cross-compiling shebang, but let's face it: your time is precious, so is mine, and if you just dive in head first in trying to produce a cross compiler from the GNU vanilla tarballs, whilst having no previous experience in the matter, you'll find soon enough that:
- Compiling a cross compiler manually is absolute hell!
- glibc and gcc have to be the worst offenders for breaking things that worked perfectly fine before with newer versions. All those rumours you've heard about needing to apply a massive amount of patches to gcc and gblic before you can compile them as cross-compilers? They're true!
Now, if you really, really, insist on compiling a whole toolchain manually, then my advice is to try to stick as close as possible to this guide (trust me!), but don't come whining that I didn't warn you you were about to step through the gates of hell beforehand...
Using CrossTool-NG
The MUCH better approach, to automate the cross compilation setup process is to use Yann E. Morin's most excellent Crosstool-NG. And, yes, despite being an automated tool, it will not compromise on our using the latest versions of gcc, glibc, binutils, etc.
So off we go:
cd /usr/srcWell, isn't that nice - a kernel like configuration menu. Alrighty then, here are the options you want to change for the SheevaPlug:
mkdir sheeva
cd sheeva
wget http://ymorin.is-a-geek.org/download/crosstool-ng/crosstool-ng-1.4.2.tar.bz2
tar -xjvf crosstool-ng-1.4.2.tar.bz2
cd crosstool-ng-1.4.2
./configure
make
make install
cd ..
ct-ng menuconfig
- Paths and misc options
- Try features marked as EXPERIMENTAL: check
- Prefix directory: "/usr/local/${CT_TARGET}" (this will result in all the tools residing in /usr/local/arm-unknown-linux-gnueabi) or, preferred: "/usr/local/sheeva". One important note though: DO NOT USE /usr/local or any directory containing data you plan to keep. THIS DIRECTORY WILL BE ERASED. You have been warned.
- Use LAN mirror: Let's try to use a mirror there, so - check
- Prefer the mirror: check
- Base URL: ftp://ftp.heanet.ie/mirrors/ftp.gnu.org/gnu/ (this is an Irish mirror - not entirely sure it works, as the downloads didn't seem much faster, but worth a try)
- Stop after extracting tarballs: check (we'll need to patch glibc!)
- Number of parallel jobs: 2x the number of cores you have. In my case that would be 8 (quad core)
- Maximum log level to see: change from "INFO" to "EXTRA"
- Target Options
- Target Architecture: arm
- Use EABI: check
- Emit assembly for CPU: arm926ej-s
- Floating point: change from "hardware (FPU)" to "software"
- Toolchain options: don't change anything
- Operating System:
- Target OS: change from "bare-metal" to "linux"
- Get kernel headers from: change from "kernel's 'headers_install'" to "Use custom headers"
- Path to custom headers directory/tarball: "/usr/src/sheeva"
- GMP and MPFR:
- GMP and MPFR: check
- GMP version: 4.2.4 (latest)
- MPFR version: 2.4.1 (latest)
- binutils
- binutils version: 2.19.51.0.2
- C compiler:
- gcc version: 4.3.3
- C++: check
- C-library:
- C library: glibc
- glibc version: 2.9
- Threading implementation to use: make sure "ntpl" is set.
Kernel headers & glibc 2.9 patch
Before you can run the build, we still need to setup our kernel headers.
At the time of this writing, the most recent kernel is linux-2.6.31.2, so just download and extract it somewhere. Then, within the linux-2.6.31.2/ directory, run the command:
make headers_install ARCH=arm INSTALL_HDR_PATH=/usr/src/sheevaThe INSTALL_HDR_PATH should match exactly what you used for "Path to custom headers directory/tarball" in the menuconfig of Crosstool-NG.
You should now find that you have an "include" directory containing the relevant kernel headers in /usr/src/sheeva
Once you have everything setup, you need to go back to the sheeva directory (because this is where the ct-ng config resides) and launch the build with:
cd /usr/src/sheevaIf you followed what we did above, you will note that we checked the "Stop after extracting tarballs" option, and indeed the build process stops after "Extracting and patching toolchain components". The reason we did that is because glibc 2.9 needs to be manually patched to prevent the following error (undefined reference to `_begin') from occurring:
ct-ng build
[ALL ] /usr/src/sheeva/targets/arm-unknown-linux-gnueabi/build/build-libc/elf/librtld.os: In function `_dl_start_final':This known problem is documented here and if you have a look at the diff file, you'll see that we can easily manually patch this problem by editing "targets/src/glibc-cvs-2.9/elf/Makefile" and replace, on line 314:
[ALL ] raise.c:(.text+0x54c): undefined reference to `_begin'
[ALL ] /usr/src/sheeva/targets/arm-unknown-linux-gnueabi/build/gcc-core-shared/lib/gcc/arm-unknown-linux-gnueabi/4.3.3/../../../../arm-unknown-linux-gnueabi/bin/ld: /usr/src/sheeva/targets/arm-unknown-linux-gnueabi/build/build-libc/elf/ld.so: hidden symbol `_begin' isn't defined
[ALL ] /usr/src/sheeva/targets/arm-unknown-linux-gnueabi/build/gcc-core-shared/lib/gcc/arm-unknown-linux-gnueabi/4.3.3/../../../../arm-unknown-linux-gnueabi/bin/ld: final link failed: Nonrepresentable section on output
[ALL ] collect2: ld returned 1 exit status
(...)
[ERROR] Build failed in step 'Installing C library'
-e 's/\. = 0 + SIZEOF_HEADERS;/& _begin = . - SIZEOF_HEADERS;/' \with
-e 's/\. = .* + SIZEOF_HEADERS;/& _begin = . - SIZEOF_HEADERS;/' \
Building the cross-compiler
Once you have made this change, just launch ct-ng menuconfig again (from the /usr/src/sheeva directory, as this is where the config file resides) and remove the checkmark on "Stop after extracting tarballs" (in the "Paths and misc options" menu)
Now, you can launch the actual build of the tools with
ct-ng buildThis will take a little while (about 15 mins on my machine) but should complete successfully. If you have a multiple core system, hopefully you didn't forget to change the "Number of parallel jobs" option (which is equivalent to the -j option of make, i.e. number of concurrent jobs to run at once), as this dramatically reduces the time it takes to compile the toolchain. The -j option should always be your friend when compiling a large source, like a Linux kernel for instance.
Time for a tea or coffee break!
Note that, near the end of the compilation, you might get the error "[ERROR] libtool.m4: error: problem compiling FC test program". From what I gather however, FC is a test for the Fortran 90 compiler, which we couldn't care less about, so, unless you got another error, I will consider this build process of success, and you should now have the whole set of cross compiler tools in /usr/local/sheeva/bin. Time to compile ourselves a new kernel!
Installing U-boot mkimage
While we now have the tools to create arm binaries, we're still missing the last piece of the puzzle to boot a kernel, and that is the mkimage utility (which is called at the end of a successful kernel compilation), which turns a newly crafted kernel into something U-Boot can actually launch.
You guessed it, no way in hell we're gonna use the Marvell provided mkimage tool - we're gonna recompile our own from source of course!
And once more, the good guys at Debian (never underestimate a Debianer!) provide us with what we need:
cd /usr/src/sheevaEasy as a pie!
wget http://ftp.de.debian.org/debian/pool/main/u/uboot-mkimage/uboot-mkimage_0.4.tar.gz
tar -xzvf uboot-mkimage_0.4.tar.gz
make
make install
Cross-Compiling a new kernel for the SheevaPlug
At long last, we're ready to compile us some custom kernel!
At this stage, we can more or less follow the Compiling Linux Kernel for the Plug Computer tutorial from openplug.org . Henceforth:
cd /usr/src/sheeva/linux-2.6.31.2There, change the options that you want. If you plan to keep using UBIFS (which you should have switched to during the SheevaPlug Installer 1.0 upgrade), then make sure that you enable BOTH:
make ARCH=arm kirkwood_defconfig
make ARCH=arm menuconfig
- Device Drivers ---> Memory Technology Device (MTD) support ---> UBI - Unsorted block images ---> Enable UBI
- File systems ---> Miscellaneous filesystems ---> UBIFS file system support
- Device Drivers ---> Generic Driver Options, and make sure that both "Create a kernel maintained /dev tmpfs (EXPERIMENTAL)" and "Automount devtmpfs at /dev" are enabled. Without those, you won't see init messages on the serial console after the root filesystem is mounted.
make -j8 ARCH=arm CROSS_COMPILE=/usr/local/sheeva/bin/arm-unknown-linux-gnueabi- uImageand in about 3 minutes (vs. one hour if done on the SheevaPlug) you should end up with something like
Image Name: Linux-2.6.31.2
Created: Tue Oct 6 14:50:51 2009
Image Type: ARM Linux Kernel Image (uncompressed)
Data Size: 3151992 Bytes = 3078.12 kB = 3.01 MB
Load Address: 0x00008000
Entry Point: 0x00008000
Image arch/arm/boot/uImage is ready
Building the modules
You probably still have a few modules to compile as well, which you can do as follows
make -j8 ARCH=arm CROSS_COMPILE=/usr/local/sheeva/bin/arm-unknown-linux-gnueabi- modulesJust don't forget that you'll need to copy the content of the ./modules directory to your Ubuntu image to keep the kernel happy
make -j8 ARCH=arm CROSS_COMPILE=/usr/local/sheeva/bin/arm-unknown-linux-gnueabi- INSTALL_MOD_PATH=./modules modules_install
Testing the new kernel on the plug
In a next post, we'll see how to actually flash that kernel on the plug. Before we do that though, we want to ensure that it can boot our current system, so we'll try to launch it from an USB stick.
Off we go then to copy your shiny new kernel to an USB stick with
mount /dev/sde1 /mnt/usbFor the test part, we'll need the serial console ("screen /dev/ttyUSB0 115200,-crtscts" or similar), with which we first reboot our system, and interrupt the autoboot to get to the Marvel >> prompt. From there, the following set of commands should boot the existing system with our new kernel:
cp arch/arm/boot/uImage /mnt/usb/
sync
umount /mnt/usb
usb startSuccess? Then my work here is done.
fatload usb 0 0x00800000 uImage
bootm 0x00800000
