OpenOCD and the SheevaPlug on Linux x86_64

This is a tutorial for people like me who want to start with JTAG and OpenOCD access on the SheevaPlug. Most of this stuff regarding the installation of OpenOCD on Linux x86_64 is of course generic and can be used for other FTDI based JTAG devices like.

First of all, if you're running a 64 bit OS (Vista, Win7), a word of advice: Don't waste your time trying to get OpenOCD to work on a 64 bit Windows - there's no ready-made package that works (the 32 bit version just don't), so the only way is to recompile the whole shebang yourself, and it's a complete mess, as you will also need a 64 bit gcc-like compiler, and figure out how to get a 64 bit version of a the FTDI GPL library talk to the 64 bit libUSB drivers.
If I ever manage to get a Windows version going, you'll be the first to hear about it, but for the time being, if you value your time, stick with Linux.

Now, the distro I am using is Slackware x86_64. There are of course tutorials on how to compile OpenOCD for Linux here and there, but things are slightly different for 64 bit version, which is why I'm posting my own version.

1. Before anything else, you want to have the serial console to the SheevaPlug through the USB FTDI device. For that you need to compile the kernel driver for the "USB FTDI Single Port Serial Driver" either as a module, or directly in your kernel. You will find that option, in the kernel, under: "Device Drivers ---> USB support ---> USB Serial Converter support ---> USB FTDI Single Port Serial Driver"
   Once you have the driver loaded, with the USB cable connected to the sheevaplug, you should get a /dev/ttyUSB0 device, which you can use as a regular serial port.
   The best way to connect to the SheevaPlug in Linux (or to any serial terminal for that matter) has to be to use screen, with a command like:
   

   screen /dev/ttyUSB0 115200,-crtscts

   Once you have confirmed that you can connect to the plug using the FTDI USB Serial converter driver, you can move to building OpenOCD, as it's a good indication that Linux should have no trouble accessing the rest of the FTDI chip functionalities


2. • OPTION 1: Using the proprietary libftd2xx library from FTDI, provided for completion. Note that using this option will kill any opened serial console whenever you launch openocd, whereas this does not occur with OPTION 2 below, which I strongly advise you to use.
     On an x86_64 MUST pick up the 64 bit version of the Linux drivers from the FTDI website. At the time of this post, that means you must download libftd2xx0.4.16_x86_64.tar.gz, NOT libftd2xx0.4.16.tar.gz. What's more, when you install the library, it has to go into lib64, NOT lib else you will get the ominous
     

     checking whether ftd2xx library works...
     configure: error: Cannot build & run test program using ftd2xx.lib

     Thus:
     

     cd /usr/src
     wget http://www.ftdichip.com/Drivers/D2XX/Linux/libftd2xx0.4.16_x86_64.tar.gz
     tar -xzvf libftd2xx0.4.16_x86_64.tar.gz
     cp libftd2xx0.4.16_x86_64/libftd2xx.so.0.4.16 /usr/local/lib64
     ln -s /usr/local/lib64/libftd2xx.so.0.4.16 /usr/local/lib64/libftd2xx.so
     # even if you specify --libdir=/usr/local/lib64 on openocd compilation
     # it requires libftd2xx.so.0 in /usr/lib64!
     ln -s /usr/local/lib64/libftd2xx.so.0.4.16 /usr/lib64/libftd2xx.so.0

   • OPTION 2 (PREFERRED): Using the GPL/Open source version of the FTDI library

     cd /usr/src
     wget http://www.intra2net.com/en/developer/libftdi/download/libftdi-0.16.tar.gz
     tar -xzvf libftdi-0.16.tar.gz
     cd libftdi-0.16
     # by default, libftdi will go in /usr/local/lib, whereas, on x86_64, it should go to lib64, thus
     ./configure --with--libdir=/usr/local/lib64
     make
     make install
     # openocd looks in /usr/lib64
     ln -s /usr/local/lib64/libftdi.so.1.16.0 /usr/lib64/libftdi.so.1

3. Now it's time to get OpenOCD compiled. For that:
   

   svn checkout svn://svn.berlios.de/openocd/trunk openocd
   cd openocd
   ./bootstrap
   # OPTION 1 - PROPRIETARY LIB FROM FTDI
   ./configure --libdir=/usr/local/lib64 --enable-maintainer-mode --enable-ft2232_ftd2xx
   # OPTION 2 - GPL FTDI LIBRARY
   ./configure --libdir=/usr/local/lib64 --enable-maintainer-mode --enable-ft2232_libftdi
   make
   make install

   Note that if you don't use the "--enable-maintainer-mode" option, you will get an error when make reaches the creation of the documentation, which is not a problem per se, but will prevent the openocd configuration scripts to be copied over on make install
   
   
4. If the above completed successfully, then you have openOCD ready to run. To connect to the SheevPlug then, issue a:
   

   openocd -f /usr/local/share/openocd/scripts/board/sheevaplug.cfg

   At this stage, if you are getting the following, then you need to reboot the plug and launch openOCD in the early stages of reboot:
   

   Error: JTAG scan chain interrogation failed: all zeroes
   Error: Check JTAG interface, timings, target power, etc.
   Error: Trying to use configured scan chain anyway...
   Error: feroceon.cpu: IR capture error; saw 0x00 not 0x..1
   Warn : Errors during IR capture, continuing anyway...
   Error: unexpected Feroceon EICE version signature

   What you really want to see when launching openOCD is:
   

   root@stella:/usr/src/openocd# openocd -f /usr/local/share/openocd/scripts/board/sheevaplug.cfg
   Open On-Chip Debugger 0.3.0-in-development (2009-10-07-01:35) svn:2808
   $URL: svn://svn.berlios.de/openocd/trunk/src/openocd.c $
   For bug reports, read http://svn.berlios.de/svnroot/repos/openocd/trunk/BUGS
   2000 kHz
   jtag_nsrst_delay: 200
   jtag_ntrst_delay: 200
   dcc downloads are enabled
   Warn : use 'feroceon.cpu' as target identifier, not '0'
   Info : clock speed 2000 kHz
   Info : JTAG tap: feroceon.cpu tap/device found: 0x20a023d3 (mfg: 0x1e9, part: 0x0a02, ver: 0x2)

5. OpenOCD is a client server software, so when you launch openOCD, you are really only launching the server part. To actually send JTAG commands and all that Jazz, you need to open a client session with:
   

   telnet localhost 4444

   This should then provide you with the On-Chip Debugger prompt.
   
   
6. Now, JTAG operations are a bit tricky (there are various JTAG states involved) so you can't really send any JTAG command you want (eg. NAND ID) any time you like and expect a meaningful reply. If you looked in the cfg file for the SheevaPlug, you'll see that before any NAND access operation is performed, a "sheevaplug_init" command is being called, so this is what we'll do. Once it's run, you will get a whole lot of interesting commands working, as illustrated with the session below:
   

   > sheevaplug_init
   target state: halted
   target halted in ARM state due to debug-request, current mode: Supervisor
   cpsr: 0x000000d3 pc: 0xffff0000
   MMU: disabled, D-Cache: disabled, I-Cache: disabled
   0 0 1 0: 00052078
   > scan_chain
        TapName            | Enabled |   IdCode      Expected    IrLen IrCap  IrMask Instr
   ---|--------------------|---------|------------|------------|------|------|------|---------
    0 | feroceon.cpu       |    Y    | 0x20a023d3 | 0x20a023d3 | 0x04 | 0x01 | 0x0f | 0x0c
   > nand probe 0
   NAND flash device 'NAND 512MiB 3,3V 8-bit' found
   > nand list
   #0: NAND 512MiB 3,3V 8-bit (Hynix) pagesize: 2048, buswidth: 8,
       blocksize: 131072, blocks: 4096
   > nand info 0
   #0: NAND 512MiB 3,3V 8-bit (Hynix) pagesize: 2048, buswidth: 8, erasesize: 131072
   #0: 0x00000000 (128kB) erase state unknown (block condition unknown)
   #1: 0x00020000 (128kB) erase state unknown (block condition unknown)
   (...)
   #4094: 0x1ffc0000 (128kB) erase state unknown (block condition unknown)
   #4095: 0x1ffe0000 (128kB) erase state unknown (block condition unknown)

Accessing an SD card on the SheevaPlug

Not something I knew beforehand.
USB devices are either auto detected or can be accessed as regular SCSI Disk devices (/dev/sda# - by the way, the SheevaPlug has no problem powering a 7200 rpm 2.5" HDD [UPDATE: until you try it for a few days, and its PSU blows up), but not SD Cards.

The device name to use then is: /dev/mmcblk0 for the raw device and /dev/mmcblk0p1 for the first partition.

Watching the Daily Show if you're not in the US

Well, it finally happened. Daily video streaming for the last episode of the "The Daily Show" is no longer available from the Republic Of Ireland (was working fine 'till yesterday).

The solution:

• Get a VPS, private server, whatever box in the US with enough bandwidth and an ssh server
  
• Using putty add the following tunnel:
  - source port: 80
  - Destination: media.mtvnservices.com:80
• Edit your C:\Windows\System32\drivers\etc\hosts file and add the line:
  127.0.0.1 media.mtvnservices.com
• Establish an ssh connection, and then open a browser to www.thedailyshow.com

"No soup for you"? Yeah, I'd like to see you try!

SheevaPlug - Let's get cracking!

Alright, now that we have our new shiny SheevaPlug, let's get started.

1. Of course, first thing we want is a serial console to the Linux OS, so we'll plug the mini <-> std USB cable provided to one of our USB ports and find out if Vista (x64) is able to figure out the FT2232D driver itself... which it can't (surprise, surprise). No big deal. The PDF readme from the DevKit CD tells you that you can find the TeraTerm Windows drivers in the SheevaPlug_Host_SWsupportPackageWindowsHost.zip or, for a more up to date version, at http://www.ftdichip.com/Drivers/D2XX.htm. Installing the drivers with Vista autodetect is still a pain, as you don't get the option to browse to the driver directory (yes, Microsoft, all your users are complete tools - that's why they use your products in the first place!), but with the device manager, you're good to go. Yay, another COM port! Yay a set of USB Serial converters!
   
   
2. Alrighty, if we have our COM port, so we're good to go with the ever versatile putty. The FIRST thing you want to know is that the serial console is set to 115200 bauds, and the SECOND thing is that the default logon is root/nosoup4u (Globalscale: even their passwords are cool!). And we're in... Niiiiiice! dmesg, df -h, top, they're all waiting for you, but first you might as well wanna change the date & time, as mine was set to Apr 22 1953. That's like, way old! Also, since you're gonna wanna try a reboot to see what the console spills out, I'll just mention that there's a very loooong pause during boot after the line "eth0: link up, full duplex, speed 1 Gbps", so just be patient, you will get your logon prompt. Oh and you'll see warnings here and there as well, but what do you expect.
   
   
3. What do we have here? Ubuntu? Huh, well, at least it's Debian based and we have a good chunk of space left. We could have fared much much worse (read anything that's based on Red Hat). We'll install armedslack soon enough but let's get cracking then. Of course, we'll want to recompile stuff on the device itself (something that can't be done on a WRT!). Let's start by compiling the very handy memtester to do a little bit of torture test, and see how much the baby heats up when using RAM.
   
   
4. According to google, installing gcc on the SheevaPlug ios as easy as running: "apt-get install build-essential"... Except that you are missing the cache directory so the command fails. You need to issue a
   

   mkdir -p /var/cache/apt/archives/partial

   before you can use apt-get (and you might want to add this line to your /etc/rc.local as well). But we're still not out of the woods yet, as the package upgrade seems to start up fine, but after a while, you get errors like:

   Failed to fetch http://ports.ubuntu.com/pool/main/b/binutils/binutils_2.19.0.20090110-0ubuntu1_armel.deb  404 Not Found

   Checking that URL confirms that the package is no longer hosted there.
   Drats!
   
   
5. The easiest solution at this stage is to update your system as a whole with "apt-get update". This might take a while, but it seemed to get the problem sorted, as the subsequent "apt-get install build-essential" completed hapilly. You might also want to run an "apt-get autoremove" after that, to free up some space. Good, now we can compile natively on the plug.
   
   
6. "-bash: wget: command not found"? Who on earth delivers a system without wget? "apt-get install wget" then, but sheesh! But then...

   root@debian:/usr/src# wget http://pyropus.ca/software/memtester/old-versions/memtester-4.1.2.tar.gz
   --2009-10-02 14:05:28--  http://pyropus.ca/software/memtester/old-versions/memtester-4.1.2.tar.gz
   Resolving pyropus.ca... failed: Name or service not known.
   wget: unable to resolve host address `pyropus.ca'

   Indeed, DNS resolution doesn't seem to work out of the box and the default /etc/resolv.conf goes something like:

   domain lan
   search lan
   nameserver 127.0.0.1

   It's clear that with only these 3 lines, you're not gonna get very far (it's a wonder apt-get still managed to find its way - must be using direct IPs). Let's just comment the domain & search lines and add our ISP's nameservers before the 127.0.0.1 to get some resolution going. Now, please be aware that any changes you make here won't hold on reboot, so for more details on how to fix the default annoyance, you might as well have a look at the New Plugger Howto from openplug.org.
   
   
7. Good, now we can wget memtester, compile it and run it (and it looks like our RAM is good). Maybe I'll try mprime later on as well... But to complete the setup of a half decent Linux system, let's issue an "apt-get install screen.

Next, we'll try to play with JTAG and OpenOCD...

SheevaPlug - first impressions

At last, my SheevaPlug Development Kit has arrived!

For those who don't know yet what the SheevaPlug is, you can have a look at this short CNET article or Marvell's slightly longer blurb about this wonderful little device.
I ordered mine (UK version) from Globalscale about 2-3 weeks ago, to basically replace my long trusted WRT54G running openWRT with something that has a little more punch and I/O capabilities. The truth of the matter is, I almost never use the WRT for WiFi, but having an always on & completely customizable Linux low power "server" has proved invaluable (Samba share for easy data interchange between any machine, DHCP server, bootp/tftp server, nmap & other networking tools provider, etc.), and I had been waiting for some time to see something like Marvell's plug computer come out, as it's basically a glorified hacked WRT54G, without the unneeded Wifi/network hub features.

For those who can't be bothered to read the whole post (and who can blame them!), I'll jump right into the main points of my first impressions:

The not so good:

• Globascale's delays in shipment, and total lack of response when asked for updates. Granted delays are likely unavoidable (I doubt they're shipping Sheevaplugs by the truckload right now), but that doesn't explain why it took about 2 weeks of repeated e-mails to sales to get a simple status update on the order.
• No choice but to use Fedex for shipping if you're not in the US. Granted, once it's shipped, the delivery is fast indeed, but considering the shipping delay, you might as well want to use regular post and spare a few bucks.
  Also, since Fedex are really happy to help with the obnoxious EU VAT & Customs Duty extortion scheme, you will be hit by 46% retroactive VAT on import (€31.20 - I really wish I was making this up), meaning that, as usual, no matter what you do, amount in $ = amount in €, and you still have to pay €99 for a bloody $99 item. Better believe this is the last time I import anything with Fedex.
  HEED MY ADVICE - DO NOT IMPORT ANYTHING WITH FEDEX, EVER!!!
  
• Unlike the most excellent US removable plug (see below), the UK plug is just a common adapter to fit over the US socket, so it's not as sleekly designed. Not that big a deal, but it adds to the overall height of the device when used in plug mode. It would have been nice to see an integrated UK plug in the way the US one is. Also, the Globalscale UK devkit photos don't make that as clear as they could.
  
• The mainboard is now rev. 1.3 and very different from the reference mainboard you see in the documentation (see below). For starters, there is no longer a separate mainboard for JTAG and SDIO, and the solder-able port for an ARM JTAG connector is gone. Gone as well is the generic GPIO connector. There still seems to be a small JTAG port, but with only 8 connectors and a non-standard smaller socket too be fitted in, it will be a pain to interface with. Not sure if many people would put single board vs 2 separate boards design as a disadvantage, but I was kind of planning on using the standalone expansion board as a cheap JTAG device (it has the very well known FT2232 JTAG IC on it) but now using the SheevaPlug hardware as a JTAG interface to other boards, or as a general purpose IO device, is going to be less of a possibility for your regular electronics hobbyist.
  UPDATE: for some of the new ports pinouts (still no JTAG desc), have a look here
  
• Even idle, the device seems to run a bit hotter than I anticipated (It's certainly running hotter than my WRT54G). Definitely not that hot (akin to one of the numerous power bricks you have in your house), and most likely the heat is due to the PSU rather than the chip themselves, but still, I've seen power bricks run cooler than that (the 12V DC WRT brick being one of those). Of course, anybody who hears "heat" automatically thinks "increased power consumption", so I don't know how that translates into wasted wattage. My guess is this might be due to using a 110V/220V compatible PSU. Using a localized source voltage PSU for 220V -> 5V delivery would probably reduce the heat.
  [UPDATE 2009.10.08] The Plug does run scaringly hot when running at full load for a prolonged period of time, as I just found out through a rogue process.
  The metal parts on both USB and network sockets are almost burning when running for a long time at 100% of the CPU, and you wouldn't want to leave something in contact with them then. Still probably not hot enough to light up paper or plastic that would be in contact, yet quite scary... And it looks like the PSU unit does play some part in raising the whole temperature of the plug on full load. I also found that I had to enlarge some of the vents on my plug with a knife because they had been pourly moulded and were not entirely open. Still, I don't think Even then, there are too few vents on the current Plug and I don't think it is designed for adequate cooling! Something that definitely needs some improvement for later models.
  My advice: make sure you leave ample space around the plug, and check that all your vents are fully open.
  

The good:

• Brilliant design from Globalscale. As somebody else said, whoever designed that removable socket for corded/plugged needs to have both a raise and a corner office. And despite the point I made above, the single mainboard layout is very well done (and rumour has it there is even an eSata port in there). Those guys have sharp engineers! The whole device itself feels both sturdy, inconspicuous, yet elegant. Even the cardboard packaging the plug is shipped in is very good design, so much so that I almost felt like taking pictures of the packaging. But then I thought, only Apple fanboys ever do something like that...
• Does exactly what it says on the tin. The layout design might have changed, but you still get 2 USB ports (1 for debug), 1 SDIO port, 1 Gb Ethernet port and a computer with 512 MB RAM, 512 MB Flash, a most excellent 1.2 GHz ARM CPU, plus a JTAG device that's compatible with OpenOCD. Who could ask for more?
• Can power a 2.5" USB HDD (tested with both a 5400 rpm 100 GB and a 7200 rpm 320 GB HDD). Impressive!
  
• Loads of free space on the flash with the default Ubuntu installation
  

Alright, now it's time for opening the device and posting some pictures! Wait, are you telling me that the first thing you do when receiving a cool new toy is NOT tearing it apart to see what's inside? What on earth are you doing with your life?

Some pictures

• That very well done removable socket/cord plug concealer

From SheevaPlug

• Opening the plug (just remove the rubber pads to find the screws)
  

From SheevaPlug

• Mainboard - top

From SheevaPlug

• Mainboard - underside, with heatsink

From SheevaPlug

• Mainboard - underside, without heatsink

From SheevaPlug

• The device in action

From SheevaPlug

LILO Warning: The initial RAM disk is too big

Just re-installed Slackware 13.0 x64 on the HTPC, this time using a small boot partition on the extra non fakeRAIDed HDD, which I recently added to the whole rig. This 4th HDD is now the boot disk for the whole rig, and contains the kernel and initrd (while the Linux partition itself is on the RAID5 set).

Install went without much of a hitch, and even the first few kernel recompilations we fine, but lately, after adding more kernel features, I got the following LILO warning and the kernel simply doesn't boot:

    Warning: The initial RAM disk is too big to fit between the kernel and
   the 15M-16M memory hole.  It will be loaded in the highest memory as
   though the configuration file specified "large-memory" and it will
   be assumed that the BIOS supports memory moves above 16M.

So much for checking the "Optimizing for size" kernel option.

The root of the matter seems to be that uncompressed kernel + uncompressed initrd must fit within the 0-15 MB low memory addresses, and it looks like a bzipped 4.5 MB kernel + a gzipped 1.4 MB initrd don't cut it, regardless of whether your BIOS has set a hole for the 15M-16M range.

Unlike what you might read elsewhere ("Remove options from your kernel to reduce its size! / Why are you using such a huge kernel?" - how's that helpful, guys?), there is a solution right there in the kernel conf, through the "General Setup -> Initramfs source file(s)" option.
Somehow when you do that, the kernel is able to find all the RAM it needs for the initrd.

Now, I'll grant you that this makes it less convenient to modify your initrd options on the fly, yet I believe this is better than being restricted in your choice of kernel features. Plus it appears that the kernel boots a little bit faster when using the emebedded initrd.

Of course, that means that you need to create your uncompressed cpio archive from your initrd-tree before you can point to it, which can easily be done as follows:

cd /boot/initrd-tree
find . | cpio -H newc -o > ../initrd.cpio

Grouping XML nodes using XSL 2.0

Man, that was painful!!! Curse those XML Committee morons that should have designed a transformation language based on Perl, rather than come with the atrocity that is XSL!

So, let's say you have the following XML (This is a list of cross references between 8051 banks, custom generated by a popular disassembler), which you already sorted using the other XSL I posted today:

<banks>
 <B0_000E/>
 <B0_000E/>
 <B0_000E/>
 <B0_000E/>
 <B0_49F8/>
 <B0_49F8/>
 <B0_49F8/>
 <B0_49F8/>
 <B0_49F8/>
 <B0_49F8/>
 <B0_520A/>
 <B0_520A>
    <BD_2E9D/>
    <BD_C4B5/>
 </B0_520A>
 <B0_520A/>
 <B0_54F9/>
 <B0_54F9/>
 <B0_54F9>
    <BB_E851/>
 </B0_54F9>
 <B0_54F9/>
 <B0_59EC>
    <B0_012F/>
 </B0_59EC>
 <B0_59EC>
    <BE_012F/>
 </B0_59EC>
 <B0_59EC>
    <BF_012F/>
 </B0_59EC>
 <B0_CC5E/>
 <B0_CC5E>
    <BC_DAA6/>
    <BC_DF2E/>
 </B0_CC5E>
 <B0_CC5E>
    <BB_DA1C/>
 </B0_CC5E>
 <B0_CC5E/>
  <B0_E022/>
 <B0_E022>
    <BE_E439/>
    <BE_E458/>
    <BE_EE05/>
    <BE_EE28/>
    <BE_F15B/>
    <BE_F17A/>
    <BE_F8E5/>
    <BE_F8F9/>
 </B0_E022>
 <B0_E022>
    <BF_3BF0/>
    <BF_6C6B/>
    <BF_7F7E/>
    <BF_7F9D/>
    <BF_7FE0/>
    <BF_7FFF/>
    <BF_CD2C/>
    <BF_D103/>
    <BF_D126/>
    <BF_D146/>
    <BF_D169/>
 </B0_E022>
 <B0_E022/>
 <B0_E022>
    <BC_CE5A/>
 </B0_E022>
 <B0_E022/>
 <B0_E022/>
 <B0_E201/>
 <B0_E201/>
</banks>

Obviously now, you would like to move all the sub nodes into a single one and remove all the duplicates to obtain a file like this:

<banks>
 <B0_000E/>
 <B0_49F8/>
 <B0_520A>
    <BD_2E9D/>
    <BD_C4B5/>
 </B0_520A>
 <B0_54F9>
    <BB_E851/>
 </B0_54F9>
 <B0_59EC>
    <B0_012F/>
    <BE_012F/>
    <BF_012F/>
 </B0_59EC>
 <B0_CC5E>
    <BC_DAA6/>
    <BC_DF2E/>
    <BB_DA1C/>
 </B0_CC5E>
 <B0_E022>
    <BE_E439/>
    <BE_E458/>
    <BE_EE05/>
    <BE_EE28/>
    <BE_F15B/>
    <BE_F17A/>
    <BE_F8E5/>
    <BE_F8F9/>
    <BF_3BF0/>
    <BF_6C6B/>
    <BF_7F7E/>
    <BF_7F9D/>
    <BF_7FE0/>
    <BF_7FFF/>
    <BF_CD2C/>
    <BF_D103/>
    <BF_D126/>
    <BF_D146/>
    <BF_D169/>
    <BC_CE5A/>
 </B0_E022>
 <B0_E201/>
</banks>

Well, using the XSL (2.0) below, now you can!

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output omit-xml-declaration="yes" indent="yes"/>

<xsl:template match="/">
  <banks> <!-- Our XML root element. Modify according to yours! -->
    <!-- Select all nodes that are immediate children of our root
         element ("/*/*") and group them according to their node
         names ("local-name()")  -->
    <xsl:for-each-group select="/*/*" group-by="local-name()">
      <!-- Output a (single) node for each of these groups -->
      <xsl:element name="{local-name()}">
      <!-- NB: To remove empty nodes, move the <xsl:element>
           section inside the <xsl:for-each-group> below, and use
           {local-name(parent::*)} instead of {local-name()} -->
        <!-- Select all the children nodes from the above group
             ("current-group()/*) - Don't bother to group them. -->
        <xsl:for-each-group select="current-group()/*" group-by=".">
          <!-- Here we copy the whole (group) branch as is -->
          <xsl:copy-of select="current-group()"/>
        </xsl:for-each-group>
      </xsl:element>
    </xsl:for-each-group>
  </banks> <!-- Close our root -->
</xsl:template>
</xsl:stylesheet>

Now, Vista, where's my file actually?

If security done backwards is better than no security at all, then Microsoft are the kings of running in reverse.

You might have used some programs to edit files in "System" directories, only to wonder why the files that are there don't reflect your changes.
The answer is that, when Microsoft is unhappy about the location you edit a file in (eg: C:\Program Files (x86)\<some program>\<some file>), it creates a duplicate in C:\Users\<your user name>\AppData\Local\VirtualStore\ (eg: C:\Users<your user name>\AppData\Local\VirtualStore\Program Files (x86)\<some program>\<some file>).

Pretty sure that having 2 copies of one file is gonna be a thrill for people who want to backup their data...

...and the class blurb for SyntaxHighlighter

Great tool that SyntaxHighlighter. Followed this post and got exactly what's written on the tin.

Now, since I'm of a forgetful nature, the syntax for the <pre> modifier is <pre class="brush: <brushname>"> where <brushname> is one of text, bash, xml, cpp or any of the keywords highlighted here.

...and the UltraEdit Macro script to convert code for blogger publishing

Courtesy of Rob, to which I added the & -> &amp; conversion:

InsertMode
ColumnModeOff
HexOff
UnixReOff
IfSel
Find SelectText "&"
Replace All "&"
Find SelectText "<"
Replace All "&lt;"
Find SelectText ">"
Replace All "&gt;"
Find SelectText """
Replace All "&quot;"
Find SelectText "^t"
Replace All "  "
Else
GotoLine 1 1
Find "&"
Replace All "&amp;"
Find "<"
Replace All "&lt;"
Find ">"
Replace All "&gt;"
Find """
Replace All "&quot;"
Find "^t"
Replace All "  "
EndIf

XSLT script to sort nodes alphabetically

The simple XSL sample you have been looking for. Works great with Saxon. "version" set to 2.0 here to prevent a warning with the latest version of Saxon, but you can change it to 1.0 for older XSLT engines.

<?xml version="1.0" encoding="UTF-8"?>

<xsl:stylesheet version="2.0" xmlns:xsl="http://www.w3.org/1999/XSL/Transform">
<xsl:output omit-xml-declaration="no" indent="yes"/>
<xsl:strip-space elements="*" />

<xsl:template match="/">
  <xsl:apply-templates />
</xsl:template>

<xsl:template match="@*|node()">
  <xsl:copy>
    <xsl:apply-templates select="@*|node()">
      <xsl:sort select="@typeName"/>
      <xsl:sort select="name(.)"/>
      <xsl:sort />
    </xsl:apply-templates>
  </xsl:copy>
</xsl:template>

</xsl:stylesheet>

Oh, and the saxon commandline syntax to apply an XSL (which should really be provided in a straightforward fashion in the commandline help) is:

C:\>saxon -s:banks.xml -xsl:sort.xsl -o:banks_sorted.xml

.NET, or the solution to a problem that didn't exist

So, you're using Visual Studio Express and you want to use that supposedly nice SerialPort .NET class (after spending more than half a day trying to make a standard COM port app that can talk nicely to putty - hint: don't rely on anything else but putty's actual port initialization code if you want something that works!).

And yeah, you're like me, so you don't care much about whatever the hell .NET brings to the OO paradigm. You just want to use a class that works ASAP, so you followed the tutorials on the net that tell you: "Just add a 'using namespace System::IO::Ports;' and then you can define a SerialPort instance", which you promptly did.

But now, of course, you're getting the bleeping error:

error C2039: 'Ports' : is not a member of 'System::IO'

Yeah, that's because with all this supposedly improved technology, Visual Studio is still not able to tell you that you need to add a

#include "system.dll"

before you can see ALL of the System namespace. "Intellisense", my ass!

In short: C0239 = need to add a #include to a dll (?!?, when did that become a C++ standard) BEFORE you can start using "using namespace"

Oh, and good luck figuring that one out if you don't (want to) have a clue about .NET...

More 8032 / 8051 trickery

Don't worry, the (serial) printf handler is coming. In the meantime, some more hard to spot Mediatek 8032/8051 (Geez, couldn't at least agree on the base architecture name, and use declinations that make sense?) compiler trickery with lcall:

ROM:C32C                 lcall   ROM_B613
ROM:C32F                 lcall   copy_4_bytes_to_xRAM ; input: DPTR = dest xRAM address
ROM:C32F                                         ;        PC   = src bytes address
ROM:C32F ; ---------------------------------------------------------------------------
ROM:C332                 .byte 0x00,0xC,0x00,0x00         ; 0
ROM:C336 ; ---------------------------------------------------------------------------
ROM:C336                 mov     DPTR, #0xFFFC
ROM:C339                 lcall   ROM_B613
ROM:C33C                 lcall   copy_4_bytes_to_xRAM ; input: DPTR = dest xRAM address
ROM:C33C                                         ;        PC   = src bytes address
ROM:C33C ; ---------------------------------------------------------------------------
ROM:C33F                 .byte 0,0xC,0,0         ; 0
ROM:C343 ; ---------------------------------------------------------------------------
ROM:C343                 mov     R6, #0x12
ROM:C345                 mov     R7, #0

When people extensively start/have to use instructions in other ways than they were designed for (shouldn't all lcalls' return flow of execution pcik up right after the instruction itself?), you know that you have a poorly designed CPU, no matter what people manage to achieve with it.

Yeah, it's another lcall'ed function that doesn't return where it's supposed to and breaks our nice disassembly flow. Thanks for making us having to manually edit our subroutine ends in IDA Pro, morons!

So, how does it work this time? Similarly to the previous trick actually:

ROM:B498 ; =============== S U B R O U T I N E =======================================
ROM:B498
ROM:B498 ; input: DPTR = dest xRAM address
ROM:B498 ;        PC   = src bytes address
ROM:B498
ROM:B498 copy_4_bytes_to_xRAM:                   ; CODE XREF: ROM:2A49p
ROM:B498                                         ; ROM:2A77p ...
ROM:B498                 mov     R0, DPL
ROM:B49A                 mov     B, DPH          ; DPTRd [DPHd DPLd] -> B R0 = Dest Pointer
ROM:B49D                 pop     DPH
ROM:B49F                 pop     DPL             ; DPTRs [DPHs DPLs] = Src Pointer
ROM:B4A1                 lcall   copy_byte
ROM:B4A4                 lcall   copy_byte
ROM:B4A7                 lcall   copy_byte
ROM:B4AA                 lcall   copy_byte
ROM:B4AD                 clr     A
ROM:B4AE                 jmp     @A+DPTR         ; DPTRs (= original return PC) has
ROM:B4AE ; End of function copy_4_bytes_to_xRAM  ; been incremented 4 times at this stage
ROM:B4AE
ROM:B4AF
ROM:B4AF ; =============== S U B R O U T I N E =======================================
ROM:B4AF
ROM:B4AF
ROM:B4AF copy_byte:                              ; CODE XREF: copy_4_bytes_to_xRAM+9p
ROM:B4AF                                         ; copy_4_bytes_to_xRAM+Cp ...
ROM:B4AF                 clr     A
ROM:B4B0                 movc    A, @A+DPTR      ; read [DPTRs]
ROM:B4B1                 inc     DPTR            ; DPTRs++ (as well as [DPTRs] -> A)
ROM:B4B2                 xch     A, DPH
ROM:B4B4                 xch     A, B            ; DPTRs <-> DPTRd (B R0)
ROM:B4B6                 xch     A, DPH
ROM:B4B8                 xch     A, R0
ROM:B4B9                 xch     A, DPL
ROM:B4BB                 xch     A, R0           ; A is still [DPTRs] at this stage
ROM:B4BC                 movx    @DPTR, A        ; [DPTRs] -> [DPTRd]
ROM:B4BD                 inc     DPTR            ; DPTRd++
ROM:B4BE                 xch     A, DPH          ; DPTRs <-> DPTRd
ROM:B4C0                 xch     A, B
ROM:B4C2                 xch     A, DPH
ROM:B4C4                 xch     A, R0
ROM:B4C5                 xch     A, DPL
ROM:B4C7                 xch     A, R0
ROM:B4C8                 ret
ROM:B4C8 ; End of function copy_byte

Gotta love these scores of xch instructions. Kind of the 3 cups & one red ball street magician classic. The only thing you need to know though, is that all a sequence like the following 3 lines does:

xch A, register1
xch A, register2
xch A, register1

is simply exchanging the content of register 1 & register 2, and keeping A unchanged.

I won't say this isn't a nice trick to access a small amount of data right within your program, but I'll take my general purpose register generous 68000 any time over your crappy 8051. It's a wonder really: How comes there are so few consumer devices using embedded 68000, when it would be so much more efficient for embedded applications? If you're gonna play with these kind of intel x32/x51 hurdles, it won't be that much different to go RISC all the way!

Search for low pointer values in IDA Pro

Use the following regexp:

DPTR, #0x[0-9A-F]{3}$

This will lock only on DPTR assignment values in 0x0000 0x0FFF

Switch statements, 8032/8051 style

Now this next trick is a little bit more clever.

If, when disassembling 8032 / 8051 code, you ever see the start of an lcall'ed routine that looks like this

ROM:0000B5BA                 pop     DPH             ; Data Pointer, High Byte
ROM:0000B5BC                 pop     DPL             ; Data Pointer, Low Byte

and then proceeds to use DPTR for movc instructions, the effective DPTR address used in that routine will be the next address after the LCALL (because lcall did push the current PC on the stack)

This is effectively used by, hum, some code, to implement switch statements that might be a bit tricky to detect during reverse engineering, as the disassembler obviously expect lcall to return, and will start disassembly the switch table.

Example code:

ROM:00006CA7 ROM_6CA7:                               ; CODE XREF: ROM_6C60+15j
ROM:00006CA7                                         ; ROM_6C60+1Bj ...
ROM:00006CA7                 mov     DPTR, #0xC46A
ROM:00006CAA                 movx    A, @DPTR
ROM:00006CAB                 xrl     A, #0x60
ROM:00006CAD                 jnz     ROM_6CF4
ROM:00006CAF                 mov     DPTR, #0xB03E
ROM:00006CB2                 movx    A, @DPTR
ROM:00006CB3                 lcall   case_switch_byte
ROM:00006CB3 ; ---------------------------------------------------------------------------
ROM:00006CB6                 .word ROM_6CE4
ROM:00006CB8                 .byte 0
ROM:00006CB9                 .word ROM_6CE4
ROM:00006CBB                 .byte 0x20
ROM:00006CBC                 .word ROM_6CE0
ROM:00006CBE                 .byte 0x2A
ROM:00006CBF                 .word ROM_6CDC
ROM:00006CC1                 .byte 0x2B
ROM:00006CC2                 .word ROM_6CDE
ROM:00006CC4                 .byte 0x2D
ROM:00006CC5                 .word ROM_6CE2
ROM:00006CC7                 .byte 0x2F
ROM:00006CC8                 .word ROM_6CDA
ROM:00006CCA                 .byte 0x39
ROM:00006CCB                 .word ROM_6CD2
ROM:00006CCD                 .byte 0x5A
ROM:00006CCE                 .word 0
ROM:00006CD0                 .word 0x6CEC
ROM:00006CD2 ; ---------------------------------------------------------------------------
ROM:00006CD2
ROM:00006CD2 ROM_6CD2:                               ; DATA XREF: ROM_6C60+6Bo
ROM:00006CD2                 mov     DPTR, #0xB03E
ROM:00006CD5                 mov     A, #0x30 ; '0'
ROM:00006CD7                 movx    @DPTR, A
ROM:00006CD8                 sjmp    ROM_6D54
ROM:00006CDA ; ---------------------------------------------------------------------------

with case_switch_byte being:

ROM:B5BA
ROM:B5BA ; =============== S U B R O U T I N E =======================================
ROM:B5BA
ROM:B5BA ; Iput: A = matching case value (byte)
ROM:B5BA
ROM:B5BA case_switch_byte:                       ; CODE XREF: ROM:4046p
ROM:B5BA                                         ; ROM_4552+178p ...
ROM:B5BA                 pop     DPH             ; Data Pointer, High Byte
ROM:B5BC                 pop     DPL             ; DPTR = lcall return address
ROM:B5BE                 mov     R0, A
ROM:B5BF
ROM:B5BF loop_through_cases:                     ; CODE XREF: case_switch_byte+24j
ROM:B5BF                 clr     A
ROM:B5C0                 movc    A, @A+DPTR      ; NB: @ is confusing. It's just A+DPTR
ROM:B5C1                 jnz     valid_dest      ; make sure dest @ != 0
ROM:B5C3                 mov     A, #1
ROM:B5C5                 movc    A, @A+DPTR
ROM:B5C6                 jnz     valid_dest
ROM:B5C8                 inc     DPTR
ROM:B5C9                 inc     DPTR            ; if null dest, just use the next
ROM:B5C9                                         ; word as dest address
ROM:B5CA
ROM:B5CA dest_match:                             ; CODE XREF: case_switch_byte+1Fj
ROM:B5CA                 movc    A, @A+DPTR
ROM:B5CB                 mov     R0, A
ROM:B5CC                 mov     A, #1
ROM:B5CE                 movc    A, @A+DPTR
ROM:B5CF                 mov     DPL, A          ; Data Pointer, Low Byte
ROM:B5D1                 mov     DPH, R0         ; dest into DPTR
ROM:B5D3                 clr     A
ROM:B5D4                 jmp     @A+DPTR
ROM:B5D5 ; ---------------------------------------------------------------------------
ROM:B5D5
ROM:B5D5 valid_dest:                             ; CODE XREF: case_switch_byte+7j
ROM:B5D5                                         ; case_switch_byte+Cj
ROM:B5D5                 mov     A, #2
ROM:B5D7                 movc    A, @A+DPTR
ROM:B5D8                 xrl     A, R0           ; cmp val with parameter
ROM:B5D9                 jz      dest_match
ROM:B5DB                 inc     DPTR
ROM:B5DC                 inc     DPTR
ROM:B5DD                 inc     DPTR            ; skip 3 bytes to next switch table entry
ROM:B5DE                 sjmp    loop_through_cases
ROM:B5DE ; End of function case_switch_byte
ROM:B5DE
ROM:B5E0

The same kind of routine also exists for a word parameter instead of a byte.
Most of the time, the case values will follow some kind of logical order, so if you see a bunch of sequencing bytes of word, interlaced with what look like offsets, and preceded by an lcall, you might want to chack what's on the other end of that lcall.

OR, the preferred way once you have made your initial pass at identifying code, look for a function that starts by popping DPH and DPL, and seek all the lcalls that cross reference to it to identify the switch tables.

Oh, and for those who might wonder, of course, as soon as you pop the PC address that's been enqueued on the stack, the lcall never returns, and becomes exactly like a jump.

Coming next: How the hell are these bloody strings and other data sitting in standalone data sections referenced, where there does not appear to be any obvious address referencing to them anywhere in the disassembly...

Now I remember why I hate the 8032...

...and pretty much EVERY CPU designed by intel, ever.

It's not like they could have turned a 16 bit CPU into a 24 or 32 bit addressing one, because that's way harder to do than extending the 32 bit x86 to 64 bit for instance (or the original 16 bit x86 to 32 bit)...

"Look Ma! It's a flock of randomly chosen flying search engine terms to direct a certain kind of people here!": Mediatek, MT8226, 8226, LCD, Samsung, 8032, 8051, MT13x9, bank, page, 0xFFFF, reverse engineering, firmware.

The usual trick, apparently originally devised by Philips, is to use one of the 8032 ports (usually P1) as an extention of the 16 bit address bus, thus acting as a ROM bank selector => if you use 8 bits of that port, you now have 24 bit addressing.

Of course, this doesn't mean that your PC will suddenly become 24 bit aware, and since it still remains 16 bit, you need to have overlap code, for each of your 64 KB banks ([0000-FFFF]) so that, as far as the CPU is concerned, the same bank is still being used.

This usually results in LOADS AND LOADS of ROM space wasted with the following kind of crap:

ROM:8301 ; --------------------------------------------------------------------------
ROM:8301                 mov     DPTR, #0x7E2D
ROM:8304                 ljmp    bank_0E
ROM:8307 ; ---------------------------------------------------------------------------
ROM:8307                 mov     DPTR, #0xE4D1
ROM:830A                 ljmp    bank_0B
ROM:830D ; ---------------------------------------------------------------------------
ROM:830D                 mov     DPTR, #0xF51E
ROM:8310                 ljmp    bank_0F
ROM:8313 ; ---------------------------------------------------------------------------
ROM:8313                 mov     DPTR, #0xFA0D
ROM:8316                 ljmp    bank_0F
ROM:8319 ; ---------------------------------------------------------------------------
ROM:8319                 mov     DPTR, #0xF924
ROM:831C                 ljmp    bank_0F
ROM:831F ; ---------------------------------------------------------------------------
ROM:831F                 mov     DPTR, #0xFDA7
ROM:8322                 ljmp    bank_0D

with the bank switching function of the like:

ROM:8288 ; ---------------------------------------------------------------------------
ROM:8288 bank_0E:                                ; CODE XREF: ROM:84F6j
ROM:8288                                         ; ROM:852Cj ...
ROM:8288                 mov     A, P1           ; Port 1
ROM:828A                 anl     A, #0xF         ; 8 bits for high addressing => 16 banks
ROM:828C                 cjne    A, #0xE, ROM_8291 ; bank 15 (0x0E)
ROM:828F                 clr     A
ROM:8290                 jmp     @A+DPTR
ROM:8291 ; ---------------------------------------------------------------------------
ROM:8291 ROM_8291:                               ; CODE XREF: ROM:828Cj
ROM:8291                 swap    A
ROM:8292                 rr      A
ROM:8293                 push    ACC             ; Accumulator
ROM:8295                 mov     A, #0xBF ; '+'
ROM:8297                 push    ACC             ; Accumulator
ROM:8299                 push    DPL             ; Data Pointer, Low Byte
ROM:829B                 push    DPH             ; Data Pointer, High Byte
ROM:829D                 ljmp    ROM_BF70
ROM:82A0 ; ---------------------------------------------------------------------------
(...)
ROM:BF70 ; ---------------------------------------------------------------------------
ROM:BF70 ROM_BF70:                               ; CODE XREF: ROM:829Dj
ROM:BF70                 anl     P1, #0xF0       ; Go trough bank 0
ROM:BF73                 orl     P1, #0xE        ; Set the high address lines to bank 0x0E on Port 1
ROM:BF76                 ret
ROM:BF76 ; ---------------------------------------------------------------------------

As indicated above, each bank has to DUPLICATE this whole bank switching business at the exact same address. And there will be a short period where code is actually ran in Bank 0 (right after the "anl P1, #0xF0") before switching to the actual destination bank (here Bank 0Eh)

Only thing I can say is, seeing how crappy a CPU it is (don't get me started about how poorly optimized all the 8032 code I see seems to be), licensing for the 8032 must come really REALLY cheap!

But then again, having SoC makers like MediaTek design their systems on the cheap might have its advantages...

Why does windows find the need to modify the partition table at boot...

...or how I almost lost 1.5 TB of data today.

If you use software/fake RAID, this might happen to you too.

See, this morning I thought I would upgrade my HTPC's BIOS just for the heck of it. This is the PC that has a 2TB RAID5 array based on nVIDIA's MediaShield.
Now, despite what I had selected during the BIOS update, the BIOS settings and DMI data were reset after reboot, which means that the HDDs were back to individual IDE emulated drives, rather than members of the RAID array.

Normally, this wouldn't be a big deal, except that, before I cancelled the Windows boot, it was apparently able to look at the disks (using the MediaShield driver), find out that the capacity of the disk it was booting from (now a single 1TB IDE/AHCI HDD) was less than the capacity reported in the partition table, and re-write the partition table of HDD1 to reduce the dimensions of the last partition.

Of course, re-writting a partition table without anybody asking you to is the shortest way to screw up a disk or RAID array, and screw up it did: As soon as I restored the RAID settings in the BIOS and booted Windows, my 1.5 TB data partition was now identified as unformatted and gone! Talk about massive data loss...

No respectable O/S should ever modify a partition table without asking the user first. It's just common sense: The O/S is never, and I have to stress out that part, NOT EVER, smarter than its user (no matter what the O/S developers might think, or how smart they think they are themselves). You do not modify a partition table without asking, EVER, it's really simple as that!

Now, after much cursing, and some accidental good luck, I found that if the first drive was disconnected from the RAID5 array (which happened accidentally as I was trying to invert HDD#2 and HDD#3, since it originally looked like the BIOS upgrade has modified the SATA IDs), the rest of the array booted fine, albeit in degraded mode, and saw the old 1.5 TB data partition alright. Definitely makes sense with the fact that Windows would of course only have modified the partition table of the boot drive while the HDDs were in IDE mode.
But of course, as soon as you remove one drive from your RAID5 array, and boot in degraded mode, the array will flag that drive as failed on next reboot

From there on, the solution is to re-add the drive to the array to resync. Takes a while, but if you trust your other disks not to fail duing the super-lengthy re-sync, probably the safest solution.
Otherwise, it's probably a good idea to have a copy of the Master Boot Record (i.e. the first 512 bytes) of every single drive from your array, and restore it using a decent O/S like Linux. Plus, as experience will show you time and again, it's also always good practice to keep a copy of the MBRs of all your disks that contain important data, so that you can try to address any kind of partition formatting catastrophe.

Displaying International Domain Names in the browser's address bar

• In IE (>7): Tools -> Internet Options -> Advanced -> International -> Always show encoded addresses -> Uncheck
  A typical "All or nothing" Microsoft solution to a problem that demands a smart approach. Well done Redmond!
• In Firefox (from Marc Blanchet's post): about:config and filter using keyword "IDN". There you have two possibilities:
  
  1. Create a "network.IDN.whitelist.com" boolean key and set it to true => will display all the .com IDNs using their Unicode characters, a.k.a. "the Microsoft way"
  2. Create a new network.IDN.whitelist boolean entry for your full domain (eg: "network.IDN.whitelist.xn--abc.com") and set it to true.

Of course, none of this is really satisfying, as domains that do not present any risk of homograph attack (eg: sequence of glyphs that are not used in any language, or one glyph characters that cannot be mistaken for any other), are still not displayed as Unicode by default in those browsers, which really sucks.

The way to address this issue would be for unicode.org to maintain a list of lookalike characters (i.e. characters considered as dangerous to use in an IDN because they can easily be mistaken for another character). The browser could then use this list (along with a list of all the valid Unicode characters at the time the list was created, as you want to take provision against new spoofable Unicode characters) to sort this issue.

Of course, Unicode is likely to have cold feet about producing such a list, as they could end up being sued if they miss a lookalike and end up being sued as a result, but really, as the authority, it should be part of their role...

Unicode charts as a single image

Well, one guy was crazy enough to create a single image containing all the Unicode charts from unicode.org and put it online, and he was even nice enough to provide both an interface to dynamically visualize the rendered image, as well as allow people to download the whole thing (which, even if slightly outdated, I find super useful).

Props to Ian then!

Update 2009.08.24 - You might also be interested in this

Windows Command Prompt in UTF-8

1. Change the default font to Lucida
   
2. issue the command chcp 65001
   

Convert a grayscale or RGB layer to an alpha channel in Photoshop CS 4

Goddammit! I don't know if 3D graphics designer use Photoshop or not, and if they do, how they manage to put with it all day, but talk about a product not being intuitive... On the other hand, it's true that I know a thing or two about how Adobe doesn't care that much about any of the software they design to be either customer friendly or intuitive...

So, today, in our new "What we thought we'd do in about 30 seconds in Photoshop CS4 but ended up doing in hours" saga, we'll see how to convert a shadow effect layer into an alpha channel. The purpose here is to display a nice transparent text overlay texture, with a blending shadow against a non uniform background in a 3D animation (eg. DirectX, OpenGL, ...). 

And because I'm in a generous mood today, I'm even gonna drop some illustrations here in there. Let's say that you are creating an animated "motivational poster", like the one on the left, and your goal is to do some nice animation of the motto like a zoom out for instance, while making sure the motto has a shadow for added impact.

You will of course break down your animation between a static background and a text layer, loaded as a texture, and animate the latter.

Now, if you merely save the text layer with a shadow effect in Photoshop and use that as your texture, it will not blend nicely at all, as the shadow will be a solid colour, and eat part of the the nice aperture green logo. Therefore you want to add an alpha component to your texture, which you'll use for shadow blending.

At this stage, I'll suppose that you have your nice text layer with its shadow effect. The good part about all this ordeal is that you won't have to create a separate layer out of the effect, or play with adding/substracting selections, because a black text display with its shadow is pretty much exactly what you need to use for the alpha channel. The only thing is, if you use coloured or texturized text, you'll want to convert it to solid black before the next operations, through a duplicate layer or something.

Now, if you're a Photoshop novice and you've done your research, you should be aware that the alpha components of an image is dealt with as a channel in Photoshop, and that there is a tab for that (again non-intuitive as hell: where is my "right click -> new channel" option, Adobe? And how am I supposed to know what the small icons at the bottom represent when there is no bleeping contextual help in your product?!?). So you'd think that, with all the graphics people dealing with channels, depths masks and alpha all day long, there'd be a bloody "Convert Layer to Channel" function readily available by default in the interface. Well, if it was straightforward to work with alpha channels in Photoshop, that would make the life of graphics designers too easy, and obviously this is not something that you would ever want.

So, after cursing for about 10 minutes about those non-intuitive menus, we have to fallback to looking on the internet for some helpful souls (and I got to give it to CreativeCow there, for providing all the actual solution steps):

1. Search into your Photoshop installation directory for .atn files (AcTioNs) and locate the one labelled "Video Actions.atn". Mine was in "D:\Program Files\Adobe\Adobe Photoshop CS4 (64 Bit)\Presets\Actions"
   
   
2. In Photoshop, go to Window -> Actions. You will see a set of "Default Actions", but of course, not all the Actions that come with Photoshop have been loaded by default in one of the more obscure menu of the product, where noone would mind if they had been.
   
   
3. Then click on what has to be the most useful button for advanced users, and therefore the one that's been made as easy to overlook as possible, namely that bleeping contextual dropdown menu. And I really have to wonder what is it with this late redesign of UIs by Adobe and Microsoft (eg. Office 2007), whose only purpose seems to make the power users' life a living hell by hiding the most powerful feature in a place where the sun don't shine no more...
   
   
4. In that menu, you will find a "Load Action" option, which you should use to navigate to the folder where you spotted the "Video Actions.atn" file, and open that file. Alternatively, you can skip step 1 and directly select "Video Actions" in that menu to make the Video Actions appear. Why "Video Actions" is in that menu but not in the main action window by default is really beyond me here...
   
   
5. OK, now we're talking: In the "Video Actions" section, the first 2 actions are: "Alpha Channel from Visible Layers" and "Alpha Channel from Visible Layers (inverted)". The Ark of the Covenant at last! And all we had to do was get our face and soul melted for it - a bargain! Merrily we go then by setting only our "black text with shadow" layer visible (make sure you don't select the background) and then clicking the little "Play" button in the Actions menu with "Alpha Channel from Visible Layers" selected. You should end up with a new channel that looks like the one on the left. In this image, white is for pixels with no alpha (completely opaque) whereas black is for pixels with 100% alpha (completely transparent).
   
   
6. Now, before we save our image with its newfound alpha channel, we want to lay it over black solid background for the shadow, so just create one and save only the text layer (with or without the shadow effect - it shouldn't matter) over the black background as your image with alpha. You might want to enable the alpha channel (set the eye) after creating the black background to have a better idea of what exactly you're saving.
   
   
7. Now because there does not appear to be clear conventions as to what a zero alpha value is meant to represent (this can vary from application to application), you might find that the alpha mask you get in your application does the opposite effect of what you wanted. If that's the case, rather than redo the steps using the "Inverse" function for layer to alpha we've seen in the menu, you can simply double click on your Alpha channel in Photoshop and toggle between "Color Indicates Masked Area" / "Color Indicates Selected Area", as this will invert your Alpha channel

And that, sir, is how you do it.

Of course, if you look at the actions, you can probably figure out how to achieve the same from the various menus (haven't really figured that one just yet, especially where a on/off selection is converted to a grayscale range). I also haven't figured out how you can insert one of those actions into the standard Photoshop menus (wouldn't it sit nicely in the layer menu?). Oh and finally, if like me you've cursed Photoshop's stupid and uninformative handling of undo (Why is it that a 10 year old program like Paint Shop Pro 5 does a much better job at undo than Photoshop?), note that you can access the undo history from the Window menu as well, where you will actually get a description of the actions you have taken.

More vi search and replace

To replace ^M characters that did not quite make it into carriage return when displayed in vi:

:%s/<Ctrl-V><Ctrl-M>/<Ctrl-V><Enter>/g

Note that <Ctrl-V> is a magic sequence that allows vi to uninterpret the next special key, inserting its control code instead.

redhat/centos services and mdadm quicknotes

Always seem to forget stuff there. The command I'm looking for to manage services/change runlevel on redhat/centos is chkconfig (chkconfig --list).

Also, to re-add a partition to an existing (failed) RAID array:

mdadm /dev/md0 --manage --add /dev/sdb1

Mysql logging

One day or another, you'll want to check what exactly your MySQL server is doing (to check for SQL injections for instance). Therefore:

• To log all SQL queries, add the following to your my.cnf (in the [mysqld] section)
  

  log=/var/log/mysqld.log

• To log queries that actually modify the database:
  

  log-bin

  If you don't modify the default log-bin parameters, this would create a <servername>-bin.###### & index in your /var/lib/mysql, which you can then analyze with mysqlbinlog
  

Getting some bloody files from Gigabyte

There's not ONE motherboard manufacturer to save the others. NOT ONE!
Listen, we all know that you're hardware developers, and that providing files is an afterthough, but that is no reason to:
- force customers to use God-awful, slow, and completely useless interfaces on your website when all they want is access BIOSes or drivers
- host your bloody files on a server that must still be powered by DX2 66MHz CPU with 8 MB RAM. Yeah, it really looks that slow from our end!!!
- suddenly decide that you're gonna change all the files location, because it was a bit too convenient for customer to google and access the data directly, so what better way to say "f*ck you!" than changing the interface and means of accessing the files every 6 months or so.

Asus, Gigabyte, MSI, Supermicro, whatever: YOUR SUPPORT & DOWNLOADS WEBISTES ALL SUCK, BIG TIME!!!

It's not like we're asking for much. We just want to access the BIOS, drivers and utilities without having to jump in slow motion (or should I say bullet-time, considering how time crawling to a standstill page refreshes appear to be) through hoops. And I won't hold you much of a grudge for designing websites that suck, because that's not your main area of expertise, but having them run on underperforming hardware is hardly the best way to put your customers in confidence.

All this rant to tell you that, if you actually want to download a file from GigaByte, don't bother going to gigabyte.com.tw. Just use ftp://download.gigabyte.ru/