I recently switched this server from running Gentoo Linux to Ubuntu Linux. In the process I only wanted to replace the OS and not loose my home directory and other data. 160G hard drives which are partitioned identically so that I can used RAID in a mirrored setup (RAID1). There are 4 partitions on the drive: a 10G primary partition which I had intended for windows but never used, 100M primary partition used as /boot, a 2G logical partition for the root filesystem (/), a 500M swap partition (Not raided gives me 1G of swap), and the rest ~151G used as LVM for the rest of the system.

Running # cat /proc/mdstat gives the following output:

Personalities : [raid1]
md2 : active raid1 hde6[0] hdg6[1]
      147717568 blocks [2/2] [UU]

md1 : active raid1 hde5[0] hdg5[1]
      1999936 blocks [2/2] [UU]

md0 : active raid1 hde2[0] hdg2[1]
      96320 blocks [2/2] [UU]

unused devices: <none>

Running # lvscan gives the following ouput:

ACTIVE            '/dev/vg0/tmp' [2.00 GB] inherit
ACTIVE            '/dev/vg0/home' [20.00 GB] inherit
ACTIVE            '/dev/vg0/usr' [15.00 GB] inherit
ACTIVE            '/dev/vg0/local' [5.00 GB] inherit
ACTIVE            '/dev/vg0/opt' [5.00 GB] inherit
ACTIVE            '/dev/vg0/var' [5.00 GB] inherit
ACTIVE            '/dev/vg0/data' [88.87 GB] inherit

I wanted to format md0, md1, /dev/vg0/tmp, /dev/vg0/usr, /dev/vg0/local, /dev/vg0/opt, /dev/vg0/var, and keep the data on /dev/vg0/home, /dev/vg0/data. I did a backup of /etc for reference and /var/www because I would need to restore some stuff. I also backed up my mysql databases since they are also stored in /var. I did a backup of most of the important data on /dev/vg0/data and /dev/vg0/home. Some data I did not backup (media) because there was just too much of it.

Once all of my planning and backing up was done I booted with the Ubuntu Dapper Drake alternate i386 disk which is required for RAID and/or LVM configurations. My first supprise was that there was no GUI install. We were back to the Debian console install manager. Everything went smoothly until I got to the partitioning. It did not detect my existing RAID and LVM configuration. I was able to get my existing configuration up and running via the command line on another Virtual Terminal(VT), but by that point I had the installer wedged in a state where it just would not continue.

I had to reboot and start all over again. Here are the steps I took to get it working:

1. boot the installer
2. switch VTs to get a console
3. run modprobe raid1
4. run modprobe dm_mod
5. run mdrun
6. run vgchange -a y
7. cat /proc/mdstat and lvscan should give output like above
8. change back to the VT with the installer running in it
9. continue with installer steps
10. when you get to the partitioning step, it should recognise your raid and lvm devices

I ended up getting everything installed with no loss of data.

I previously talked about build environments in Windows and promised to do some more testing of the reliability of certain tools. I turns out that the only set of tools which actually works as expected are the ones provided by Cygwin. I could not get the AT&T UWIN tools to install and they are just as heavy weight as Cygwin, which at least provides BASH. I thought the GNUWin32 tools had promise, but unfortunately they have problems with quoting, and don't provide make. I have also determined that MSys has similar quoting problems and a very outdated make.

There are a handful of other options, but Cygwin just seems to be the best. The real downside to Cygwin is that its setup tool that has a UI more suited to torturing its users than letting them install packages. Luckily someone realized that Debian got the model mostly right and created an apt like tool for Cygwin.

The site I linked to above suggests installing rxvt. The command they give to replace the Cygwin Bash shell link needs some help try adding -e /bin/bash --login -i to the end of it.

The comments over here also provide some potential terminal emulator and shell alternatives for Windows.

Recently I have been playing a bit with makefiles on win32 and Unix. I want to be able to accommodate a couple of build environments without using automake, configure and friends. I am also following Peter Miller's suggestions from Recursive Make Considered Harmful.

The first step to doing this is removing reliance on external tools. There are a couple of things you can rely on, but not very many; the least of which is shell syntax. Very helpful is the <a style="font-family: times new roman;" href="http://gmsl.sourceforge.net/">GNU Make Standard Library</a> (GMSL). While I have only used a couple functions so far, it hints that there is much that can be done right in the makefile that many people rely on external tools for. The GMSL for instance provides a tr(translate) command and associative arrays.

Some of the things I am currently finding most useful are the gmake built in functions 'define', 'call' and 'eval'. These three constructs allow for some very interesting techniques (avoid using spaces between arguments or you will get a cryptic missing separator error). For those of you who are looking for ways to deal with some standard win32 compatibility things here are a couple of snippets:

ifeq ($(notdir $(SHELL)),cmd.exe)
#make didn't find a bourn shell.
#We probably don't have rm, cp or a proper mkdir either
CP=copy $(subst /,\,$(1))
RM=for %f in ( $(1) ) do if exist %f del %f
MKDIR=mkdir $(subst /,\,$(1))
else
#make found some variety of bourn shell,
#assume that we have rm, cp and a sane mkdir
CP=cp $(1)
RM=rm -f $(1)
MKDIR=mkdir -p $(1)
endif

In order to use the piece above you need to get out of the habit of doing:

clean:
    $(RM) file1 file2 file3
And use the following instead:
#!make
clean:
    $(call RM, file1 file2 file3)

This probably will require some retraining to be consistent about it, but ultimately will make your makefiles more easily cross platform.

My unfortunately discovery is that not all gmake versions are the same. I know of 3 separate sources of gmake for win32: MinGW, MSys and Cygwin. The make from MSys is the least useful; it is an old version (3.79.x) and does not seem to handle define properly. The make from Cygwin works beautifully, except that when directories change, it uses its /cygdrive/c/path notation, which is more than a little annoying if you are trying to catch directory changes in another tool like VIm or Emacs. The make from MinGW may be slightly more useful in that define works, but I was having trouble with the test suite. Thinking about it now, it is possible that I may have messed things up with pathing. I will have to check that and report back.

While I am talking about makefiles and compatibility on win32 I might as well also talk about my discoveries about sed. The goal of my work was to require only gmake, a compiler and sed. As it turns out not all versions of sed are created equal, much like the problems with make described above. Apparently the sed that ships with MSys is doing some strange things with backslash escaping. The following should work identically using either Bourne shell or cmd.exe:

target:
   sed -e 's/ *\\ *$$//' < file.in > file.out

This works with the sed from http://gnuwin32.sf.net/ but not with the GNU sed that ships with MSys. Strangely the following works with the MSys sed:

target:
    sed -e 's/ *\\\\ *$$//' < file.in > file.out

Maybe I should just cut my losses and add a * since given the data, I know I will never have more than one backslash:

target:
    sed -e 's/ *\\\\* *$$//' < file.in > file.out

That should make work with either sed, but it is not very helpful for the general case.

Update: I just looked again at http://unxutils.sourceforge.net/, who's make I previously discounted because the version listed is old (3.78.1), and noticed that the news section says they have updated to 3.80. I will try the GNU make and sed from this package and see what I get.

Update 2: It appears that just MSys is broken. Both the make from MinGW and the make from UnxUtils both pass the gmsl-test once I fixed my path problem. I still noticed some thing funny about those two make programs, Cygwin's make continues to work flawlessly.

The last sed command, above, does not work, and the sed provided by MSys broke several sed scripts that I was using. I don't know if taking the scripts from a file would work better since there would be no escaping issues. At this point I recommend staying away from MSys.