GNU Make, win32 and sed

January 26, 2006 at 09:31 PM | categories: Programming | View Comments

Recently I have been playing a bit with makefiles on win32 and Unix. I want to be able to accommodate a couple of build environments without using automake, configure and friends. I am also following Peter Miller's suggestions from Recursive Make Considered Harmful.

The first step to doing this is removing reliance on external tools. There are a couple of things you can rely on, but not very many; the least of which is shell syntax. Very helpful is the <a style="font-family: times new roman;" href="http://gmsl.sourceforge.net/">GNU Make Standard Library</a> (GMSL). While I have only used a couple functions so far, it hints that there is much that can be done right in the makefile that many people rely on external tools for. The GMSL for instance provides a tr(translate) command and associative arrays.

Some of the things I am currently finding most useful are the gmake built in functions 'define', 'call' and 'eval'. These three constructs allow for some very interesting techniques (avoid using spaces between arguments or you will get a cryptic missing separator error). For those of you who are looking for ways to deal with some standard win32 compatibility things here are a couple of snippets:

ifeq ($(notdir $(SHELL)),cmd.exe)
#make didn't find a bourn shell.
#We probably don't have rm, cp or a proper mkdir either
CP=copy $(subst /,\,$(1))
RM=for %f in ( $(1) ) do if exist %f del %f
MKDIR=mkdir $(subst /,\,$(1))
else
#make found some variety of bourn shell,
#assume that we have rm, cp and a sane mkdir
CP=cp $(1)
RM=rm -f $(1)
MKDIR=mkdir -p $(1)
endif

In order to use the piece above you need to get out of the habit of doing:

clean:
    $(RM) file1 file2 file3
And use the following instead:
#!make
clean:
    $(call RM, file1 file2 file3)

This probably will require some retraining to be consistent about it, but ultimately will make your makefiles more easily cross platform.

My unfortunately discovery is that not all gmake versions are the same. I know of 3 separate sources of gmake for win32: MinGW, MSys and Cygwin. The make from MSys is the least useful; it is an old version (3.79.x) and does not seem to handle define properly. The make from Cygwin works beautifully, except that when directories change, it uses its /cygdrive/c/path notation, which is more than a little annoying if you are trying to catch directory changes in another tool like VIm or Emacs. The make from MinGW may be slightly more useful in that define works, but I was having trouble with the test suite. Thinking about it now, it is possible that I may have messed things up with pathing. I will have to check that and report back.

While I am talking about makefiles and compatibility on win32 I might as well also talk about my discoveries about sed. The goal of my work was to require only gmake, a compiler and sed. As it turns out not all versions of sed are created equal, much like the problems with make described above. Apparently the sed that ships with MSys is doing some strange things with backslash escaping. The following should work identically using either Bourne shell or cmd.exe:

target:
   sed -e 's/ *\\ *$$//' < file.in > file.out

This works with the sed from http://gnuwin32.sf.net/ but not with the GNU sed that ships with MSys. Strangely the following works with the MSys sed:

target:
    sed -e 's/ *\\\\ *$$//' < file.in > file.out

Maybe I should just cut my losses and add a * since given the data, I know I will never have more than one backslash:

target:
    sed -e 's/ *\\\\* *$$//' < file.in > file.out

That should make work with either sed, but it is not very helpful for the general case.

Update: I just looked again at http://unxutils.sourceforge.net/, who's make I previously discounted because the version listed is old (3.78.1), and noticed that the news section says they have updated to 3.80. I will try the GNU make and sed from this package and see what I get.

Update 2: It appears that just MSys is broken. Both the make from MinGW and the make from UnxUtils both pass the gmsl-test once I fixed my path problem. I still noticed some thing funny about those two make programs, Cygwin's make continues to work flawlessly.

The last sed command, above, does not work, and the sed provided by MSys broke several sed scripts that I was using. I don't know if taking the scripts from a file would work better since there would be no escaping issues. At this point I recommend staying away from MSys.

blog comments powered by Disqus