Building a GHC cross-compiler

Posted on April 13, 2016
Tags: haskell, software, openwrt

(Update: I used this blog post as the basis for a page about OpenWRT on HaskellWiki.)

I’d like to be able to run Haskell programs on my OpenWRT box. Since my OpenWRT box has an x86_64 processor, it could almost run the same binaries that my desktop Linux machine does. However, the reason it can’t is because it uses a different C library. Desktop Linux (specifically Ubuntu in my case) uses glibc. OpenWRT used uclibc in past versions, and uses musl on trunk. Since binaries are normally linked dynamically (and in the case of glibc, they pretty much have to be), the target machine needs to have the same C library that the executable was built with.

One approach would be to dynamically link against the C library that OpenWRT uses. However, an even more robust approach is to link the executable statically. Then it can run on any x86_64 Linux machine, regardless of what libraries it has installed.

Unfortunately, glibc doesn’t really work correctly with fully static linking. Therefore, another C library needs to be used. The best choice seems to be musl. Coincidentally, musl is also the C library used by OpenWRT trunk, but since we’re going to link statically, it wouldn’t have to be.

So, how do we compile and link Haskell programs against musl? One approach is to use a musl version of ghc. Unfortunately, that approach requires a full musl-based Linux system to run on. The author of that post recommends a musl-based Gentoo distribution, but that seemed kind of icky to me. I almost could use the OpenWRT box as the musl-based build system, but that would require a working gcc on the OpenWRT box. Currently, on OpenWRT trunk, the gcc package is broken.

What I really want is a cross-compiler, which will run on my glibc-based Ubuntu box, but will build executables that use statically-linked musl. That, in turn, requires having a cross-compiling toolchain (gcc, binutils, etc.) before building the ghc cross-compiler. There are several ways to satisfy this requirement, but after a recommendation on the haskell-cafe mailing list, I decided to give Crosstool-NG a try.

I discovered that, in addition to the packages already installed on my system, I needed to install these packages in order to get Crosstool-NG to work:

ppelletier@patrick64:~/src$ sudo apt-get install gperf bison texinfo help2man python-dev

So, I got started with Crosstool-NG:

ppelletier@patrick64:~/src$ curl -O http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.xz
ppelletier@patrick64:~/src$ tar -xf crosstool-ng-1.22.0.tar.xz
ppelletier@patrick64:~/src$ cd crosstool-ng/
ppelletier@patrick64:~/src/crosstool-ng$ ./configure --prefix=/usr/local
ppelletier@patrick64:~/src/crosstool-ng$ make
ppelletier@patrick64:~/src/crosstool-ng$ sudo make install
ppelletier@patrick64:~/src/crosstool-ng$ mkdir ~/crosstool-ng
ppelletier@patrick64:~/src/crosstool-ng$ cd ~/crosstool-ng
ppelletier@patrick64:~/crosstool-ng$ ct-ng x86_64-unknown-linux-gnu
ppelletier@patrick64:~/crosstool-ng$ ct-ng menuconfig

In the menuconfig, I selected C-library > C library > musl, and it’s also a good idea to de-select Paths and misc options > Render the toolchain read-only, because I’m going to install the GHC cross-compiler in the same directory tree, so the directories need to be writable.

Then, I was able to do:

ppelletier@patrick64:~/crosstool-ng$ ct-ng build

And a little over an hour later, I could do:

ppelletier@patrick64:~/x-tools$ export PATH="${PATH}:${HOME}/x-tools/x86_64-unknown-linux-musl/bin"
ppelletier@patrick64:~/x-tools$ x86_64-unknown-linux-musl-gcc --version
x86_64-unknown-linux-musl-gcc (crosstool-NG crosstool-ng-1.22.0) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Next, I read the instructions for cross-compiling ghc, and then I did:

ppelletier@patrick64:~/src$ curl -O http://downloads.haskell.org/~ghc/7.10.3/ghc-7.10.3b-src.tar.xz
ppelletier@patrick64:~/src$ tar -xf ghc-7.10.3b-src.tar.xz
ppelletier@patrick64:~/src$ cd ghc-7.10.3/

And then I attempted to build GHC. I ran into some issues along the way, so I’m going to present the fixes to those issues now, rather than reconstructing the exact order in which I did things.

The main issue is that GHC needs ncurses on the target. I’m not sure why, but it does. So, I had to go and cross-compile ncurses:

ppelletier@patrick64:~/src/ncurses-6.0$ ./configure --host=x86_64-unknown-linux-musl --prefix=${HOME}/x-tools/x86_64-unknown-linux-musl
ppelletier@patrick64:~/src/ncurses-6.0$ make

Well, that ended up failing with:

x86_64-unknown-linux-musl-gcc -DHAVE_CONFIG_H -I. -I../include -DNDEBUG -O2 --param max-inline-insns-single=1200 -c ../ncurses/lib_gen.c -o ../objects/lib_gen.o
In file included from ./curses.priv.h:325:0,
                 from ../ncurses/lib_gen.c:19:
_20591.c:843:15: error: expected ')' before 'int'
../include/curses.h:1631:56: note: in definition of macro 'mouse_trafo'
 #define mouse_trafo(y,x,to_screen) wmouse_trafo(stdscr,y,x,to_screen)
                                                        ^
make[1]: *** [../objects/lib_gen.o] Error 1
make[1]: Leaving directory `/home/ppelletier/src/ncurses-6.0/ncurses'
make: *** [all] Error 2

I found the answer in wereHamster’s Dockerfile. In ncurses/base/MKlib_gen.sh, line 65 needs to be changed from:

preprocessor="$1 -DNCURSES_INTERNALS -I../include"

to:

preprocessor="$1 -P -DNCURSES_INTERNALS -I../include"

Okay, now the make succeeds, and after make, I do:

ppelletier@patrick64:~/src/ncurses-6.0$ make install

However, this is not enough. I thought that by installing it under the cross toolchain’s prefix, gcc would automatically pick up the ncurses includes. But it didn’t. So, I need to specify --with-curses-includes and --with-curses-libraries to ghc’s configure. Unfortunately, there doesn’t seem to be a single value that works for --with-curses-includes, and it doesn’t seem to be possible to specify more than one value for --with-curses-includes.

If I use --with-curses-includes=/home/ppelletier/x-tools/x86_64-unknown-linux-musl/include, then the build is unable to find ncurses.h. However, if I use --with-curses-includes=/home/ppelletier/x-tools/x86_64-unknown-linux-musl/include/ncurses, then ncurses.h includes ncurses/ncurses_dll.h, which is not found. My really ugly solution was:

ppelletier@patrick64:~/x-tools/x86_64-unknown-linux-musl/include/ncurses$ ln -s . ncurses

Now we’re ready to actually build ghc successfully. In the ~/src/ghc-7.10.3/mk directory, I copied build.mk.sample to build.mk, and then uncommented BuildFlavour = quick-cross. Next I did:

ppelletier@patrick64:~/src/ghc-7.10.3$ ./configure --target=x86_64-unknown-linux-musl --with-curses-includes=/home/ppelletier/x-tools/x86_64-unknown-linux-musl/include/ncurses --with-curses-libraries=/home/ppelletier/x-tools/x86_64-unknown-linux-musl/lib --prefix=/home/ppelletier/x-tools/x86_64-unknown-linux-musl
ppelletier@patrick64:~/src/ghc-7.10.3$ make
ppelletier@patrick64:~/src/ghc-7.10.3$ make install

It turns out that this toolchain still links dynamically by default, so I have to specify -static -optl-static when compiling Haskell programs. For example:

ppelletier@patrick64:~/programming/haskell$ x86_64-unknown-linux-musl-ghc -O -static -optl-static contact.hs
[1 of 1] Compiling Main             ( contact.hs, contact.o )
Linking contact ...
ppelletier@patrick64:~/programming/haskell$ file contact
contact: ELF 64-bit LSB  executable, x86-64, version 1 (SYSV), statically linked, not stripped
ppelletier@patrick64:~/programming/haskell$ ldd contact
        not a dynamic executable

The contact executable can now be run on either my Ubuntu box or my OpenWRT box.