Building a GHC cross-compiler
(Update: I used this blog post as the basis for a page about OpenWRT on HaskellWiki.)
I’d like to be able to run Haskell programs on my OpenWRT box. Since my OpenWRT box has an x86_64 processor, it could almost run the same binaries that my desktop Linux machine does. However, the reason it can’t is because it uses a different C library. Desktop Linux (specifically Ubuntu in my case) uses glibc. OpenWRT used uclibc in past versions, and uses musl on trunk. Since binaries are normally linked dynamically (and in the case of glibc, they pretty much have to be), the target machine needs to have the same C library that the executable was built with.
One approach would be to dynamically link against the C library that OpenWRT uses. However, an even more robust approach is to link the executable statically. Then it can run on any x86_64 Linux machine, regardless of what libraries it has installed.
Unfortunately, glibc doesn’t really work correctly with fully static linking. Therefore, another C library needs to be used. The best choice seems to be musl. Coincidentally, musl is also the C library used by OpenWRT trunk, but since we’re going to link statically, it wouldn’t have to be.
So, how do we compile and link Haskell programs against musl? One approach is to use a musl version of ghc. Unfortunately, that approach requires a full musl-based Linux system to run on. The author of that post recommends a musl-based Gentoo distribution, but that seemed kind of icky to me. I almost could use the OpenWRT box as the musl-based build system, but that would require a working gcc on the OpenWRT box. Currently, on OpenWRT trunk, the gcc package is broken.
What I really want is a cross-compiler, which will run on my glibc-based Ubuntu box, but will build executables that use statically-linked musl. That, in turn, requires having a cross-compiling toolchain (gcc, binutils, etc.) before building the ghc cross-compiler. There are several ways to satisfy this requirement, but after a recommendation on the haskell-cafe mailing list, I decided to give Crosstool-NG a try.
I discovered that, in addition to the packages already installed on my system, I needed to install these packages in order to get Crosstool-NG to work:
~/src$ sudo apt-get install gperf bison texinfo help2man python-dev
So, I got started with Crosstool-NG:
~/src$ curl -O http://crosstool-ng.org/download/crosstool-ng/crosstool-ng-1.22.0.tar.xz
~/src$ tar -xf crosstool-ng-1.22.0.tar.xz
~/src$ cd crosstool-ng/
~/src/crosstool-ng$ ./configure --prefix=/usr/local
~/src/crosstool-ng$ make
~/src/crosstool-ng$ sudo make install
~/src/crosstool-ng$ mkdir ~/crosstool-ng
~/src/crosstool-ng$ cd ~/crosstool-ng
~/crosstool-ng$ ct-ng x86_64-unknown-linux-gnu
~/crosstool-ng$ ct-ng menuconfig
In the menuconfig, I selected C-library > C library > musl, and it’s also a good idea to de-select Paths and misc options > Render the toolchain read-only, because I’m going to install the GHC cross-compiler in the same directory tree, so the directories need to be writable.
Then, I was able to do:
~/crosstool-ng$ ct-ng build
And a little over an hour later, I could do:
~/x-tools$ export PATH="${PATH}:${HOME}/x-tools/x86_64-unknown-linux-musl/bin"
~/x-tools$ x86_64-unknown-linux-musl-gcc --version
x86_64-unknown-linux-musl-gcc (crosstool-NG crosstool-ng-1.22.0) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions. There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Next, I read the instructions for cross-compiling ghc, and then I did:
~/src$ curl -O http://downloads.haskell.org/~ghc/7.10.3/ghc-7.10.3b-src.tar.xz
~/src$ tar -xf ghc-7.10.3b-src.tar.xz
~/src$ cd ghc-7.10.3/
And then I attempted to build GHC. I ran into some issues along the way, so I’m going to present the fixes to those issues now, rather than reconstructing the exact order in which I did things.
The main issue is that GHC needs ncurses on the target. I’m not sure why, but it does. So, I had to go and cross-compile ncurses:
~/src/ncurses-6.0$ ./configure --host=x86_64-unknown-linux-musl --prefix=${HOME}/x-tools/x86_64-unknown-linux-musl
~/src/ncurses-6.0$ make
Well, that ended up failing with:
x86_64-unknown-linux-musl-gcc -DHAVE_CONFIG_H -I. -I../include -DNDEBUG -O2 --param max-inline-insns-single=1200 -c ../ncurses/lib_gen.c -o ../objects/lib_gen.o
In file included from ./curses.priv.h:325:0,
from ../ncurses/lib_gen.c:19:
_20591.c:843:15: error: expected ')' before 'int'
../include/curses.h:1631:56: note: in definition of macro 'mouse_trafo'
#define mouse_trafo(y,x,to_screen) wmouse_trafo(stdscr,y,x,to_screen)
^
make[1]: *** [../objects/lib_gen.o] Error 1
make[1]: Leaving directory `/home/ppelletier/src/ncurses-6.0/ncurses'
make: *** [all] Error 2
I found the answer in wereHamster’s Dockerfile. In ncurses/base/MKlib_gen.sh
, line 65 needs to be changed from:
preprocessor="$1 -DNCURSES_INTERNALS -I../include"
to:
preprocessor="$1 -P -DNCURSES_INTERNALS -I../include"
Okay, now the make
succeeds, and after make
, I do:
~/src/ncurses-6.0$ make install
However, this is not enough. I thought that by installing it under the cross toolchain’s prefix, gcc would automatically pick up the ncurses includes. But it didn’t. So, I need to specify --with-curses-includes
and --with-curses-libraries
to ghc’s configure
. Unfortunately, there doesn’t seem to be a single value that works for --with-curses-includes
, and it doesn’t seem to be possible to specify more than one value for --with-curses-includes
.
If I use --with-curses-includes=$HOME/x-tools/x86_64-unknown-linux-musl/include
, then the build is unable to find ncurses.h
. However, if I use --with-curses-includes=$HOME/x-tools/x86_64-unknown-linux-musl/include/ncurses
, then ncurses.h
includes ncurses/ncurses_dll.h
, which is not found. My really ugly solution was:
~/x-tools/x86_64-unknown-linux-musl/include/ncurses$ ln -s . ncurses
Now we’re ready to actually build ghc successfully. In the ~/src/ghc-7.10.3/mk
directory, I copied build.mk.sample
to build.mk
, and then uncommented BuildFlavour = quick-cross
. Next I did:
~/src/ghc-7.10.3$ ./configure --target=x86_64-unknown-linux-musl --with-curses-includes=$HOME/x-tools/x86_64-unknown-linux-musl/include/ncurses --with-curses-libraries=$HOME/x-tools/x86_64-unknown-linux-musl/lib --prefix=$HOME/x-tools/x86_64-unknown-linux-musl
~/src/ghc-7.10.3$ make
~/src/ghc-7.10.3$ make install
It turns out that this toolchain still links dynamically by default, so I have to specify -static -optl-static
when compiling Haskell programs. For example:
~/programming/haskell$ x86_64-unknown-linux-musl-ghc -O -static -optl-static contact.hs
[1 of 1] Compiling Main ( contact.hs, contact.o )
Linking contact ...
~/programming/haskell$ file contact
contact: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, not stripped
~/programming/haskell$ ldd contact
not a dynamic executable
The contact
executable can now be run on either my Ubuntu box or my OpenWRT box.