Wednesday, July 14, 2010

Cross compiling the Linux kernel from Mac OS X

So I picked up a 13" MacBook and have been fiddling around with it. I like it, sue me.

One of the first things I did (as any Linux developer would) was to install darwin ports. I noticed some interesting things in there. A few that I needed (git) and a few that completely surprised me (dpkg and apt).

One thing that was missing was a Linux cross-compiler. So I did what any self-respecting Linux developer on a Mac would do: I built one.

Don't get too excited. I've only built one worthy of compiling a kernel (which means no C library, no userspace, etc).

The result of my work is here (built on 10.6.3):


You may notice the extra elf.h file, which is needed in /usr/include/elf.h for some programs in the kernel to compile natively on the host (e.g. modpost). The gcc and binutils will unpack in /opt/local/.

In order to cross-compile, you will need to add a few things to your kernel make command line:

make ARCH=x86_64 CROSS_COMPILE=x86_64-linux-gnu- ...

You may notice, like I did, scripts/genksyms/parse.c has a #include for malloc.h which is not on Darwin. You may safely delete that line.

Note that you must already have /opt/local/bin in your PATH. Using ARCH=i386 will also work and compile 32-bit kernels. One last point, the sources for gcc/binutils came from Ubuntu's Jaunty.

Happy hacking...

15 comments:

  1. http://www.macports.org/ports.php?by=name&substr=i386-elf-gcc

    Darwinports is dead and out-of-date btw. Use macports.

    ReplyDelete
  2. Or are you cross-compiling linux kernel? i'm confused

    ReplyDelete
  3. Ah, sorry I am using macports, and yes, I am compiling the kernel cross-compile from macosx :)

    ReplyDelete
  4. What is your development environment like?

    I just got a new macbook pro. I am thinking of using QEMU to test my kernels and Eclipse to do the development...

    ReplyDelete
  5. I downloaded your tools. I tried compiling the kernel and I got this message just before creating the bzImage:


    arch/x86/boot/header.S: Assembler messages:
    arch/x86/boot/header.S:236: Error: can't resolve `VO__end' {*UND* section} - `VO__text' {*UND* section}
    make[1]: *** [arch/x86/boot/header.o] Error 1

    Do you have any idea why would this happen? I am on 10.6.4, trying to compile kernel 2.6.35.2.

    FYI I also get this a few lines before:

    MODPOST vmlinux.o
    WARNING: modpost: Found 45 section mismatch(es).
    To see full details build your kernel with:
    'make CONFIG_DEBUG_SECTION_MISMATCH=y'

    make: *** [bzImage] Error 2

    ReplyDelete
  6. @Rafael, My environment is pretty simple. Just the toolchain I posted, and a Linux box. I'm using this mostly for test building modules, so I've only done 1 or 2 full kernel compiles just to test the toolchain.

    It sounds to me like you have some whacked out options set. Try a stock kernel source tree with "make defconfig" and see what that gets you.

    ReplyDelete
  7. I encounter Rafael Fernandez's problem too,and i use 'make defconfig' and no change.

    ReplyDelete
  8. @Anonymous What is the command you actually use to compile it?

    ReplyDelete
  9. I used 'make ARCH=x86_64 CROSS_COMPILE=x86_64-linux-gnu-'

    I still haven't figured out how to compile it. I tried also installing libelf.

    ReplyDelete
  10. Just want to say *thanks* for elf.h. I was beginning to think that I needed to get a VM to cross compile it but this was all that was standing in my way. Thank you thank you thank you.

    ReplyDelete
  11. Have you seen this error during "LD .tmp_vmlinux1":

    kernel/built-in.o: In function `alloc_pid':elfcore.c:(.text+0x1a17c): undefined reference to `wrong_size_cmpxchg'

    My make cmd from MacBook is:

    127 sudo make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi- menuconfig
    128 sudo make ARCH=arm CROSS_COMPILE=arm-none-linux-gnueabi-

    ReplyDelete
  12. @Rafael Fernandez (and for my future reference, took me some hours to figure out)

    arch/x86/boot/header.S: Assembler messages:
    arch/x86/boot/header.S:236: Error: can't resolve `VO__end' {*UND* section} - `VO__text' {*UND* section}
    make[1]: *** [arch/x86/boot/header.o] Error 1

    This is an indication that zoffset.h is empty because your sed does wrong things. Use GNU sed.

    ReplyDelete
  13. I posted a script that generates the first stage cross compiler here: http://www.noondaysystems.com/. I used gcc-4.5.2 but it auto switches to whatever version you download.

    A second stage is needed to build anything other than a kernel, and Ben Collins' notes above are needed to build a working linux kernel.

    The second stage, or Ben's notes, basically mean putting together the files that create a linux installation on your mac. It's not all that bad, really. But there is a sort of chicken-and-egg problem for things like the header "elf.h".

    Thanks, Ben, for posting this!

    ReplyDelete
  14. Something else was expected here!

    CC kernel/bounds.s
    dyld: Library not loaded: /opt/local/lib/libintl.8.dylib

    mmmm, how could I solve this? thanks!

    ReplyDelete
  15. Hey there,
    thanks for this really useful stuff! I'm trying to compile some module for an 32-intel architecture but I can't figure out what's the problem. I think something is missing but I don't have a clue.. may you give me a hand? Thanks. Here is the chunk of output I obtain when I try to compile:


    mobertuz:os bertuz$ make ARCH=i386 CROSS_COMPILE=x86_64-linux-gnu- -C ~/Desktop/os/linux-2.6.19 M=/Users/bertuz/Desktop/os/module_test modules
    /bin/sh: line 1: 2198 Trace/BPT trap: 5 x86_64-linux-gnu-gcc -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -m32 -S -o /dev/null -xc /dev/null > /dev/null 2>&1
    /bin/sh: line 1: 2201 Trace/BPT trap: 5 x86_64-linux-gnu-gcc -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -pipe -msoft-float -mpreferred-stack-boundary=2 -S -o /dev/null -xc /dev/null > /dev/null 2>&1
    /bin/sh: line 1: 2203 Trace/BPT trap: 5 x86_64-linux-gnu-gcc -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -pipe -msoft-float -mtune=i386 -S -o /dev/null -xc /dev/null > /dev/null 2>&1
    /bin/sh: line 1: 2205 Trace/BPT trap: 5 x86_64-linux-gnu-gcc -Wall -Wundef -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -O2 -pipe -msoft-float -falign-functions=0 -S -o /dev/null -xc /dev/null > /dev/null 2>&1
    dyld: Library not loaded: /opt/local/lib/libintl.8.dylib
    Referenced from: /opt/local/bin/x86_64-linux-gnu-gcc
    Reason: image not found

    cheers!
    Matteo

    ReplyDelete