HOWTO Cross Compile Python 2.5
Recently I needed to compile Python for an embedded target. I used version 2.5 though it looks like that choice may have made it harder for me. I used 2.5 because I didn’t want to have to figure out how to cross compile the cElementTree extension. Unfortunately I still ended up having to figure out how to get PyXML to build for my target. Fortunately I did get everything to work with a bit of fiddling. For posterity, here are some notes about what I did and what problems I had.
I started with Klaus Reimer’s ARM cross-compiling howto and made made some updates required by changes between Python 2.2 and 2.5.
The changes I made are captured in an updated patch to apply against the 2.5 source tree. The changes made the configure.in are to disable rules that cause configure failures when cross compiling because they look for files on the target system or require a test program to be compiled and run. The other changes I added over the patch from Klaus Reimer are for a specific issue I had where md5 and sha hash algorithms were not built because setup.py which builds the modules is not cross compile aware and detected the OpenSSL libraries on my build machine rather than the target.
You can apply the patch with the following command:
lambacck:~/tmp/Python2.5$ patch -p1 < ../Python2.5_xcompile.patch
I next generated a "host" python as in Reimer's instructions:
lambacck:~/tmp/Python2.5$ ./configure
lambacck:~/tmp/Python2.5$ make python Parser/pgen
lambacck:~/tmp/Python2.5$ mv python hostpython
lambacck:~/tmp/Python2.5$ mv Parser/pgen Parser/hostpgen
lambacck:~/tmp/Python2.5$ make distclean
I exported necessary variables like CC, CXX, AR, RANLIB, LD, CFLAGS, CXXFLAGS, LDFLAGS for my target. The key is for these to point to the libraries you are going to use.
To build and “install” python for the my target I used the following commands:
lambacck:~/tmp/Python2.5$ ./configure --host=ppc-linux --build=i686-pc-linux-gnu --prefix=/python
lambacck:~/tmp/Python2.5$ make EXTRA_CFLAGS="$CFLAGS" HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED='ppc-linux-gcc -shared' CROSS_COMPILE=yes
lambacck:~/tmp/Python2.5$ make install EXTRA_CFLAGS="$CFLAGS" HOSTPYTHON=./hostpython BLDSHARED='ppc-linux-gcc -shared' CROSS_COMPILE=yes prefix=/home/lambacck/tmp/dest/python
Note that I needed to use EXTRA_CFLAGS to add my target specific CFLAGS because for some reason the Python configure process does not honor the ones I provided while doing configure. The LDFLAGS variable, however, did work.
Also notice that I set the prefix in the configure step to be /python and the set another prefix in the make install step. The prefix in the configure step is where we are putting python on the file system on the target. The prefix in the install step is where we are putting all of the bits to be able to package them up to be ready to send to the target.
After all that I have a (mostly) functional python environment on my target, but I still needed to get PyXML built. I downloaded the latest distribution, modified setup.py so that expat was forced into little-endian mode and ran the following commands:
lambacck:~/tmp/PyXML-0.8.4$ LDSHARED='ppc-linux-gcc -shared' CC="${CC}" OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ~/tmp/Python-2.5/hostpython -E setup.py build
lambacck:~/tmp/PyXML-0.8.4$ LDSHARED='ppc-linux-gcc -shared' CC="${CC}" OPT='-DNDEBUG -g -O3 -Wall -Wstrict-prototypes' ~/tmp/cross_python/Python-2.5/hostpython -E setup.py install --prefix=$HOME/tmp/dest/python/ --install-scripts=/home/lambacck/tmp/dest/python/bin
It looks like Distutils and the Python build process in general could use some cross compile support. I think this is currently far harder than it has to be.
Update Oct 10, 2005: I would like to point out that on the system that I the executable used to compile hostpython was called gcc and I then updated paths so that a gcc targeted at ppc called gcc was first in my path. Therefore in both cases the compiler was gcc and not gcc for the hostpython and ppc-linux-gcc for the target python.
Update Dec 6, 2006: It looks like double dashes have been turned into single dashes by WordPress. Specifically the host, build, prefix and install-scripts arguments to configure and setup.py should have double dashes.
Update Jun 30, 2010: Translation to Portuguese. Commands converted to use arm processor. Thanks Anuma.
April 15th, 2007 at 8:44 am
Hi,
I need Python 2.5 for my target and I’ve searched a patch and I found your site, so thank you for this. The patch is rightly applied and I haven’t compilation errors. But now I have a other problem, when I try to run Python on my target and import time module I have an error :
Traceback (most recent call last):
File “”, line 1, in
ImportError: File not found
I’m not sure that it’s a Python error (other import, like sys works), but maybe a filesystem error. So I would like to know if you have had this problem or know this ?
Thank you
Christophe
April 15th, 2007 at 9:01 am
I didn’t have that problem, do you have you put it in a proper directory structure on your target?
In my example above, I tell it that python will be installed to the /python directory on the target and then override to $HOME/tmp/dest/python
This is required so that python knows where to look for libraries.
Also note that I setup.py looks for some system libraries in standard places and if you don’t modify it to look in the right place, it could create problems.
-Chris
April 20th, 2007 at 1:41 pm
Hi,
Firstly, thanks for your answer !
I think my problem was due to an incorrectly path. When I try to install Python on my target to /usr/lib/python2.5, it doesn’t work but when I install it on /usr/local/lib/python2.5 it’s ok.
So, now Python and “import time” command work :-) but not module dependences during compilation. I explain : I try to cross-compile Twisted module that needs zope.interface module. I compiled this one and it works on target (“import zope.interface” command is OK) but when I compile Twisted it says that zope.interface is not installed (I try to set PYTHONHOME)
I have also another question : I used “-prefix” argument (like in your example) in the install command line but I have : invalid command name ‘–prefix=/home/christ/buildroot/build_i686/root/usr/local/lib/python’
message, so I try DESTDIR argument at the beginning of the command line, is it correct ? And why it doesn’t work for me (I also try with PyXML-0.8.4 module) ?
Christophe
April 20th, 2007 at 2:15 pm
Twisted is going to be hard. They have a complicated build and install system. You will probably need to hack their setup script into submission :)
Not sure about the prefix problem. Did you use two dashes in front of prefix? Your example looks like only one.
April 20th, 2007 at 2:37 pm
It doesn’t put my mind at ease :-( I don’t know Python at all. I just want to compile it with some module to have Freevo on my target.
Yes, I also try with two dashes.
April 20th, 2007 at 3:17 pm
It gives you an error on install or run? I don’t see anything that would cause an error because it checked for zope.interface on install. I suspect you have a path problem again.
You might need to go and build and install the individual Twisted packages separately. You also probably only need a subset of the packages.
Also check what version of Twisted you need. Freevo says not 2.5 or higher – does that mean less than 2.5?
April 21st, 2007 at 5:27 am
I have the error on build. Just for check, I try Python binary of my system instead of hostpython and I didn’t have the problem of “zope.interface not found”. I think there is configuration error…
April 21st, 2007 at 6:39 am
Sorry, I have also the problem with Python of my system. During compilation tests, zope.interface was installed in /usr/local/lib/python/site-package and haven’t seen it.
April 22nd, 2007 at 10:04 am
There are probably a number of things going on here. The installation programs for Python do not expect or support a cross compile situation. If you are not familiar with Python development, you are likely to run into a lot of problems that you are not going to know how to debug.
You might find things easier if you forget the cross compile thing and try setting everything up in a chroot environment. It looks like your target architecture is i686, which means that this is pretty easily possible.
It is going to be really hard to help you with this without knowing more about what you are doing. I would suggest taking this to comp.lang.python since the wider Python community might be able to help you more.
May 12th, 2007 at 2:58 am
thanks for your article.
following your direction, i compiled python 2.5 successfully, but i encountered a serious problem that the python test suite get blocked when test running into test_asynchat.py. but it’s weird that
it’s ok if i only run “python test_asynchat.py”
did you try to run the test? do you have the same problem?
btw, my cpu is an arm s3c2410.
thanks.
May 12th, 2007 at 9:15 am
I did not try to do any tests. If you are running test, make sure you run them on the target and not the build host.
September 26th, 2007 at 9:02 am
Python for CRIS Etrax…
Using a howto from this site I managed to cross-compile python 2.5.1 for my embedded linux board . Although AXIS provides python 2.4 with their SDK and configures it with no threads and stuff, I compiled the the original python 2.5.1 with only this pat…
October 14th, 2007 at 1:51 am
[...] (http://www.ailis.de/~k/archives/19-ARM-cross-compiling-howto.html#python), Christopher Lambacher (http://whatschrisdoing.com/blog/2006/10/06/howto-cross-compile-python-25/) and Jo Uthus (http://avr32linux.org/twiki/bin/view/Main/Python). Klaus was the first one (that I [...]
October 16th, 2007 at 7:43 am
It worked alright for me.
I used the version 2.5 from 2006-09-04 when checking out the python code from subversion.
December 4th, 2007 at 2:32 pm
Hi Chris,
thanks for the recipe. I try to cross compile Python 2.5 for the D-Link DNS-323 (which is a NAS).
I failed when trying to execute this line:
./configure –host=ppc-linux –build=i686-pc-linux-gnu –prefix=/python
Maybe “ppc-linux” is wrong, I guess for my device the proper setting would be “arm-linux-gcc”. But no matter of the value, I get the message:
configure: error: invalid variable name: –host
Did I miss something?
Bernhard
December 4th, 2007 at 5:31 pm
Hi Chris,
me again. After a look into Reimer’s instructions who used –target= I figured out that the proper argument is –host=
I have Python2.5 running on the DNS-323, thank you again for describing the procedure here.
Cheers,
Bernhard
December 4th, 2007 at 5:39 pm
Okay, now I see what’s the problem here. When entering two consecutive dashes, in the “leave a reply” field on your page, the website will show only one dash.
The proper argument is therefore – -host= (with 2 consecutive dashes). Sorry for messing up your blog. :-)
December 5th, 2007 at 3:23 pm
Hi Chris,
I still have some problems with Python on the DNS-323. After installation (the DNS-323 has no /usr/bin… structure), I can start the Python interpreter. When I try to import the “time” module, it first didn’t find it (the file …/lib/python2.5/lib-dynload/time.so). I expanded the LD_LIBRARY_PATH variable so that the file is found but now I get the following error:
>>> import time
dlopen(“./time.so”, 2);
: can’t resolve symbol ‘__deregister_frame_info’
Do you have any ideas?
Bernhard
December 6th, 2007 at 8:01 am
Bernhard: I have no idea what is going on. I don’t think you want to set the LD_LIBRARY_PATH. Which path on your device did you install python to? If you followed my instructions exactly (i.e. you in the configure step you set prefix to /python) you would have to install the result to /python on the target device. If you don’t, it won’t be able to find libraries. You might be able to fix it by setting PYTHON_PATH to the correct locations.
December 6th, 2007 at 7:41 pm
Chris, I configured and cross-compiled Python once again using “/python” as prefix setting. I installed the resulting files into /mnt/HD_a2/fun_plug.d/python and installed a symbolic link from /python to this directory.
Additionally, I’ve set the variable PYTHONHOME to /python.
I think, the problem isn’t that the library file isn’t found but the file includes a symbol that cannot be resolved. In a former approach I’ve got the information that there is no library named “time”.
The error messaged referred to above is the result of finding the shard object time.so. This file obviously includes a symbol “__deregister_frame_info” that cannot be resolved.
Unfortunately I didn’t find any information about it in the internet so far…
December 7th, 2007 at 8:15 am
__deregister_frame_info looks to be part of GLibC. Are you sure that you are compiling against the version embedded in the DNS-323. Part of the process requires that you do not use the system libraries from the build system.
How did you set-up your cross compile tool chain?
December 9th, 2007 at 10:18 am
Chris, thanks for replying! I cross compiled the tool chain following the instructions given in http://wiki.dns323.info/howto:crosscompile . With the resulting tool chain I cross compiled Python following your instructions (resp. the ones of Reimer).
I also tried to copy the libraries created when building the tool chain onto the DNS-323 but I still didn’t get Python running.
I gave up and chroot’ed into debian (sarge) on my DNS-323. This enables me to do all the things I wanted to. It is a complete debian installation with support of the existing debian packages. There is also a Python (2.4) package available and a lot of additional Python related packages (including PIL).
Although it would be interesting why I didn’t get Python running natively on the DNS-323, the is no need for me to do it anymore.
Thanks again for offering help,
Bernhard
December 11th, 2007 at 1:20 am
[...] saw that its now possible to build python extensions using mingw32 and that people had managed to cross compile python using mingw. Hmm, I pondered….. ‘lets cross compile Python25 using mingw and then build pygobject [...]
March 16th, 2008 at 10:33 pm
Hello,
Via a friend, I know this page. I have a problem , please help me.
First,
Info relate:
OS : uclinux
Platform : unix version 2.4.22 (or 2.4.26)
Chip on board : ARM7
I want to get a python package on network to “make” in linux (Fedora core 1 or anything if it maybe done) and run
python on board with above info .After I made package 2.2.3 and 2.5.2 of Python, I tried run on board such: ./python
with is a script file and get error :
BINFMT_FLAT: bad magic/rev (0×1010100, need 0×4)
BINFMT_FLAT: bad magic/rev (0×1010100, need 0×4)
python: applet not found
I tried with python 2.5.2 package on board chip mips and platform 2.6 ( core 6 ) is ok (although when “make” still
have error but I can run it on this board).
Please help me this problem , show details about it such as : get package in where? number version of python for
fitting. How to configure this package to run good on board? and any info about it .
Thank you so much
March 17th, 2008 at 7:37 am
PhongPham, I don’t exactly know what is wrong, but if you can get Python 2.5.2 working on another board, I would expect that something is wrong with your cross compile environment. Have you been able to build other programs for your ARM7 target? Are you sure that you are passing the right arguments to GCC through the standard make environment variables?
March 18th, 2008 at 9:29 pm
Thank Christopher Lambacher,
I haven’t built other program for your ARM7 target yet. Actually, I don’t sure I pass right arguments to GCC through the standard make environment variables (I’m a newbie).
I will decribe for you :
OS : uclinux
Platform : unix version 2.4.22 (or 2.4.26)
Chip on board : ARM7
I got a package with “.tgz” . After i tar it ( tar zxvf ), will have a file in tared package. I used a other file to call via , in this file I assign some thing such :CC, CXX, AR, RANLIB, LD, CFLAGS, CXXFLAGS, LDFLAGS for my target to run board.
I used:
AS=arm-elf–as
AR=arm-elf–ar
RANLIB=arm-elf–ranlib
STRIP=arm-elf–strip
CC=arm-elf–gcc
CXX=arm-elf–g++
LD=arm-elf–ld
CFLAGS=”-Wl -elf2flt=”-s32768″ -Wa,-elf2flt=”-s32768″ -fno-strict-aliasing -I$INSTALL_DIR/include $CFLAGS_EXTRA”
CXXFLAGS=”-Wl -elf2flt=”-s32768″ -Wa,-elf2flt=”-s32768″ -fno-strict-aliasing -I$INSTALL_DIR/include $CFLAGS_EXTRA”
LDFLAGS=”-L$INSTALL_DIR/lib”
to configure but don’t success.I wonder that these macros is right or wrong. If wrong I need how to modify.
Thanks
March 19th, 2008 at 7:44 am
PhongPham,
Try to get a simple C program running first to make sure that your build environment is working correctly. Python is a complex piece of software and lots of things can go wrong even if your build environment is set up correctly.
You will have to look elsewhere for info on how to set up your build environment for you particular device. If this is for work and not hobby you might want to contract me to help you get started.
March 20th, 2008 at 9:06 pm
Thank Christopher Lambacher ,
I’ll try it as you show me.
I have other question relate with it. Above I remind to environment on Fedora core 6 such “I tried with python 2.5.2 package on board chip mips and platform 2.6 ( core 6 ) is ok (although when “make” still have error:
“/bin/sh: line 2: ./python: cannot execute binary file
make: *** [sharedmods] Error 126 ”
but I can run it on this board ( GCC 4.0.4, binutils2.17, uClibc0.9.28.3)).”
I don’t understand why I can’t import modules : time, random, etc,… but I can import os module and some thing are available. Please explain for me about it, and how to set up environment variable, to import modules for using it in running on board :
./python
: in this script I need import and use modules
Thank you so much !
March 20th, 2008 at 9:38 pm
Sorry ! Error in comment :
Thank Christopher Lambacher ,
I’ll try it as you show me.
I have other question relate with it. Above I remind to environment on Fedora core 6 such “I tried with python 2.5.2 package on board chip mips and platform 2.6 ( core 6 ) is ok (although when “make” still have error:
“/bin/sh: line 2: ./python: cannot execute binary file
make: *** [sharedmods] Error 126 ”
but I can run it on this board ( GCC 4.0.4, binutils2.17, uClibc0.9.28.3)).”
I don’t understand why I can’t import modules : time, random, etc,… but I can import os module and some thing are available. Please explain for me about it, and how to set up environment variable, to import modules for using it in running on board :
./python
: in this script I need import and use modules
Thank you so much !
March 20th, 2008 at 9:39 pm
I don’t see why still error comment :
./python my_script
my_script : in this script I need import and use modules.
March 20th, 2008 at 10:17 pm
PhongPham,
The os module is built into the python runtime. The other modules you mentioned are not. This likely means that you have some problem with your PYTHONPATH. Most likely the –prefix you used in the configure step does not match the path where you copied the installed modules. In my example I set it to /python and copied the resulting tree from the install step there. You need to modify according to your own setup.
March 20th, 2008 at 11:43 pm
Thanks for your reply soon.
I tried by : add a new file have a simple function in Lib folder. When I mount on Python2.5.2 folder which have python app, I can use : from my_file import my_function but when use: from random import Random , I met error:
“from math import log as _log, exp as _exp, pi as _pi, e as _e, ceil as _ceil
ImportError: No module named math ”
So i want use it , I must import all (import math…) ? or How must I do ?. Please guide me details.
Thank you again !
March 21st, 2008 at 1:47 am
I see PYTHONPATH by :
import sys
print str(sys.path)
and gain :
['/working/kenvi/phongpham/2.8.0.0/portcodemono/Python-2.5.2', '/working/kenvi/phongpham/2.8.0.0/portcodemono/Python-2.5.2', '/usr/local/lib/python25.zip', '/working/kenvi/phongpham/2.8.0.0/portcodemono/Python-2.5.2/Lib', '/working/kenvi/phongpham/2.8.0.0/portcodemono/Python-2.5.2/Lib/plat-linux2',
'/working/kenvi/phongpham/2.8.0.0/portcodemono/Python-2.5.2/Lib/lib-tk', '/working/kenvi/phongpham/2.8.0.0/portcodemono/Python-2.5.2/Modules', '/working/kenvi/phongpham/2.8.0.0/portcodemono/Python-2.5.2/build/lib.linux-mips-2.5']
And follow you, What do I miss or mistake when can’t import as above?
Thanks
March 24th, 2008 at 1:38 pm
How could I cross-compile python for the wii? It has a powerpc processor. There’s a .elf compiler for the wii here: http://www.devkitpro.org/
April 7th, 2008 at 10:06 am
[...] a howto from this site I managed to cross-compile python 2.5.1 for my embedded linux board . Although AXIS provides python [...]
May 13th, 2008 at 1:32 pm
[...] cross-compile-aware at all but I did manage to build a working interpreter. Today Kristian found a guide on how to cross-compile the modules as well and so we were able to build a complete Python [...]
June 18th, 2008 at 4:41 am
I cannot cross-compile Python 2.5.2, I have execute the following commands:
~/Chris/Python-2.5.2$ patch -p1 FAILED……
Then I run the following commands:
~/Chris/Python-2.5.2$ ./configure
~/Chris/Python-2.5.2$ make python Parser/pgen
~/Chris/Python-2.5.2$ mv python hostpython
~/Chris/mv Parser/pgen Parser/hostpgen
~Chris/Python-2.5.2$ make distclean
~/Chris/Python-2.5.2$ export CC=arm-linux-gcc
~/Chris/Python-2.5.2$ export CXX=arm-linux-g++
~/Chris/Python-2.5.2$ export AR=arm-linux-ar
~/Chris/Python-2.5.2$ export LD=arm-linux-ld
~/Chris/Python-2.5.2$ export RANLIB=arm-linux-ranlib
~/Chris/Python-2.5.2$ export CFLAGS=
~/Chris/Python-2.5.2$ export CPPFLAGS=-I/opt/emlix/netdcu10/include
~Chris/Python-2.5.2$ export LDFLAGS=-L/opt/emlix/netdcu10/lib
~Chris/Python-2.5.2$ ./configure –target=arm-linux –host=arm-linux –build=i686-pc-linux-gnu –prefix=/python
But then I get the following error because the “./configure” tries to run test; that is:
“checking for %zd printf() format support… configure: error: cannot run test program while cross compilingSee `config.log’ for more details.”
Because of this error no “Makefile” is built and then I cannot execute:
make EXTRA_CFLAGS=”$CFLAGS” HOSTPYTHON=./hostpython HOSTPGEN=./Parser/hostpgen BLDSHARED=’ppc-linux-gcc -shared’ CROSS_COMPILE=yes
Please, could anybody tell me what I am doing wrong? Thanks in advance
June 18th, 2008 at 4:53 am
Sorry in my previous post, the first command was (I misspelled it)
patch -p1 < ../Python2.5_xcompile.pat
And the result was:
patching file configure
Hunk 1 FAILED at 1.
Hunk 2 FAILED at 984.
— bla, bla, a lot of Hunk failed—–
231 out of 231 hunks FAILED — saving rejects to file configure.rej
patching file configure.in
Hunk 1 FAILED at 3342.
1 out of 1 hunk FAILED — saving rejects to file configure.in.rej
patching file Makefile.pre.in
Hunk 1 succeeded at 173 (offset 3 lines).
Hunk 2 succeeded at 201 (offset 3 lines).
Hunk 3 succeeded at 351 (offset 3 lines).
Hunk 4 FAILED at 476.
Hunk 5 succeeded at 784 (offset 9 lines).
Hunk 6 succeeded at 897 (offset 9 lines).
1 out of 6 hunks FAILED — saving rejects to file Makefile.pre.in.rej
patching file setup.py
Hunk 3 succeeded at 578 (offset 1 line).
Hunk 4 succeeded at 595 (offset 1 line).
June 18th, 2008 at 6:56 am
Genar: There were some changes in 2.5.2 that make the patch not apply cleanly. I believe that if you follow the comment above yours to dholm.com they had it working with 2.5.2.
You can also look in the rej files to see what failed and try to apply the patches manually.
-Chris
August 12th, 2008 at 12:00 pm
Hi Chris.
Great article. I too am looking for the 2.5.2 patch. I’ve poked around everywhere but couldn’t find it. I tried the dholm.com address above but it’s a dead link now.
Any ideas?
BTW: I’m cross compiling for a mips64 bit Cavium Processor.
I’m pretty good (at least I think I am, as he pats himself on the back) at cross compiling. I must admit, this python stuff is a PAIN to cross compile. :)
August 13th, 2008 at 11:02 am
I am trying to follow the instructions from “K’s clutter loft”.
I just don’t know the configure scripting to get the ./configure script to NOT do the “checking for %zd printf() format support” test.
Any pointers there? Do you know that scripting? Once I get that done I think it will work.
Thanks again.
September 9th, 2009 at 5:38 am
Hi Chris,
I need to do cross compile Python 3.1 in Linux environment. I couldn’t find any patch for Python3.1. Do you have patch for Python3.1?
Thanks & Regards,
Anand
September 9th, 2009 at 7:03 am
Sorry,
I have not had the opportunity to work with this on Python 3.1.
September 11th, 2009 at 5:58 am
Hi Chris,
Thank you for your reply. I want to know how complex is the work to create a patch for any Python version. For example If i want to create a patch for Python3.1, is it a very cumbersome process? Does this work require a speacial expertise? If possible please explain me about your approach in doing this. So that i can try for myself.
Thanks & regards,
Anand
October 9th, 2009 at 1:25 am
I don’t want to steal traffic, but I’ve posted a 3.1 patch to my site:
http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html
October 16th, 2009 at 11:07 pm
[...] instructions work both with Open Embedded and the DENX ELDK. If you need to go cross compile Python 2.5, 2.6 or 3.1. ELDK and Open Embedded include a version of Python pre-done for you. You will need a [...]
June 30th, 2010 at 9:07 pm
Hi ,
I did a translation of your tutorial for portuguese and for xScale platform. I put a reference for your blog !
Thanks by this tutorial !
[Ed Note: Perma-Link]