Comments for What's Chris Doing? http://whatschrisdoing.com/blog Programming, photography, music and life. Tue, 08 May 2012 14:07:33 +0000 hourly 1 http://wordpress.org/?v=3.3.1 Comment on Cross Compiling Python Extensions by Yegor Yefremov http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-522 Yegor Yefremov Tue, 08 May 2012 14:07:33 +0000 http://whatschrisdoing.com/blog/?p=92#comment-522 Chris, do you have an idea, how to solve this issue: https://groups.google.com/d/topic/comp.lang.python/0rDWl2d8PDY/discussion Thanks Chris,

do you have an idea, how to solve this issue: https://groups.google.com/d/topic/comp.lang.python/0rDWl2d8PDY/discussion

Thanks

]]> Comment on Cross Compiling Python Extensions by Yegor Yefremov http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-509 Yegor Yefremov Tue, 03 Jan 2012 12:08:16 +0000 http://whatschrisdoing.com/blog/?p=92#comment-509 Chris: are you going to provide your slides for PyCon before the event itself? Chris: are you going to provide your slides for PyCon before the event itself?

]]> Comment on Cross Compiling Python Extensions by Yegor Yefremov http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-507 Yegor Yefremov Thu, 29 Dec 2011 14:19:00 +0000 http://whatschrisdoing.com/blog/?p=92#comment-507 I've got it!!! The first problem with checks was, that setup.py while checking for headers has also tried to execute the binary. That was not so good to execute arm binary on x86 ;-) The second is trickier as it concerns library paths. During linking the only -L option I see is -L/usr/lib despite specifying LDFLAGS. This output of the linker_so at the biginning of setup.py: Linker_so: ['/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc', '-shared', '-L/home/user/Documents/projects/br-queue-python/output/host/usr/arm-unknown-linux-gnueabi/sysroot/lib', '-L/home/user/Documents/projects/br-queue-python/output/host/usr/arm-unknown-linux-gnueabi/sysroot/usr/lib'] at the end I solved this by providing library_dirs myself. Any idea? I wish you good luck for your PyCon talk. And I hope Python devs will understand the difference between compilation and cross-compilation. And some common framework linke autotools should be also available, so there is no need to write own checks. Thanks for helping. Happy New Year! I’ve got it!!!

The first problem with checks was, that setup.py while checking for headers has also tried to execute the binary. That was not so good to execute arm binary on x86 ;-)

The second is trickier as it concerns library paths. During linking the only -L option I see is -L/usr/lib despite specifying LDFLAGS. This output of the linker_so at the biginning of setup.py:

Linker_so: ['/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc', '-shared', '-L/home/user/Documents/projects/br-queue-python/output/host/usr/arm-unknown-linux-gnueabi/sysroot/lib', '-L/home/user/Documents/projects/br-queue-python/output/host/usr/arm-unknown-linux-gnueabi/sysroot/usr/lib']

at the end I solved this by providing library_dirs myself. Any idea?

I wish you good luck for your PyCon talk. And I hope Python devs will understand the difference between compilation and cross-compilation. And some common framework linke autotools should be also available, so there is no need to write own checks.

Thanks for helping. Happy New Year!

]]> Comment on Cross Compiling Python Extensions by Yegor Yefremov http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-505 Yegor Yefremov Thu, 29 Dec 2011 08:49:45 +0000 http://whatschrisdoing.com/blog/?p=92#comment-505 No. That didn't help. define PYTHON_NETIFACES_BUILD_CMDS (cd $(@D); \ CC="$(TARGET_CC)" \ CFLAGS="$(TARGET_CFLAGS)" \ PYTHONXCPREFIX="$(HOST_DIR)/usr/" \ LDSHARED="$(TARGET_CC) -shared" \ LDFLAGS="-L$(HOST_DIR)/lib -L$(HOST_DIR)/usr/lib" \ $(HOST_DIR)/usr/bin/python setup.py build -x) endef Build result: python-netifaces 0.6 Building (cd /home/user/Documents/projects/br-queue-python/output/build/python-netifaces-0.6; CC="/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc" CFLAGS="-pipe -Os -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64" PYTHONXCPREFIX="/home/user/Documents/projects/br-queue-python/output/host/usr/" LDSHARED="/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc -shared" LDFLAGS="-L/home/user/Documents/projects/br-queue-python/output/host/lib -L/home/user/Documents/projects/br-queue-python/output/host/usr/lib" /home/user/Documents/projects/br-queue-python/output/host/usr/bin/python setup.py build -x) running build Setting prefix Setting prefix running build_ext checking for getifaddrs... not found. (cached) checking for getnameinfo... not found. (cached) checking for socket IOCTLs... not found. (cached) checking for optional header files... netash/ash.h netatalk/at.h netax25/ax25.h neteconet/ec.h netipx/ipx.h netpacket/packet.h linux/irda.h linux/atm.h linux/llc.h linux/tipc.h linux/dn.h. (cached) checking whether struct sockaddr has a length field... no. (cached) checking which sockaddr_xxx structs are defined... at ax25 in in6 ipx un ash ec ll atmpvc atmsvc dn irda llc. (cached) building 'netifaces' extension /home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -pipe -Os -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -fPIC -DHAVE_NETASH_ASH_H=1 -DHAVE_NETATALK_AT_H=1 -DHAVE_NETAX25_AX25_H=1 -DHAVE_NETECONET_EC_H=1 -DHAVE_NETIPX_IPX_H=1 -DHAVE_NETPACKET_PACKET_H=1 -DHAVE_LINUX_IRDA_H=1 -DHAVE_LINUX_ATM_H=1 -DHAVE_LINUX_LLC_H=1 -DHAVE_LINUX_TIPC_H=1 -DHAVE_LINUX_DN_H=1 -DHAVE_SOCKADDR_AT=1 -DHAVE_SOCKADDR_AX25=1 -DHAVE_SOCKADDR_IN=1 -DHAVE_SOCKADDR_IN6=1 -DHAVE_SOCKADDR_IPX=1 -DHAVE_SOCKADDR_UN=1 -DHAVE_SOCKADDR_ASH=1 -DHAVE_SOCKADDR_EC=1 -DHAVE_SOCKADDR_LL=1 -DHAVE_SOCKADDR_ATMPVC=1 -DHAVE_SOCKADDR_ATMSVC=1 -DHAVE_SOCKADDR_DN=1 -DHAVE_SOCKADDR_IRDA=1 -DHAVE_SOCKADDR_LLC=1 -I/home/user/Documents/projects/br-queue-python/output/host/usr/include/python2.7 -c netifaces.c -o build/temp.linux-i686-2.7/netifaces.o netifaces.c:143:6: error: #error You need to add code for your platform. netifaces.c:268:1: warning: 'string_from_sockaddr' defined but not used netifaces.c:360:1: warning: 'add_to_family' defined but not used error: command '/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc' failed with exit status 1 I have such a premonition, that CC won't be honored. For the test I just renamed the gcc binary and build_ext was not complaining during the test. The only thing that failed was netifaces.c compilation, because of absent compiler binary. The test_build method in netifaces setup.py has tree return possibilities: - no error - CompileError - DistutilsExecError But since the return value is boolean I don't know which part failed. Print is not visible in this method. Will have to dig deeper. No. That didn’t help.

define PYTHON_NETIFACES_BUILD_CMDS
(cd $(@D); \
CC=”$(TARGET_CC)” \
CFLAGS=”$(TARGET_CFLAGS)” \
PYTHONXCPREFIX=”$(HOST_DIR)/usr/” \
LDSHARED=”$(TARGET_CC) -shared” \
LDFLAGS=”-L$(HOST_DIR)/lib -L$(HOST_DIR)/usr/lib” \
$(HOST_DIR)/usr/bin/python setup.py build -x)
endef

Build result:

python-netifaces 0.6 Building
(cd /home/user/Documents/projects/br-queue-python/output/build/python-netifaces-0.6; CC=”/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc” CFLAGS=”-pipe -Os -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64″ PYTHONXCPREFIX=”/home/user/Documents/projects/br-queue-python/output/host/usr/” LDSHARED=”/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc -shared” LDFLAGS=”-L/home/user/Documents/projects/br-queue-python/output/host/lib -L/home/user/Documents/projects/br-queue-python/output/host/usr/lib” /home/user/Documents/projects/br-queue-python/output/host/usr/bin/python setup.py build -x)
running build
Setting prefix
Setting prefix
running build_ext
checking for getifaddrs… not found. (cached)
checking for getnameinfo… not found. (cached)
checking for socket IOCTLs… not found. (cached)
checking for optional header files… netash/ash.h netatalk/at.h netax25/ax25.h neteconet/ec.h netipx/ipx.h netpacket/packet.h linux/irda.h linux/atm.h linux/llc.h linux/tipc.h linux/dn.h. (cached)
checking whether struct sockaddr has a length field… no. (cached)
checking which sockaddr_xxx structs are defined… at ax25 in in6 ipx un ash ec ll atmpvc atmsvc dn irda llc. (cached)
building ‘netifaces’ extension
/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes -pipe -Os -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -D_FILE_OFFSET_BITS=64 -fPIC -DHAVE_NETASH_ASH_H=1 -DHAVE_NETATALK_AT_H=1 -DHAVE_NETAX25_AX25_H=1 -DHAVE_NETECONET_EC_H=1 -DHAVE_NETIPX_IPX_H=1 -DHAVE_NETPACKET_PACKET_H=1 -DHAVE_LINUX_IRDA_H=1 -DHAVE_LINUX_ATM_H=1 -DHAVE_LINUX_LLC_H=1 -DHAVE_LINUX_TIPC_H=1 -DHAVE_LINUX_DN_H=1 -DHAVE_SOCKADDR_AT=1 -DHAVE_SOCKADDR_AX25=1 -DHAVE_SOCKADDR_IN=1 -DHAVE_SOCKADDR_IN6=1 -DHAVE_SOCKADDR_IPX=1 -DHAVE_SOCKADDR_UN=1 -DHAVE_SOCKADDR_ASH=1 -DHAVE_SOCKADDR_EC=1 -DHAVE_SOCKADDR_LL=1 -DHAVE_SOCKADDR_ATMPVC=1 -DHAVE_SOCKADDR_ATMSVC=1 -DHAVE_SOCKADDR_DN=1 -DHAVE_SOCKADDR_IRDA=1 -DHAVE_SOCKADDR_LLC=1 -I/home/user/Documents/projects/br-queue-python/output/host/usr/include/python2.7 -c netifaces.c -o build/temp.linux-i686-2.7/netifaces.o
netifaces.c:143:6: error: #error You need to add code for your platform.
netifaces.c:268:1: warning: ‘string_from_sockaddr’ defined but not used
netifaces.c:360:1: warning: ‘add_to_family’ defined but not used
error: command ‘/home/user/Documents/projects/br-queue-python/output/host/usr/bin/arm-none-linux-gnueabi-gcc’ failed with exit status 1

I have such a premonition, that CC won’t be honored. For the test I just renamed the gcc binary and build_ext was not complaining during the test. The only thing that failed was netifaces.c compilation, because of absent compiler binary.

The test_build method in netifaces setup.py has tree return possibilities:
- no error
- CompileError
- DistutilsExecError

But since the return value is boolean I don’t know which part failed. Print is not visible in this method. Will have to dig deeper.

]]> Comment on Cross Compiling Python Extensions by Christopher Lambacher http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-504 Christopher Lambacher Wed, 28 Dec 2011 21:10:02 +0000 http://whatschrisdoing.com/blog/?p=92#comment-504 Yegor: It looks like you are missing -L options in your LDFLAGS. It should be LDFLAGS="-L$(HOST_DIR)/lib/ -L$(HOST_DIR)/usr/lib/". Yegor: It looks like you are missing -L options in your LDFLAGS. It should be LDFLAGS=”-L$(HOST_DIR)/lib/ -L$(HOST_DIR)/usr/lib/”.

]]> Comment on Cross Compiling Python Extensions by Yegor Yefremov http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-503 Yegor Yefremov Wed, 28 Dec 2011 08:14:01 +0000 http://whatschrisdoing.com/blog/?p=92#comment-503 Hi Chris, I've a problem cross-compiling netifaces from Alastair Houghton for ARM using Buildroot. I've installed distutilscross and specified all needed environment variables, but setuptools 0.6c11 doesn't seem to be interested in specified CC. The -x option will be recognized, but all netifaces tests fail: running build Setting prefix Setting prefix running build_ext checking for getifaddrs... not found. (cached) checking for getnameinfo... not found. (cached) checking for socket IOCTLs... not found. (cached) That is how I configure netifaces for cross-compiling: define PYTHON_NETIFACES_BUILD_CMDS (cd $(@D);\ CC="$(TARGET_CC) -pthread" \ LDSHARED="$(TARGET_CC) -pthread -shared" \ PYTHONXCPREFIX="$(HOST_DIR)/usr/" \ LDFLAGS="$(HOST_DIR)/lib/ $(HOST_DIR)/usr/lib/" \ $(HOST_DIR)/usr/bin/python setup.py build -x) endef Do you have any idea? Thank you in advance and merry Xmas and Happy New Year! P.S. such statement as --plat-name=linux-arm will be not recognized Hi Chris,

I’ve a problem cross-compiling netifaces from Alastair Houghton for ARM using Buildroot. I’ve installed distutilscross and specified all needed environment variables, but setuptools 0.6c11 doesn’t seem to be interested in specified CC. The -x option will be recognized, but all netifaces tests fail:

running build
Setting prefix
Setting prefix
running build_ext
checking for getifaddrs… not found. (cached)
checking for getnameinfo… not found. (cached)
checking for socket IOCTLs… not found. (cached)

That is how I configure netifaces for cross-compiling:

define PYTHON_NETIFACES_BUILD_CMDS
(cd $(@D);\
CC=”$(TARGET_CC) -pthread” \
LDSHARED=”$(TARGET_CC) -pthread -shared” \
PYTHONXCPREFIX=”$(HOST_DIR)/usr/” \
LDFLAGS=”$(HOST_DIR)/lib/ $(HOST_DIR)/usr/lib/” \
$(HOST_DIR)/usr/bin/python setup.py build -x)
endef

Do you have any idea?

Thank you in advance and merry Xmas and Happy New Year!

P.S. such statement as –plat-name=linux-arm will be not recognized

]]> Comment on Cross Compiling Python Extensions by James Le Cuirot http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-502 James Le Cuirot Wed, 28 Dec 2011 00:14:55 +0000 http://whatschrisdoing.com/blog/?p=92#comment-502 Hi Chris. Sorry again for the noise but I've just made an important discovery. Whilst patching this feature into Python directly, I wondered where the default values for PREFIX and EXEC_PREFIX come from. It turns out you can already override these by setting the PYTHONHOME environment variable. The catch is that all the Python stuff you need to complete the build (e.g. setuptools) needs to be installed in the prefix as the only files it touches from the host are the native Python executable and library. If the build needs to execute any native Python extensions then this probably won't work but I doubt any Python build system does this. In the Gentoo case, it will automatically install prerequisites like setuptools for you. I'll play with this more tomorrow. Hi Chris. Sorry again for the noise but I’ve just made an important discovery. Whilst patching this feature into Python directly, I wondered where the default values for PREFIX and EXEC_PREFIX come from. It turns out you can already override these by setting the PYTHONHOME environment variable. The catch is that all the Python stuff you need to complete the build (e.g. setuptools) needs to be installed in the prefix as the only files it touches from the host are the native Python executable and library. If the build needs to execute any native Python extensions then this probably won’t work but I doubt any Python build system does this. In the Gentoo case, it will automatically install prerequisites like setuptools for you. I’ll play with this more tomorrow.

]]> Comment on Cross Compiling Python Extensions by Christopher Lambacher http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-501 Christopher Lambacher Mon, 26 Dec 2011 22:07:32 +0000 http://whatschrisdoing.com/blog/?p=92#comment-501 I'll be doing a talk on this stuff at PyCon 2012: https://us.pycon.org/2012/schedule/presentation/11/ I’ll be doing a talk on this stuff at PyCon 2012: https://us.pycon.org/2012/schedule/presentation/11/

]]> Comment on Cross Compiling Python Extensions by Christopher Lambacher http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-500 Christopher Lambacher Mon, 26 Dec 2011 22:06:36 +0000 http://whatschrisdoing.com/blog/?p=92#comment-500 Hi James, setuptools is required for distutils. You should be able to get non-setuptools packages to work using: python -c "import setuptools; execfile('setup.py')" -x build You can substitute any other command for build as you require (including bdist_egg). Hi James, setuptools is required for distutils. You should be able to get non-setuptools packages to work using:

python -c “import setuptools; execfile(‘setup.py’)” -x build

You can substitute any other command for build as you require (including bdist_egg).

]]> Comment on Cross Compiling Python Extensions by James Le Cuirot http://whatschrisdoing.com/blog/2009/10/16/cross-compiling-python-extensions/comment-page-1/#comment-499 James Le Cuirot Mon, 26 Dec 2011 21:07:17 +0000 http://whatschrisdoing.com/blog/?p=92#comment-499 Ah it works if the package imports setuptools instead of distutils. I was trying it with rdiff-backup, which imports distutils. I gather entry points are specific to setuptools. Is there any way to make it magically work in this situation? Ah it works if the package imports setuptools instead of distutils. I was trying it with rdiff-backup, which imports distutils. I gather entry points are specific to setuptools. Is there any way to make it magically work in this situation?

]]>