Building Python extensions for Windows with only free tools

April 19, 2007 - 11:26pm — frank

Prior to Python 2.5, trying to create Python extension modules for Windows with only free tools was an extremely frustrating experience. Python has traditionally been (and still is) compiled with proprietary Microsoft tools, making it difficult to interoperate with gcc. Your options were:

Trying to get MinGW to work by using this very painful method.
For a while, Microsoft offered a free set of basic compiler tools that could be used to create extensions (it was still fairly difficult, but possible). Unfortunately, Microsoft withdrew the free tools fairly soon after Python 2.4 came out.
Recompile Python with MinGW. In theory this could work (I never made it ...) but then you are locked into your own MinGW-only world, and cannot interoperate with the standard Python distribution.

With Python 2.5, free (as in open source!) tools are finally supported without having to perform any weird voodoo on the standard distribution. You have two basic options:

MinGW
Cygwin with its mingw package.

Each method has its pros and cons:

With MinGW, you work with standard DOS/Windows tools (i.e. a "DOS box"). With Cygwin you work inside a POSIX/Linux-like environment. This is a matter of personal taste.
There are a lot of libraries that assume you're either on a Windows box with Visual C++ or in a full POSIX environment with gcc. Very few (I've found) support MinGW natively. Cygwin lets you compile libraries that require the full ./configure; make; make install process. MinGW provides a minimal POSIX environment if you install their MSYS package.
Cygwin is painless to keep up-to-date - simply run its setup.exe and it will automatically find updates. MinGW is harder to keep up to date and figure out which packages you need (stable vs. candidate vs. snapshot).
MinGW produces only Windowns-native binaries (which in the current discussion is a good thing). You can do the same with Cygwin, but you have to double-check to make sure a package didn't get linked with any Cygwin DLL (which would make it non-distributable).

My recommendation:

If you plan to only compile Python modules and not compile any other C/C++ libraries or programs, go with MinGW. (Or, if you are more comfortable in a DOS/Windows environment, then choose MinGW.)
Use Cygwin if you routinely compile non-Python libraries that require the POSIX ./configure; make; make install steps, or if you feel more comfortable in a POSIX environment.

I've used both, and both are excellent tools, it really comes down to personal preference and a few technical issues. Both methods are shown below.

For brevity, I've used my personal folder names below, i.e. c:\frank, etc. There is nothing special about these names; adjust them to suit your preference.

The Cygwin way

If you don't already have Cygwin installed, go to their site to get started, or see my notes on installing Cygwin
Install Python 2.5 to c:\frank\py25
Run cygwin-setup and add the following packages:
- gcc-mingw
- gcc-mingw-core
- gcc-mingw-g++
- make
  This will pull in several other packages.
Continue with configuring distutils

The MinGW way

Go to the MinGW site and download the following packages (I recommend getting the "Candidate" instead of the "Stable" versions):
- gcc-core
- gcc-g++
- binutils
- runtime
- utils
- w32api
Unpack all of them to a folder, e.g. c:\frank\mingw.
Add c:\frank\mingw\bin to your PATH (My Computer ➛ Properties ➛ Advanced ➛ Environment Variables).

Configuring distutils

You now have to tell Python to use the Cygwin/MinGW compiler when building extensions. To do this, create a file c:\frank\py25\lib\distutils\distutils.cfg:

[build]
compiler = mingw32

This same file works for both Cygwin and MinGW setups.

Update: 2008-09-10

Recent versions of Cygwin binutils have version numbers that are breaking the version number parsing, resulting in errors like:

ValueError: invalid version number '2.18.50.20080625'

To fix this, edit distutils/version.py. At line 100, replace:

version_re = re.compile(r'^(\d+) \. (\d+) (\. (\d+))? ([ab](\d+))?$',
re.VERBOSE)

with

version_re = re.compile(r'^(\d+) \. (\d+) (\. (\d+))? (\. (\d+))?$',
re.VERBOSE)

Trying it out

Download a Python package that needs a compiler, e.g. PyCrypto
From a shell or DOS box, unpack and chdir to the unpacked directory.
python setup.py bdist_wininst
This will create an installer:
dist/pycrypto-2.0.1.win32-py2.5.exe

You can also just do python setup.py install here, but I have multiple win32 machines, and instead of setting up the development environment on each one, I prefer to make one .exe and copy it to the other machines.

For MinGW (assuming it worked), you are done!

Cygwin Sanity Check

The point of setting up Cygwin as shown above is to be able to create Python extensions that do not rely on the Cygwin runtime (so they can be freely redistributed). You can check this by using objdump on the .pyd files that were created, e.g.:

objdump -p build/lib.win32-2.5/Crypto/Cipher/AES.pyd|grep DLL

This should show:

    DLL
vma:            Hint    Time      Forward  DLL       First
    DLL Name: msvcr71.dll
    DLL Name: python25.dll

If you ever see a name beginning with cyg, for example cygwin1.dll, STOP!. This means you've linked in part of Cygwin, and your binary package is no longer freely redistributable. (Passing the --host= and --build= flags to gcc should alleviate this, see notes below.)

Building non-Python libraries (Cygwin)

When building a non-Python library for win32 under Cygwin, for a package that uses the standard ./configure; make; make install method, I use the following recipe:

./configure \
      --prefix=/cygdrive/c/frank/mingw_libs \
      CC="gcc -mno-cygwin" \
      CXX="g++ -mno-cygwin" \
      --host=i686-pc-mingw32 \
      --build=i686-pc-cygwin

Explanation:

--prefix=/cygdrive/c/frank/mingw_libs: I install all my mingw libraries using c:\frank\mingw_libs as the root directory. This keeps them separated from the rest of the system (don't want to accidentally mix them with Cygwin libs!). I use /cygdrive/c/frank/mingw_libs instead of c:/frank/mingw_libs since not all scripts understand the c:/ notation.

When possible, I try to build static libraries, but if you are building (and subsequently linking to) DLLs, make sure to add e.g. c:\frank\mingw_libs\bin to your (Windows) PATH.
CC= and CXX=: I always set both the C and C++ compilers so it doesn't try to default to Cygwin's gcc/g++.
--host= and --build=: These lines let gcc know to configure itself as a cross-compiler. This should keep it from attempting to link with any Cygwin libraries, since (as a cross-compiler) it assumes they are incompatible.

Conclusion

Python 2.5 finally supports creating extensions with free tools. While the debate may continue over why a free language like Python requires a proprietary compiler, for the time being at least, you can stop worrying about it and start compiling!

Written in WikklyText.

Comments

June 29, 2009 - 2:56pm — Kevin Horn (not verified)

Invalid Version Number w/ MinGW

Just wanted to mention that the "invalid version number" problem mentioned in the article also applies to MinGW (at least for my setup).

The same patch to distutils/version.py works to fix the problem.

February 27, 2009 - 4:37am — bharling (not verified)

Thankyou so much

Have been looking for an alternative MSVC for creating extensions for years! Your example worked perfectly first time using cygwin. Thanks!

January 28, 2009 - 4:31am — Anonymous (not verified)

building extension with psyco

please i try to us this format to build psyco, but i got the error message

C:\psyco>python setup.py bdist_wininst PROCESSOR = 'i386' running bdist_wininst running build running build_py creating build creating build\lib.win32-2.5 creating build\lib.win32-2.5\psyco copying py-support\init.py -> build\lib.win32-2.5\psyco copying py-support\classes.py -> build\lib.win32-2.5\psyco copying py-support\core.py -> build\lib.win32-2.5\psyco copying py-support\kdictproxy.py -> build\lib.win32-2.5\psyco copying py-support\logger.py -> build\lib.win32-2.5\psyco copying py-support\profiler.py -> build\lib.win32-2.5\psyco copying py-support\support.py -> build\lib.win32-2.5\psyco running build_ext Traceback (most recent call last): File "setup.py", line 180, in <module>

kwds )

File "C:\Python25\lib\distutils\core.py", line 151, in setup dist.run_commands() File "C:\Python25\lib\distutils\dist.py", line 974, in run_commands self.run_command(cmd) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "C:\Python25\lib\distutils\command\bdist_wininst.py", line 107, in run self.run_command('build') File "C:\Python25\lib\distutils\cmd.py", line 333, in run_command self.distribution.run_command(command) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "C:\Python25\lib\distutils\command\build.py", line 112, in run self.run_command(cmd_name) File "C:\Python25\lib\distutils\cmd.py", line 333, in run_command self.distribution.run_command(command) File "C:\Python25\lib\distutils\dist.py", line 994, in run_command cmd_obj.run() File "C:\Python25\lib\distutils\command\build_ext.py", line 264, in run force=self.force) File "C:\Python25\lib\distutils\ccompiler.py", line 1175, in new_compiler return klass (None, dry_run, force) File "C:\Python25\lib\distutils\cygwinccompiler.py", line 292, in init CygwinCCompiler.init (self, verbose, dry_run, force) File "C:\Python25\lib\distutils\cygwinccompiler.py", line 84, in init get_versions() File "C:\Python25\lib\distutils\cygwinccompiler.py", line 401, in get_versions

from distutils.version import StrictVersion File "C:\Python25\lib\distutils\version.py", line 100 version_re = re.compile(r'^(\d+) \. (\d+) (\. (\d+))? (\. (\d+))?$', ^ IndentationError: unindent does not match any outer indentation level

please how can i get off this?

Written in WikklyText.

November 2, 2008 - 11:44pm — Yuzisee (not verified)

Automatically installing DLLs

setup(name=packagename,
      version='0.09',
      data_files = [('.',['../project/Release/myDLL.dll'])],
      ext_modules = [module1])

I have included the entry: data_files = [('.',['../project/Release/myDLL.dll'])],

Now, when you run python setup.py install, The ../project/Release/myDLL.dll will be copied to your Python Root.

If you wanted instead to copy myDLL.dll to the DLLs folder within the python root, change this line to data_files = [('DLLs',['../project/Release/myDLL.dll'])],

If you wanted to copy multiple files to the DLLs folder within the python root, change this line to data_files = [('DLLs',['../project/Release/myDLL.dll','../project/Release/myOtherDLL.dll'])],

With this line in place, your bdist_wininst will automatically install your DLL for you.

For more information, visit: http://docs.python.org/distutils/setupscript.html?highlight=data_files#installing-additional-files

Written in WikklyText.

September 13, 2008 - 7:27pm — Tim (not verified)

Error During Test Build

I did the MinGW install and tried to build pycrypto using the install method. But, it failed saying that gcc could not find a file or directory. I noticed that the command it failed did -IC:\Python25\PC. I do not have this PC directory in my python directory. Am I missing something? Do I need to install something?

July 25, 2008 - 7:44am — Basil (not verified)

Excellent post - please clarify "Build non-Python" part

This is an excellent resource for a POSIX newbie like me, because recently I've been trying to compile libmcrypt and its python-mcrypt extension for win32 with little success.

I have the Cygwin install, and libmcrypt compiles and installs perfectly *IFF* I use default (i.e. Cygwin-dependent) settings during ./configure. That's if I just use "./configure" without any extra switches. When I try to follow your recipe for the configure part, the subsequent "make" step fails on the first source file with the message "unknown definition: '_rpl_malloc'." I say that I "try" to follow because I am not completely sure what the "mingw_libs" dir is supposed to contain. Is it just an independent copy of the "C:/mingw/lib" dir, or is it the root MinGW install dir, i.e. "C:/MinGW" ?

My second question is: let's say that I do build libmcrypt successfully without the cygwin dependencies using ./configure; make; make install like you say here. What's the next step in distutils? It appears that cygwin creates ".deps" and ".lib" subdirs throughout the project when it compiles. Those dirs contain a handful of .dll and .a files, and some others. I assume these are necessary for distutils to create the final .dll and .lib for distribution, but how do I do that part?

p.s. I do not have to create the libpython25.a file because I am using a binary install of Python 2.5 on WinXP, and 2.5 already ships with the lib.

Your help is much appreciated,
Basil

Written in WikklyText.

July 26, 2008 - 11:32am — Basil (not verified)

Progress update

Following your cool recipe for ./configure, I ran into a problem during make: undefined reference to '_rpl_malloc'. I scratched my head and googled it. It turns out that this is due to a bug in autoconf. I followed the advice for the workaround described here and now I am able to fully compile mcrypt in Cygwin!

For this to work, I had to enter the following two commands just prior to running ./configure:

export ac_cv_func_malloc_0_nonnull=yes
export ac_cv_func_realloc_0_nonnull=yes

Although disabling 'realloc' is not described anywhere, I took a guess at it after make complained about it, and it seemed to work.

The second part is building the Python extension in distutils using the resulting DLL. The DLL for libmcrypt is created by Cygwin in C:\py\extend\libmcrypt-2.5.8\lib\.libs as "libmcrypt-4.dll", with a companion "export definitions" file suitably named "libmcrypt-4.dll.def". The DEF file is just a text file containing the names exported by the DLL. The next step is to create an import library "mcrypt.lib" that statically links to the DLL through the DEF file. In my case, I used a tool in VS 2003 called "lib.exe" in the "Vc7\bin" folder to generate "mcrypt.lib" by following these instructions. Once I had the .lib file, I ran:

setup.py build bdist_wininst

which builds the binary installer for python-mcrypt. After running this installer, one has to manually copy the DLL file to the Python path, otherwise DLL loading will fail in Python during import mcrypt. If there is a way for the DLL to be packaged and installed by the installer automatically, please, please let me know. Likewise, if there is a way to build the installer using MinGW alone, I'd like to hear it as well. (After all, this post is all about the free tools!)

I hope my notes here will help someone out there who may experience these problems so that they can work around them instead of throwing up their hands in frustration. Really, libmcrypt is a very fine library, and it's a shame that its maintainers decided to drop support for win32 builds.

Again, frank, thanks for this very useful tutorial!
Basil

Written in WikklyText.

July 26, 2008 - 5:06pm — frank

mcrypt update

Nice writeup! I was just about to try and investigate this, glad to see you solved it. Thanks for contributing your solution.

I'm not really sure how to bundle the DLLs automatically. For the libraries I've built so far, I've wimped out and used static linking. But I think your solution is the way to go.

July 27, 2008 - 4:06pm — Basil (not verified)

Next challenge - mhash

I guess I beat you to that solution, but your thought counts too :-) It was a fun several days to figure that one out.

But I am currently not having as much success with mhash, the companion library to mcrypt. Really, I suppose I could resort to using PyCrypto's hashing features, but mhash is just newer/more modern.

Anyway, it looks like the authors of mhash did not provide for cross-platform portability to Windows very well. There is much more difficulty to get it to compile even under Cygwin; then once it's compiled, distutils has even more trouble with it, regarding include files etc.. I have all but given up on this one.

Perhaps this is common trouble with non-Python libraries: if they are not meant for win32 distribution, very little can be done by a *NIX newbie to get them to work on such a platform..

July 27, 2008 - 7:59pm — frank

mhash

I think you are right that this is a common problem. I ran into the same thing with (I think it was) libxml2. It seems many packages understand two configurations — (a) pure win32 with MS VS ($$) and (b) pure POSIX with gcc. Its a shame that the official Python keeps getting out of sync with the freely available Visual Studio Express, that would make things so much easier.

It sounds like in the mhash case, though, its not even compatible with Windows, so that's an even harder problem.

(By the way, this whole compatibility thing can get very political, which I think is a shame. For example, libiconv (FSF version) is deliberately incompatible with MS VS and will only work with mingw. So now you have the reverse problem as above, and (as far as I can tell) it was done purely to make a political point.)

July 28, 2008 - 12:21am — Basil (not verified)

mhash hope

Bummer. My "last stand" on this is to try to compile it with mingw. I get the following errors (besides a few warnings from the include files, which I choose to ignore for now):

mhash.c:230: error: initializer element is not constant
mhash.c:230: error: (near initialization for `MHASH_Type.tp_getattro')
mhash.c:231: error: initializer element is not constant
mhash.c:231: error: (near initialization for `MHASH_Type.tp_setattro')
mhash.c:250: error: initializer element is not constant
mhash.c:250: error: (near initialization for `MHASH_Type.tp_alloc')
mhash.c:251: error: initializer element is not constant
mhash.c:251: error: (near initialization for `MHASH_Type.tp_new')
mhash.c:252: error: initializer element is not constant
mhash.c:252: error: (near initialization for `MHASH_Type.tp_free')

A search on this reveals that it is because I don't have the Unix-y "libpython25.a" static library. But I have it in my C:\Python25\libs! It ships with Python 2.5 now.

This may be my last light of hope at the end of this tunnel, or just another compatibility freight train trying to run me down.

Any thoughts before I abandon beating my brain against the bricks? :-)

Written in WikklyText.

July 28, 2008 - 6:11am — frank

mhash

Ugh, "initializer element is not constant". I have no idea why this error pops up so much, but usually when I see errors like this (and a quick scan of the C code shows no obvious problem), I bail out. Sorry I don't know any tricks here!

December 13, 2007 - 4:18pm — cwu (not verified)

Error on winXP

Thanks for the excellent post.
I have followed your MinGW way on my winXP machine (activestate python 2.5.1). When I built pyXML, I got the errors attached below.
It looks like the the problem is at the linking with python libraries, but I have verified "-LC:\Python25\libs" is used and "C:\Python25\libs\python25.lib" exists. Did I miss something?

Thanks.

E:\Downloads\PyXML-0.8.4>python setup.py build
running build
running build_py
running build_ext
building '_xmlplus.parsers.pyexpat' extension
creating build\temp.win32-2.5
creating build\temp.win32-2.5\Release
creating build\temp.win32-2.5\Release\extensions
creating build\temp.win32-2.5\Release\extensions\expat
creating build\temp.win32-2.5\Release\extensions\expat\lib
X:\app\win32\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -DXML_NS=1 -DXML_DTD=1
-DBYTEORDER=1234 -DXML_CONTEXT_BYTES=1024 -DHAVE_MEMMOVE=1 -DXML_STATIC= -Iexte
nsions/expat/lib -IC:\Python25\include -IC:\Python25\PC -c extensions/pyexpat.c
-o build\temp.win32-2.5\Release\extensions\pyexpat.o
X:\app\win32\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -DXML_NS=1 -DXML_DTD=1
-DBYTEORDER=1234 -DXML_CONTEXT_BYTES=1024 -DHAVE_MEMMOVE=1 -DXML_STATIC= -Iexte
nsions/expat/lib -IC:\Python25\include -IC:\Python25\PC -c extensions/expat/lib/
xmlparse.c -o build\temp.win32-2.5\Release\extensions\expat\lib\xmlparse.o
X:\app\win32\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -DXML_NS=1 -DXML_DTD=1
-DBYTEORDER=1234 -DXML_CONTEXT_BYTES=1024 -DHAVE_MEMMOVE=1 -DXML_STATIC= -Iexte
nsions/expat/lib -IC:\Python25\include -IC:\Python25\PC -c extensions/expat/lib/
xmlrole.c -o build\temp.win32-2.5\Release\extensions\expat\lib\xmlrole.o
X:\app\win32\MinGW\bin\gcc.exe -mno-cygwin -mdll -O -Wall -DXML_NS=1 -DXML_DTD=1
-DBYTEORDER=1234 -DXML_CONTEXT_BYTES=1024 -DHAVE_MEMMOVE=1 -DXML_STATIC= -Iexte
nsions/expat/lib -IC:\Python25\include -IC:\Python25\PC -c extensions/expat/lib/
xmltok.c -o build\temp.win32-2.5\Release\extensions\expat\lib\xmltok.o
writing build\temp.win32-2.5\Release\extensions\pyexpat.def
X:\app\win32\MinGW\bin\gcc.exe -mno-cygwin -shared -s build\temp.win32-2.5\Relea
se\extensions\pyexpat.o build\temp.win32-2.5\Release\extensions\expat\lib\xmlpar
se.o build\temp.win32-2.5\Release\extensions\expat\lib\xmlrole.o build\temp.win3
2-2.5\Release\extensions\expat\lib\xmltok.o build\temp.win32-2.5\Release\extensi
ons\pyexpat.def -LC:\Python25\libs -LC:\Python25\PCbuild -lpython25 -lmsvcr71 -o
build\lib.win32-2.5\_xmlplus\parsers\pyexpat.pyd
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x1b1): undef
ined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x1b8): undef
ined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x1f6): undef
ined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x1fd): undef
ined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x22b): undef
ined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x232): more
undefined references to `_imp___Py_NoneStruct' follow
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x643): undef
ined reference to `_imp___PyThreadState_Current'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x688): undef
ined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x241a): unde
fined reference to `_imp__PyString_Type'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2421): unde
fined reference to `_imp__PyString_Type'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x244c): unde
fined reference to `_imp__PyExc_TypeError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x247b): unde
fined reference to `_imp__PyExc_ValueError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x251e): unde
fined reference to `_imp__PyFile_Type'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2525): unde
fined reference to `_imp__PyFile_Type'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x257c): unde
fined reference to `_imp__PyExc_TypeError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x25ff): unde
fined reference to `_imp__PyExc_IOError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x273b): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2743): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x27ff): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2809): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2aba): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2ac2): more
undefined references to `_imp___Py_NoneStruct' follow
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2c3b): unde
fined reference to `_imp__PyExc_RuntimeError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2e12): unde
fined reference to `_imp___Py_TrueStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2e19): unde
fined reference to `_imp___Py_ZeroStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x2e50): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3149): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3150): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3519): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3597): unde
fined reference to `_imp__PyExc_RuntimeError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x37a8): unde
fined reference to `_imp__PyExc_AttributeError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x38eb): unde
fined reference to `_imp__PyExc_ValueError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3905): unde
fined reference to `_imp___Py_NoneStruct'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x393e): unde
fined reference to `_imp__PyDict_Type'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3945): unde
fined reference to `_imp__PyDict_Type'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3966): unde
fined reference to `_imp__PyExc_TypeError'
build\temp.win32-2.5\Release\extensions\pyexpat.o:pyexpat.c:(.text+0x3acd): unde
fined reference to `_imp__PyType_Type'
collect2: ld returned 1 exit status
error: command 'gcc' failed with exit status 1

January 3, 2008 - 5:37pm — cwu (not verified)

linking to python25 works for me now

Thanks for the response. Finally I got it work for me. The trick is to generate a "libpython25.a" file using pexports and dlltool (as described in many pages, e.g. above "this very painful method" link, step 2.). Or, this file can be downloaded from this link libpython25.a directly. By doing that, both cygwin and mingw ways work for me.

I have an impression that the requirement of "libpython2x.a" is not needed for new version of MinGW and python25. Is that the case for you two, Frank and Enrico? But apparently it does not work for me.

OK, finally, this is the list of everything I am using:

ActivePython 2.5.1
MinGW [installed using MinGW-5.1.3.exe, with gcc (GCC) 3.4.5 (mingw special) and GNU ld version 2.17.50 20060824]
cygwin/gcc-mingw [gcc (GCC) 3.4.4 (cygming special, gdc 0.12, using dmd 0.125) and GNU ld version 2.17.50 20060817]
pexports v0.43

Thanks you all.

Written in WikklyText.

June 28, 2008 - 12:34am — Eli Bendersky (not verified)

fresh libpython25.a

Hello
I've also followed this method with success. However, the link to the .a file here, is dead, so you can use this one.

January 4, 2008 - 8:32am — frank

Having to generate libpython25.a

Correct, I did not have to generate libpython25.a myself. I notice you are using ActivePython - could that have something do with it? Just a guess.

December 21, 2007 - 11:39am — Enrico (not verified)

linking pyton25.lib solved

Hello.

I had a similar problem trying to compile my own library. Not sure it is the same as what you describe, but I solved it by

- updating mingw using the update application in my xp start menu - making sure I was using the pre-compiled version of python rather than the one that I compiled myself (using VC++8 — ouch!)

Not sure if the update was necessary or if it was just a problem due to trying to link to the wrong version.

Frank: Thanks to the great instructions! I wish I found them a few hours earlier.. :-)

Enrico

December 14, 2007 - 9:00am — frank

PyXML errors

Hmm ... I tried this myself (I'm setup via Cygwin) and it worked. I remember when I used to use MinGW that I had to always get the latest bleeding-edge tools to make everything work. That's what annoyed me about MinGW and why I quit using it. I don't know if that helps any, but just wanted to note that it works for me on Cygwin (which is gcc (GCC) 3.4.4 (cygming special, gdc 0.12, using dmd 0.125)).

July 29, 2007 - 5:56pm — Tim (not verified)

Great stuff

Awesome post! The difficulty of building Python extensions on Win32 has been irritating me for years. I usually use the ActiveState distro, but msvccompiler.py doesn't seem to like my Visual Studio 2005 (and their distro is compiled with VC7, and compiling with two different VC's is aparently asking for trouble in the linker).

Anyway, I've kept my ActiveState, but now have a Cygwin just for building extensions. What kept me from doing this before was the worry about linking in cygwin, but your post showed how to avoid that, so thanks!

July 18, 2007 - 2:33pm — Mike Powers (not verified)

Thank You

This posting was very useful. I had tried to use MinGW before but I never got the configuration of distutils right. Now I have to remember all those packages I tried to install before and go get them!

July 18, 2007 - 7:52am — Bill Mill (not verified)

Python 2.4 supported building extensions with free tools too

I wrote about it almost exactly a year ago: http://billmill.org/compile_python_on_cygwin

July 18, 2007 - 3:57pm — frank

Hard way

Yep, that is the "painful" method noted in the links above. With Python 2.5, the import libs are already created for you. This painful method may still be required for other 3rd party libraries that refuse to compile under mingw/cygwin, but I'm not currently using any such libraries.

July 11, 2007 - 3:30am — ErikW (not verified)

Additional Vista info:

The info is great. Some more comments if you use Vista and MinGW:

Install MinGW directly under the root of your hard drive (i.e. C:\mingw)
You need to add references for the lib and libexe to path. I do this though a batch file (see below)
In the same batch file I also set the environmental variables for the search paths to include files and libraries.
The batch file was found somewhere on the internets, but I've forgotten where.
This worked for PyCrypto under Vista Ultimate, but YMMW ;-)

@echo off

echo MinGW Enviroment Command Console
echo ....
echo ....
@set MINGWROOT=C:\MinGW
@set MINGWBIN=%MINGWROOT%\bin
@set MINGWINCLUDE=%MINGWROOT%\include
@set MINGWLIB=%MINGWROOT%\lib
@set MINGWLIBEXEC=%MINGWROOT%\libexec\gcc\mingw32\3.4.5
@set MINGWBIN2=%MINGWROOT%\mingw32\bin
@set MINGWLIB2=%MINGWROOT%\mingw32\lib\ldscripts
@set MINGW=%MINGWROOT%;%MINGWBIN%;%MINGWINCLUDE%;%MINGWLIB%;%MINGWLIBEXEC%;%MINGWLIB2%;%MINGWLIB2%

rem to add more resource paths just use the set command like above and the and it to the set PATH like below

@set Path=%MINGW%;%Path%

Written in WikklyText.

May 24, 2007 - 12:34pm — er (not verified)

this was very helpful

thanks for putting this info up!

May 19, 2007 - 5:46pm — jaco (not verified)

great work

at last now compiling stuff on win32 using mingw is easy, great article

thanks