Apparently my last post was not sufficiently detailed for some people. Well, okay, so far only one person, but I am sure I will eventually get a deluge of comments asking for clarification, so I decided to beat them to it :)

The best place to start is at the beginning. IIS exposes a programming interface through DLLs that can be loaded into IIS called ISAPI. There are two flavours of ISAPI, extensions and filters. Filters operate against every request, while extensions target particular file types. Often filters are used to implement things like URL rewriters and gzip encoders, while extensions are used to add new file type handlers like php and asp. Extensions can also handle .* files, which is special IIS lingo for "send all requests to this extension".

The first problem we have is that we need to create a DLL that is loadable by IIS and exposes the expected interface. Luckily Python for Windows Extensions provides a method for doing just that. As part of the distribution, you get PyISAPI_loader.dll which can be found in the isapi module under site packages. This DLL can be copied out into your work folder and renamed with a leading underscore, something like _tgload.dll. When added to your IIS web site and loaded, it will embed python into IIS and load a python file that is named like the DLL but without the underscore (tgload.py).

You can program for either ISAPI filters or extensions with this DLL as the interface that IIS expects does not conflict and is dictated by how you load the DLL. In our case we are creating an extension, so we add the DLL via the application settings section of the tab that might be named one of "virtual directory", "home directory" or just plain "directory". If you haven't already done so you will need to click the "Create" button. Click the "Configuration..." button to open the "Application Configuration" dialog. Under the mappings tab, remove all of the existing application mappings and add a new one. The executable should point to the dll from above, which does not need to and should not exist in the directory that would normally be served by IIS. Set the extension to ".*" in order to catch all URLs, select "All Verbs" and uncheck both "Script Engine" and "Check that file exists".

Programming an ISAPI extension in Python is essentially the same as in C because the isapi module that ships with Python for Windows Extensions does an excellent job of emulating the native interface. Microsoft's documentation applies equally well to Python as it does to C. There are a couple of minor differences which I will document here as well as a couple of tips that will hopefully keep you from pulling your hair out when something goes wrong. First, add the following lines to the top of your file:

import sys
if hasattr(sys, "isapidllhandle"):
    import win32traceutil

This will detect when the file is loaded as an ISAPI extension or filter DLL and redirect stdout and stderr to the win32traceutil output collector. You can run the trace utility by executing python -m win32traceutil in order to see any output from your DLL, including uncaught exception backtraces.

You also need to export the __ExtensionFactory__() function, which returns an object that exposes the GetExtensionVersion, HttpExtensionProc and TerminateExtension methods that operate as described in the Microsoft documentation with the exception that the first variable passed will be self.

Luckily you don't need to worry about the details of how all this works, because ISAPI-WSGI provides this object for you. It translates the ISAPI interface into the Python WSGI interface. If you haven't heard of it before, WSGI is THE standard for connecting Python applications to web servers in all their forms. At this point all of the Python Frameworks talk WSGI so it is a pretty good bet for being able to connect to a Python Web app.

ISAPI-WSGI provides 2 flavours of ISAPI interface objects, ISAPISimpleHandler and ISAPIThreadPoolHandler. The ISAPISimpleHandler can only handle one request at a time, while the ISAPIThreadPoolHandler does not block IIS and offloads the handling of the URL onto a pool of threads that call back into IIS when there is data to transmit back to the client. The exposed interface is identical, so you can go ahead and use whatever your are comfortable with.

Okay so we are going to be returning an instance of ISAPISimpleHandler or ISAPIThreadPoolHandler from our module's __ExtensionFactory__() function. All we need to do is instantiate our choice of object and pass it our WSGI App interface. For TurboGears, all of our HTTP requests are handled by CherryPy so we need to dig into how CherryPy exposes a WSGI App. That is what my previous article is supposed to explain.

All of this should work without a hitch on 32bit Windows, but 64bit opens up a whole big set of problems. You cannot load 32bit DLLs into 64bit applications. There is no official build of the Python for Windows Extensions. I was able to find an old build for Python 2.4 and a current (official) build for Python 2.6, but I found no version for Python 2.5.

Now this does not mean you are dead in the water, IIS 6 (Windows Server 2003 and Windows XP) will let you choose to run IIS in 32bit mode, but everything must run in 32 bit mode. If you want to do this, search for ASP.NET 1.x and Windows x64. If you are sharing the server with other apps that you want to be running in 64bit, like ASP or ASP.NET 2+, you will need to find an alternative deployment method (I am going with TurboGears and IIS behind an Apache reverse proxy). If you are on IIS 7 (Windows Server 2008 and Windows Vista) you can configure individual application pools to run in either 64bit or 32bit mode. I don't have access to an IIS 7 server to try it out. Anyone who can should report back in the comments.

Hopefully that fills the gaps that I left in the last article. I'll leave it to someone else to distill this into newbie friendly documentation that can go on the TurboGears or ISAPI-WSGI Web sites.

On June 19, 2008, Louis wrote to the isapi_wsgi-dev Google Group asking about how to get CherryPy to work with ISAPI-WSGI. Since ISAPI-WSGI was how I was going to connect my Turbo Gears app up to IIS, I recording what I did here for posterity.

The first caveat to this is that you will not get this to work with IIS in 64 bit mode unless you can get a build of PyWin32 for x64. If you running a 64bit Windows architecture you will need to set IIS to 32bit mode and only run 32bit ISAPI dlls. On IIS6 (Windows 2003/XP) this will mean you can only run 32bit DLLs. If you are using IIS7 (Windows 2008), you will, apparently, be able to have 64bit and 32bit process pools. This situation could get better in Python 2.6 since PyWin32 seems to have a x64 build for the 2.6 alphas.

Okay, on to the explanation. The first thing to do in your DLL Python file, is to include these lines:

import sys
if hasattr(sys, "isapidllhandle"):
    import win32traceutil

This checks that we are running as an ISAPI DLL and imports win32traceutil, which redirects stdin and stdout so that you can view them with the win32traceutil message collector (just run the module on its own: python -m win32traceutil). If things go wrong, at least you will have a way of knowing what is going wrong.

My __ExtensionFactory__() looks like this:

def __ExtensionFactory__():
    # Do some pre import setup
    import os

    app_dir = os.path.join(os.path.dirname(__file__), '..', 'app')
    app_dir = os.path.normpath(app_dir)
    sys.path.append(app_dir)
    os.chdir(app_dir)

    # import my app creator
    import wsgi_myapp
    return ISAPIThreadPoolHandler(wsgi_myapp.wsgiApp)

In the do some pre-import setup, we add the application dir to the import path and change directories so that the current working dir is where the TurboGears app is (start-yourproject.py).

Then I have the module that sets up the TurboGears/CherryPy WSGI app. I started with the TurboGears start-yourapp.py file and made modifications that were appropriate for making it work properly with WSGI:

#!c:\Python25\python.exe
from turbogears import config, update_config
import cherrypy
cherrypy.lowercase_api = True

# first look on the command line for a desired config file,
# if it's not on the command line, then
# look for setup.py in this directory. If it's not there, this script is
# probably installed
update_config(configfile="prod.cfg",modulename="yourapp.config")
config.update(dict(package="yourapp"))

from yourapp.controllers import Root

cherrypy.root = Root()
cherrypy.server.start(initOnly=True, serverClass=None)

from cherrypy._cpwsgi import wsgiApp

The CherryPy critical components (for CherryPy 2.2) are:

# Grab your Root object
from yourapp.controllers import Root

# set the root object and initialize the server
cherrypy.root = Root()
cherrypy.server.start(initOnly=True, serverClass=None)

# expose the wsgiApp to be imported by the isapidll.py file above.
from cherrypy._cpwsgi import wsgiApp

Louis is using CherryPy 3 which changes things slightly from what I have done. CherryPy 3 is all WSGI all the time so we need to do less fiddling. Here is what he had for __ExtensionFactory__():

def __ExtensionFactory__():
    try:
        #cpwsgiapp = cherrypy.Application(HelloWorld(),'F:\python-work')
        app = cherrypy.tree.mount(HelloWorld())
        cherrypy.engine.start(blocking=False)
        #wsgi_apps = [('/blog', blog), ('/forum', forum)]

        #server = wsgiserver.CherryPyWSGIServer(('localhost', 8080), HelloWorld(), server_name='localhost')

        return isapi_wsgi.ISAPISimpleHandler(app)
    finally:
        # This ensures that any left-over threads are stopped as well.
        cherrypy.engine.stop()

I left in his extra comments because they show how he got to where he is. This looks like the basic start a server code shown in the tutorial. I think the key issues here are the cherrypy.engine calls. The CherryPy WSGI Wiki page does not mention the engine at all. I would guess that what he actually needs is:

def __ExtensionFactory__():
    app = cherrypy.tree.mount(HelloWorld())
    return isapi_wsgi.ISAPISimpleHandler(app)

That said, I have not used CherryPy 3, so I have no direct experience.

Update: Don't use autoreload with ISAPI-WSGI. It won't work and if you don't use the win32traceutil you won't know why.

Also, I have added more background detail on how to use ISAPI with Python and the ISAPI-WSGI package here.