I just posted lots of pictures to my Picture Gallery. There are some from our trip to Florida and some from Zane's welcome party and some random pictures of Amara. I'll also post a few choice picks to my Flickr Stream.

We recently launched the CIOC Community Site running on Plone. While we have not actually had any SPAM on the site yet, we continually get automated SPAM registrations that we then need to go and clear out. Last night Kate got fed up, so I decided to see if I could put a CAPTCHA on the form.

Looks like there is a proposal to include CAPTCHA support. The de facto standard CAPTCHA tool for Plone seems to be collective.captcha, but I wanted to be able to use the ReCAPTCHA service.

Luckily there is collective.recaptcha which is in beta and, while not listing any releases on the Plone site, is available from the Cheese Shop. But, alas, there is no documentation. It is supposed to be a drop in replacement for collective.captcha so with a little sleuthing and I got it to work. Here's how.

<div class="formControls"></div>

<div class="field"
     tal:define="error errors/captcha|nothing;"
     tal:attributes="class python:test(error, 'field error', 'field')">
  <label for="captcha" i18n:translate="label_captcha">Captcha</label>

  <span class="fieldRequired" title="Required"
        i18n:attributes="title"
        i18n:domain="plone"
        i18n:translate="label_required">(Required)</span>

  <div class="formHelp" i18n:translate="help_captcha">
    Provide the text in the image. Just to avoid spambots
  </div>
  <p tal:replace="structure here/@@captcha/image_tag" />

</div>

captcha = context.REQUEST.get('recaptcha_response_field')
view = context.restrictedTraverse('@@captcha')

if not view.verify(captcha):
    state.setError('captcha', _(u'Are you a bot? Try again...'))
    state.set(status='failure')

Yesterday Kate and I had a very indulgent day. The Winterlicious promotion is happening in Toronto right now and we decided to take full advantage of it. We booked a lunch and dinner reservation and then built a day in Toronto around it.

First we went to Barootes for lunch. I had the Chicken, Leek and Barley soup while Kate had the Baby Organic Greens with Apple Cider Vinaigrette. The we both had Penne with Spicy Italian sausage in Basil Tomato Sauce and finished it off with their "Special of the Week" which was Chocolate Layer Cake. Yummy.

Once we left there we walked to the AGO to check out the new building and have a look through the new collections. The walk was a chilly at -8C but we got there quick enough that we did completely loose the feeling in our extremities (about 10-15 minutes).

The new AGO building is a welcome change. It makes the building much more interesting and the extra exhibit space made for the opportunity to see many works that were locked away in the vaults before. Add the new works, and even if you were very familiar with what used to be on display, I am sure you will find something new to see. The new centrepiece staircase goes all the way up to the 5th floor and is outside the main structure of the building between the 4th and 5th floors similar to the staircase on the other side of the building. Disappointingly, they are having a major problem with condensation on the glass between the 4th and 5th floors of the staircase. They had to close that section of the staircase and the water is damaging the dry wall in places. After all the money and effort that went into the transformation I hope they will be able to find a quick solution to the problem and it is just a matter of working out the kinks.

After about 4 hours at the AGO, it was time for the cold walk back to King St for our dinner at Marcel's Bistro. Both Kate and I had the Salad to start, followed by Pork tenderloin in a tomato sauce with mashed potatoes & seasonal vegetables. We shared a 1/2L of the Italian Terrazze Della Luna Pinot Grigio 2007. For desert Kate had Vanilla ice cream with hot chocolate sauce in puff pastry while I had the chocolate mouse cake. Again yummy, but I think Barootes was better.

To complete the day, we had tickets to Dirty Dancing. We had excellent seats, 3rd row, first balcony center which we got for the ridiculously low price of $30 each. Thank you Red Flag Deals. The play was good, but not your traditional the story line is sung kind of musical. It was more like a play with a lot of music and dancing in it. If you liked the movie, you'll probably like the play.

In all it was a very long, fun, expensive but high value day.

Apparently my last post was not sufficiently detailed for some people. Well, okay, so far only one person, but I am sure I will eventually get a deluge of comments asking for clarification, so I decided to beat them to it :)

The best place to start is at the beginning. IIS exposes a programming interface through DLLs that can be loaded into IIS called ISAPI. There are two flavours of ISAPI, extensions and filters. Filters operate against every request, while extensions target particular file types. Often filters are used to implement things like URL rewriters and gzip encoders, while extensions are used to add new file type handlers like php and asp. Extensions can also handle .* files, which is special IIS lingo for "send all requests to this extension".

The first problem we have is that we need to create a DLL that is loadable by IIS and exposes the expected interface. Luckily Python for Windows Extensions provides a method for doing just that. As part of the distribution, you get PyISAPI_loader.dll which can be found in the isapi module under site packages. This DLL can be copied out into your work folder and renamed with a leading underscore, something like _tgload.dll. When added to your IIS web site and loaded, it will embed python into IIS and load a python file that is named like the DLL but without the underscore (tgload.py).

You can program for either ISAPI filters or extensions with this DLL as the interface that IIS expects does not conflict and is dictated by how you load the DLL. In our case we are creating an extension, so we add the DLL via the application settings section of the tab that might be named one of "virtual directory", "home directory" or just plain "directory". If you haven't already done so you will need to click the "Create" button. Click the "Configuration..." button to open the "Application Configuration" dialog. Under the mappings tab, remove all of the existing application mappings and add a new one. The executable should point to the dll from above, which does not need to and should not exist in the directory that would normally be served by IIS. Set the extension to ".*" in order to catch all URLs, select "All Verbs" and uncheck both "Script Engine" and "Check that file exists".

Programming an ISAPI extension in Python is essentially the same as in C because the isapi module that ships with Python for Windows Extensions does an excellent job of emulating the native interface. Microsoft's documentation applies equally well to Python as it does to C. There are a couple of minor differences which I will document here as well as a couple of tips that will hopefully keep you from pulling your hair out when something goes wrong. First, add the following lines to the top of your file:

import sys
if hasattr(sys, "isapidllhandle"):
    import win32traceutil

This will detect when the file is loaded as an ISAPI extension or filter DLL and redirect stdout and stderr to the win32traceutil output collector. You can run the trace utility by executing python -m win32traceutil in order to see any output from your DLL, including uncaught exception backtraces.

You also need to export the __ExtensionFactory__() function, which returns an object that exposes the GetExtensionVersion, HttpExtensionProc and TerminateExtension methods that operate as described in the Microsoft documentation with the exception that the first variable passed will be self.

Luckily you don't need to worry about the details of how all this works, because ISAPI-WSGI provides this object for you. It translates the ISAPI interface into the Python WSGI interface. If you haven't heard of it before, WSGI is THE standard for connecting Python applications to web servers in all their forms. At this point all of the Python Frameworks talk WSGI so it is a pretty good bet for being able to connect to a Python Web app.

ISAPI-WSGI provides 2 flavours of ISAPI interface objects, ISAPISimpleHandler and ISAPIThreadPoolHandler. The ISAPISimpleHandler can only handle one request at a time, while the ISAPIThreadPoolHandler does not block IIS and offloads the handling of the URL onto a pool of threads that call back into IIS when there is data to transmit back to the client. The exposed interface is identical, so you can go ahead and use whatever your are comfortable with.

Okay so we are going to be returning an instance of ISAPISimpleHandler or ISAPIThreadPoolHandler from our module's __ExtensionFactory__() function. All we need to do is instantiate our choice of object and pass it our WSGI App interface. For TurboGears, all of our HTTP requests are handled by CherryPy so we need to dig into how CherryPy exposes a WSGI App. That is what my previous article is supposed to explain.

All of this should work without a hitch on 32bit Windows, but 64bit opens up a whole big set of problems. You cannot load 32bit DLLs into 64bit applications. There is no official build of the Python for Windows Extensions. I was able to find an old build for Python 2.4 and a current (official) build for Python 2.6, but I found no version for Python 2.5.

Now this does not mean you are dead in the water, IIS 6 (Windows Server 2003 and Windows XP) will let you choose to run IIS in 32bit mode, but everything must run in 32 bit mode. If you want to do this, search for ASP.NET 1.x and Windows x64. If you are sharing the server with other apps that you want to be running in 64bit, like ASP or ASP.NET 2+, you will need to find an alternative deployment method (I am going with TurboGears and IIS behind an Apache reverse proxy). If you are on IIS 7 (Windows Server 2008 and Windows Vista) you can configure individual application pools to run in either 64bit or 32bit mode. I don't have access to an IIS 7 server to try it out. Anyone who can should report back in the comments.

Hopefully that fills the gaps that I left in the last article. I'll leave it to someone else to distill this into newbie friendly documentation that can go on the TurboGears or ISAPI-WSGI Web sites.

On June 19, 2008, Louis wrote to the isapi_wsgi-dev Google Group asking about how to get CherryPy to work with ISAPI-WSGI. Since ISAPI-WSGI was how I was going to connect my Turbo Gears app up to IIS, I recording what I did here for posterity.

The first caveat to this is that you will not get this to work with IIS in 64 bit mode unless you can get a build of PyWin32 for x64. If you running a 64bit Windows architecture you will need to set IIS to 32bit mode and only run 32bit ISAPI dlls. On IIS6 (Windows 2003/XP) this will mean you can only run 32bit DLLs. If you are using IIS7 (Windows 2008), you will, apparently, be able to have 64bit and 32bit process pools. This situation could get better in Python 2.6 since PyWin32 seems to have a x64 build for the 2.6 alphas.

Okay, on to the explanation. The first thing to do in your DLL Python file, is to include these lines:

import sys
if hasattr(sys, "isapidllhandle"):
    import win32traceutil

This checks that we are running as an ISAPI DLL and imports win32traceutil, which redirects stdin and stdout so that you can view them with the win32traceutil message collector (just run the module on its own: python -m win32traceutil). If things go wrong, at least you will have a way of knowing what is going wrong.

My __ExtensionFactory__() looks like this:

def __ExtensionFactory__():
    # Do some pre import setup
    import os

    app_dir = os.path.join(os.path.dirname(__file__), '..', 'app')
    app_dir = os.path.normpath(app_dir)
    sys.path.append(app_dir)
    os.chdir(app_dir)

    # import my app creator
    import wsgi_myapp
    return ISAPIThreadPoolHandler(wsgi_myapp.wsgiApp)

In the do some pre-import setup, we add the application dir to the import path and change directories so that the current working dir is where the TurboGears app is (start-yourproject.py).

Then I have the module that sets up the TurboGears/CherryPy WSGI app. I started with the TurboGears start-yourapp.py file and made modifications that were appropriate for making it work properly with WSGI:

#!c:\Python25\python.exe
from turbogears import config, update_config
import cherrypy
cherrypy.lowercase_api = True

# first look on the command line for a desired config file,
# if it's not on the command line, then
# look for setup.py in this directory. If it's not there, this script is
# probably installed
update_config(configfile="prod.cfg",modulename="yourapp.config")
config.update(dict(package="yourapp"))

from yourapp.controllers import Root

cherrypy.root = Root()
cherrypy.server.start(initOnly=True, serverClass=None)

from cherrypy._cpwsgi import wsgiApp

The CherryPy critical components (for CherryPy 2.2) are:

# Grab your Root object
from yourapp.controllers import Root

# set the root object and initialize the server
cherrypy.root = Root()
cherrypy.server.start(initOnly=True, serverClass=None)

# expose the wsgiApp to be imported by the isapidll.py file above.
from cherrypy._cpwsgi import wsgiApp

Louis is using CherryPy 3 which changes things slightly from what I have done. CherryPy 3 is all WSGI all the time so we need to do less fiddling. Here is what he had for __ExtensionFactory__():

def __ExtensionFactory__():
    try:
        #cpwsgiapp = cherrypy.Application(HelloWorld(),'F:\python-work')
        app = cherrypy.tree.mount(HelloWorld())
        cherrypy.engine.start(blocking=False)
        #wsgi_apps = [('/blog', blog), ('/forum', forum)]

        #server = wsgiserver.CherryPyWSGIServer(('localhost', 8080), HelloWorld(), server_name='localhost')

        return isapi_wsgi.ISAPISimpleHandler(app)
    finally:
        # This ensures that any left-over threads are stopped as well.
        cherrypy.engine.stop()

I left in his extra comments because they show how he got to where he is. This looks like the basic start a server code shown in the tutorial. I think the key issues here are the cherrypy.engine calls. The CherryPy WSGI Wiki page does not mention the engine at all. I would guess that what he actually needs is:

def __ExtensionFactory__():
    app = cherrypy.tree.mount(HelloWorld())
    return isapi_wsgi.ISAPISimpleHandler(app)

That said, I have not used CherryPy 3, so I have no direct experience.

Update: Don't use autoreload with ISAPI-WSGI. It won't work and if you don't use the win32traceutil you won't know why.

Also, I have added more background detail on how to use ISAPI with Python and the ISAPI-WSGI package here.