Making HTTPS Requests secure in Python

October 30, 2011 at 05:07 PM | categories: Programming | View Comments

Python 2.7 does not automatically validate certificates on HTTPS requests. The following seems to be the cleanest, though not necessarily fully secure method to get there.

This will work with:

It may work with other libs that depend on the HTTPSConnection class from httplib in the Python Standard Library.

import httplib
import socket
import ssl
from backports.ssl_match_hostname import match_hostname

def install_validating_https(cert_file):
    def validating_connect(self):
        "Connect to a host on a given (SSL) port."

        sock = socket.create_connection((self.host, self.port),
                                        self.timeout, self.source_address)
        if self._tunnel_host:
            self.sock = sock
            self._tunnel()
        self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, cert_reqs=ssl.CERT_REQUIRED, ca_certs=cert_file)
        match_hostname(self.sock.getpeercert(), self.host)



    httplib.HTTPSConnection.connect = validating_connect

Okay, that's the basics. backports.ssl_match_hostname is available in pypi, the rest is stdlib. This should work with 2.6+, for < 2.6 you'll want to get the ssl library from pypi (I don't know the implications for the HTTPSConnection lib for anything other than 2.7).

Somewhere in your app, call install_validating_https and pass it the path to your root ca collection file. Oh, wait, what's that? Well it probably would be a pem encoding of the Mozilla NSS certdata.txt. The pem encoding can be done with the certdata2pem.py script that comes with M2Crypto or you can get a pre-built one from the curl project (depending on your level of paranoia since you can't download the curl version over https).

Much of the initial info that went into my final solution came from the SSL Topic on the Python Wiki and the "How to validate SSL Certificates with Python" question on Stack Overflow with a sprinkling of Python bug reports.

Read and Post Comments

Notes on localization and jQueryUI Datepicker.

June 04, 2010 at 10:24 AM | categories: Programming | View Comments

The current documentation for the jQuery UI Datepicker control and localization are a little bit sparse. The overview tab has a blurb about the the $.datepicker.regional object containing a set of localizations indexed by locale code ('fr', 'en-GB', etc) and that those locales return an object with the attributes that provide a basis for the options object to be provided to the $('selector').datepicker() function, then there is a link to the localization files in Subversion. Lets be clear about how the $.datepicker.regional object is populated and how to use it:

  1. You can get all of the locales as one big file from the Google CDN as: http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.1/i18n/jquery-ui-i18n.min.js
  2. Even though you can, you probably should not include the giant file of all locales from the Google CDN because it will add 9.5KB compressed or 50KB uncompressed. Include the locales you need in a combined minified file for you site (perhaps combined with the rest of the scripts for your site)
  3. The $.datepicker.regional object will not fall back to a less specific locale, i.e. if you give it fr-CA, it will not fall back to fr if it does not have it, you would have to come up with that logic yourself.

Points 1 and 2 can be improved by adding the ability to select the locales in the jQuery UI Download builder tool. Point 3 can be improved by adding a $.datepicker.getregion() function that returns a valid object based on the normal locale parsing rules.

The function might look like:

$.datepicker.getregion = function(region) {
  var retval = $.datepicker.regional[region];
  if (retval) {
    return retval;
  }
  region = region.split('-');
  retval = $.datepicker.regional[0];
  if (region.length == 2 && retval) {
    return retval;
  }
  return $.datepicker.regional[''];
}
Read and Post Comments

ASP.NET MVC and Ionic.Zip

March 22, 2010 at 10:19 AM | categories: Programming | View Comments

I wanted to stream a dynamically generated Zip file in a ASP.NET MVC site. Ionic.Zip looked like the best option for Zip library. The Create a downloadable zip within ASP.NET example looked pretty close to what I needed, but I wanted to have a suitable ActionResult class to use in my controller. Here is what I came up with:

    /// <summary>
    /// A content result which can accepts a DotNetZip ZipFile object to write to the output stream
    /// </summary>
    public class ZipFileResult : ActionResult
    {

        public ZipFile zip {get;set;}
        public string filename {get; set;}

        public ZipFileResult(ZipFile zip)
        {
            this.zip = zip;
            this.filename = null;
        }
        public ZipFileResult(ZipFile zip, string filename)
        {
            this.zip = zip;
            this.filename = filename;
        }

        public override void ExecuteResult(ControllerContext context)
        {
            var Response = context.HttpContext.Response;

            Response.ContentType = "application/zip";
            Response.AddHeader("Content-Disposition", "attachment;" + (string.IsNullOrEmpty(filename) ? "" : "filename=" + filename));


            zip.Save(Response.OutputStream);

            Response.End();
        }
    }

The example noted above becomes:

public class MyZipFileController : Controller
{

    public ActionResult Index()
    {
        string ReadmeText= "This is a zip file dynamically generated at " + System.DateTime.Now.ToString("G");
        string filename = System.IO.Path.GetFileName(ListOfFiles.SelectedItem.Text) + ".zip";

        ZipFile zip = new ZipFile();
        zip.AddFile(ListOfFiles.SelectedItem.Text, "files");
        zip.AddEntry("Readme.txt", "", ReadmeText);
        return new ZipFileResult(zip, filename);
    }
}
Read and Post Comments

Cross Compiling Python Extensions

October 16, 2009 at 11:07 PM | categories: Computers, Programming | View Comments

If you are working with Python in an embedded environment, you will enventually want to build a Python extension with a cross tool chain. These instructions work both with Open Embedded and the DENX ELDK. If you need to go cross compile Python 2.5, 2.6 or 3.1. ELDK and Open Embedded include a version of Python pre-done for you. You will need a version of Python for the build system that matches your target enviroment.

On the host machine install setuptools and distutilscross. If you are using Python 2.6.3 you probably need to also install distribute which is a replacement for setuptools that also happens to fix a conflict between setuptools 0.6c9 and Python 2.6.3 (There is an update of setuptools coming soon too. See the update below). If you want to use virtualenv and distribute together, you might need to use virtualenv-distribute, a fork of virtualenv that uses distribute instead of setuptools.

Distutilscross is a package that uses setuptools to monkey patch an extra parameter, --cross-compile, into the distutils build command. The net result is that you can set your cross compile environment variables, plus PYTHONXCPREFIX and do python setup.py build -x to get your extensions to compile.

Lets take ELDK as an example.

$ export PYTHONXCPREFIX=/opt/eldk/ppc_4xxFP/usr
$ export CROSS_COMPILE=ppc_4xxFP-
$ export CC="${CROSS_COMPILE}gcc -pthread"
$ export LDSHARED="${CC} -shared"
$ export LDFLAGS="-L/opt/eldk/ppc_4xxFP/usr/lib -L/opt/eldk/ppc_4xxFP/lib"
$ python setup.py build -x bdist_egg --plat-name=linux-ppc

We always need to define PYTHONXCPREFIX and LDFLAGS. PYTHONXCPREFIX tells the -x (or --cross-compile) where your target built Python is. Mostly it is looking for the options used at build time from $PREFIX/lib/python2.x/config/Makefile. This will also set the compile up to use the target Python's headers. We need to set LDFLAGS as above because Distutils always generates the link directory options as pointing to the Host Python's install location.

We also need to set CC and LDSHARED because the compiler guessed out of the target Python Makefile is wrong for ELDK. If you are compiling C++ code you would also need to set the CXX environment variable.

Once you have the egg built, you can copy it over to a package directory (read an auto-indexed directory on a web server) and install it from your target using:

$ easy_install -f http://mylocalpackageplace/nest package_name

Concrete example time, what do you need in order to build the deps for and install TurboGears 2. You need the following packages: zope.interface, Genshi, and simplejson. ELDK's installation of Python 2.5 does not seem to come with sqlite so you will either need to do a new cross compile of python or get a separate install of the sqlite library. Once you have your binary eggs built using the above command copy them to your local http server, I'll use 192.168.1.99. You need to follow the Manual Installation method for TurboGears but when you get to Install TurboGears 2 add "-f http://192.168.1.99/nest/" to the command line so that you get:

(tg2env)$ easy_install -i http://www.turbogears.org/2.0/downloads/current/index -f http://192.168.1.99/nest/ tg.devtools

This is sufficient for most normal extension building situations. Unfortunately there are some extensions that use autotools and python-config instead of distutils (dbus-python and omniORBpy come to mind). Autotools should let you override the critical settings but I would like to see the situation improve.

Let me know if you have ideas on how to make Python development in an embedded environment better.

Update: PJE, the author of setuptools, has given notice of the impending release of setuptools 6c10 which fixes many bugs, including the Python 2.6.3 compile bug. See his comment below.

Read and Post Comments

Protecting Plone Registration Forms with ReCAPTCHA

February 24, 2009 at 10:56 AM | categories: Computers, Programming | View Comments

We recently launched the CIOC Community Site running on Plone. While we have not actually had any SPAM on the site yet, we continually get automated SPAM registrations that we then need to go and clear out. Last night Kate got fed up, so I decided to see if I could put a CAPTCHA on the form.

Looks like there is a proposal to include CAPTCHA support. The de facto standard CAPTCHA tool for Plone seems to be collective.captcha, but I wanted to be able to use the ReCAPTCHA service.

Luckily there is collective.recaptcha which is in beta and, while not listing any releases on the Plone site, is available from the Cheese Shop. But, alas, there is no documentation. It is supposed to be a drop in replacement for collective.captcha so with a little sleuthing and I got it to work. Here's how.

Installation

I am using the unified Unix installer so everything is done via buildout. Following the instructions on the collective.captcha page we need to add collective.recaptcha to your eggs list and collective.recaptcha to your zcml list. Then run ./bin/buildout and restart your Plone instance.

Configuration

We need to enter our private and public ReCAPTCHA keys in the collective.recaptcha setup form which is located at http://urltoyourplonesite.com/recaptcha-settings

Registration Form Modifications

Using this site as a helpful reference I was able to figure out the additions I needed to make to the registration form and validator.

You need to go in through your Plone site setup into the Zope Management Interface. Navigate to portal_skins/plone_login. Click join_form and then the customize button. You should now be able to customize the form contents. Down near the bottom of the form there is a line:

<div class="formControls"></div>

Above that line you need to add:

<div class="field"
     tal:define="error errors/captcha|nothing;"
     tal:attributes="class python:test(error, 'field error', 'field')">
  <label for="captcha" i18n:translate="label_captcha">Captcha</label>

  <span class="fieldRequired" title="Required"
        i18n:attributes="title"
        i18n:domain="plone"
        i18n:translate="label_required">(Required)</span>

  <div class="formHelp" i18n:translate="help_captcha">
    Provide the text in the image. Just to avoid spambots
  </div>
  <p tal:replace="structure here/@@captcha/image_tag" />

</div>

Note that the difference from the code provided by Mikel Larreategi is that ReCAPTACH provides the input element itself, so you need to omit the div that includes the <input type="text"> tag.

Once you have that saved, you need to go back to portal_skins/plone_login and click join_form_validate and once again click customize. At the bottom of the validation code, just before return state add:

captcha = context.REQUEST.get('recaptcha_response_field')
view = context.restrictedTraverse('@@captcha')

if not view.verify(captcha):
    state.setError('captcha', _(u'Are you a bot? Try again...'))
    state.set(status='failure')

Note that the difference in the validation code from Mikel Larreategi's is that the ReCAPTCHA inserted input tag is called recaptcha_response_field and not captcha.

Hopefully that is helpful to someone other than me :)

Read and Post Comments