Making HTTPS Requests secure in Python

October 30, 2011 at 05:07 PM | categories: Programming | View Comments

Python 2.7 does not automatically validate certificates on HTTPS requests. The following seems to be the cleanest, though not necessarily fully secure method to get there.

This will work with:

It may work with other libs that depend on the HTTPSConnection class from httplib in the Python Standard Library.

import httplib
import socket
import ssl
from backports.ssl_match_hostname import match_hostname

def install_validating_https(cert_file):
    def validating_connect(self):
        "Connect to a host on a given (SSL) port."

        sock = socket.create_connection((self.host, self.port),
                                        self.timeout, self.source_address)
        if self._tunnel_host:
            self.sock = sock
            self._tunnel()
        self.sock = ssl.wrap_socket(sock, self.key_file, self.cert_file, cert_reqs=ssl.CERT_REQUIRED, ca_certs=cert_file)
        match_hostname(self.sock.getpeercert(), self.host)



    httplib.HTTPSConnection.connect = validating_connect

Okay, that's the basics. backports.ssl_match_hostname is available in pypi, the rest is stdlib. This should work with 2.6+, for < 2.6 you'll want to get the ssl library from pypi (I don't know the implications for the HTTPSConnection lib for anything other than 2.7).

Somewhere in your app, call install_validating_https and pass it the path to your root ca collection file. Oh, wait, what's that? Well it probably would be a pem encoding of the Mozilla NSS certdata.txt. The pem encoding can be done with the certdata2pem.py script that comes with M2Crypto or you can get a pre-built one from the curl project (depending on your level of paranoia since you can't download the curl version over https).

Much of the initial info that went into my final solution came from the SSL Topic on the Python Wiki and the "How to validate SSL Certificates with Python" question on Stack Overflow with a sprinkling of Python bug reports.

Read and Post Comments

Notes on localization and jQueryUI Datepicker.

June 04, 2010 at 10:24 AM | categories: Programming | View Comments

The current documentation for the jQuery UI Datepicker control and localization are a little bit sparse. The overview tab has a blurb about the the $.datepicker.regional object containing a set of localizations indexed by locale code ('fr', 'en-GB', etc) and that those locales return an object with the attributes that provide a basis for the options object to be provided to the $('selector').datepicker() function, then there is a link to the localization files in Subversion. Lets be clear about how the $.datepicker.regional object is populated and how to use it:

  1. You can get all of the locales as one big file from the Google CDN as: http://ajax.googleapis.com/ajax/libs/jqueryui/1.8.1/i18n/jquery-ui-i18n.min.js
  2. Even though you can, you probably should not include the giant file of all locales from the Google CDN because it will add 9.5KB compressed or 50KB uncompressed. Include the locales you need in a combined minified file for you site (perhaps combined with the rest of the scripts for your site)
  3. The $.datepicker.regional object will not fall back to a less specific locale, i.e. if you give it fr-CA, it will not fall back to fr if it does not have it, you would have to come up with that logic yourself.

Points 1 and 2 can be improved by adding the ability to select the locales in the jQuery UI Download builder tool. Point 3 can be improved by adding a $.datepicker.getregion() function that returns a valid object based on the normal locale parsing rules.

The function might look like:

$.datepicker.getregion = function(region) {
  var retval = $.datepicker.regional[region];
  if (retval) {
    return retval;
  }
  region = region.split('-');
  retval = $.datepicker.regional[0];
  if (region.length == 2 && retval) {
    return retval;
  }
  return $.datepicker.regional[''];
}
Read and Post Comments

ASP.NET MVC and Ionic.Zip

March 22, 2010 at 10:19 AM | categories: Programming | View Comments

I wanted to stream a dynamically generated Zip file in a ASP.NET MVC site. Ionic.Zip looked like the best option for Zip library. The Create a downloadable zip within ASP.NET example looked pretty close to what I needed, but I wanted to have a suitable ActionResult class to use in my controller. Here is what I came up with:

    /// <summary>
    /// A content result which can accepts a DotNetZip ZipFile object to write to the output stream
    /// </summary>
    public class ZipFileResult : ActionResult
    {

        public ZipFile zip {get;set;}
        public string filename {get; set;}

        public ZipFileResult(ZipFile zip)
        {
            this.zip = zip;
            this.filename = null;
        }
        public ZipFileResult(ZipFile zip, string filename)
        {
            this.zip = zip;
            this.filename = filename;
        }

        public override void ExecuteResult(ControllerContext context)
        {
            var Response = context.HttpContext.Response;

            Response.ContentType = "application/zip";
            Response.AddHeader("Content-Disposition", "attachment;" + (string.IsNullOrEmpty(filename) ? "" : "filename=" + filename));


            zip.Save(Response.OutputStream);

            Response.End();
        }
    }

The example noted above becomes:

public class MyZipFileController : Controller
{

    public ActionResult Index()
    {
        string ReadmeText= "This is a zip file dynamically generated at " + System.DateTime.Now.ToString("G");
        string filename = System.IO.Path.GetFileName(ListOfFiles.SelectedItem.Text) + ".zip";

        ZipFile zip = new ZipFile();
        zip.AddFile(ListOfFiles.SelectedItem.Text, "files");
        zip.AddEntry("Readme.txt", "", ReadmeText);
        return new ZipFileResult(zip, filename);
    }
}
Read and Post Comments

Cross Compiling Python Extensions

October 16, 2009 at 11:07 PM | categories: Computers, Programming | View Comments

If you are working with Python in an embedded environment, you will enventually want to build a Python extension with a cross tool chain. These instructions work both with Open Embedded and the DENX ELDK. If you need to go cross compile Python 2.5, 2.6 or 3.1. ELDK and Open Embedded include a version of Python pre-done for you. You will need a version of Python for the build system that matches your target enviroment.

On the host machine install setuptools and distutilscross. If you are using Python 2.6.3 you probably need to also install distribute which is a replacement for setuptools that also happens to fix a conflict between setuptools 0.6c9 and Python 2.6.3 (There is an update of setuptools coming soon too. See the update below). If you want to use virtualenv and distribute together, you might need to use virtualenv-distribute, a fork of virtualenv that uses distribute instead of setuptools.

Distutilscross is a package that uses setuptools to monkey patch an extra parameter, --cross-compile, into the distutils build command. The net result is that you can set your cross compile environment variables, plus PYTHONXCPREFIX and do python setup.py build -x to get your extensions to compile.

Lets take ELDK as an example.

$ export PYTHONXCPREFIX=/opt/eldk/ppc_4xxFP/usr
$ export CROSS_COMPILE=ppc_4xxFP-
$ export CC="${CROSS_COMPILE}gcc -pthread"
$ export LDSHARED="${CC} -shared"
$ export LDFLAGS="-L/opt/eldk/ppc_4xxFP/usr/lib -L/opt/eldk/ppc_4xxFP/lib"
$ python setup.py build -x bdist_egg --plat-name=linux-ppc

We always need to define PYTHONXCPREFIX and LDFLAGS. PYTHONXCPREFIX tells the -x (or --cross-compile) where your target built Python is. Mostly it is looking for the options used at build time from $PREFIX/lib/python2.x/config/Makefile. This will also set the compile up to use the target Python's headers. We need to set LDFLAGS as above because Distutils always generates the link directory options as pointing to the Host Python's install location.

We also need to set CC and LDSHARED because the compiler guessed out of the target Python Makefile is wrong for ELDK. If you are compiling C++ code you would also need to set the CXX environment variable.

Once you have the egg built, you can copy it over to a package directory (read an auto-indexed directory on a web server) and install it from your target using:

$ easy_install -f http://mylocalpackageplace/nest package_name

Concrete example time, what do you need in order to build the deps for and install TurboGears 2. You need the following packages: zope.interface, Genshi, and simplejson. ELDK's installation of Python 2.5 does not seem to come with sqlite so you will either need to do a new cross compile of python or get a separate install of the sqlite library. Once you have your binary eggs built using the above command copy them to your local http server, I'll use 192.168.1.99. You need to follow the Manual Installation method for TurboGears but when you get to Install TurboGears 2 add "-f http://192.168.1.99/nest/" to the command line so that you get:

(tg2env)$ easy_install -i http://www.turbogears.org/2.0/downloads/current/index -f http://192.168.1.99/nest/ tg.devtools

This is sufficient for most normal extension building situations. Unfortunately there are some extensions that use autotools and python-config instead of distutils (dbus-python and omniORBpy come to mind). Autotools should let you override the critical settings but I would like to see the situation improve.

Let me know if you have ideas on how to make Python development in an embedded environment better.

Update: PJE, the author of setuptools, has given notice of the impending release of setuptools 6c10 which fixes many bugs, including the Python 2.6.3 compile bug. See his comment below.

Read and Post Comments

New Camera

July 01, 2009 at 03:47 PM | categories: Photography, Amara, General | View Comments

My 30th birthday came and went on the 21st of June. I got a couple of toys including a new Canon G10 Camera. I have been carrying it with me everywhere and have a few pictures up already. Most are from trips to the park. But there are also a couple from Amara's end of school year pageant (look for the pictures in the church). Unfortunately for the pageant shots there were a lot of pushy parents trying to get pictures, so there are no shots of Amara singing.

The camera is taking some getting used to. The controls are very different from my 300D (Digital Rebel) but I think I am starting to get the hang of them. My main problem is that I am not as fast with the G10 as I am with the SLR. The zoom reacts much slower that I can with the manual zoom on the SLR so composition takes longer with that. I also am still getting the hang of changing the auto focus location.

The G10 does, however, have a snappy start up and I have yet to see it have a problem keeping up with the speed I want to take pictures at, a problem that has been plaguing me with the 300D. One of the reasons I wanted the G10 was that I can capture in RAW. My big disappointment so far has been that Bibble, the software I use for my photo work does not yet have G10 support as they are between major releases. For the time being I am making due with the software that comes with the Camera, but it certainly does not have an optimized work flow. I don't want to have to shell out for something like Adobe Lightroom but if Bibble does not get their act together soon, that might be the way I am headed.

Read and Post Comments