Python Serving HTTPS with CGI

06/05/13 13:38 Filed in: Programming
One of the claims often made about Python is that it is so easy to extend the language and change the behaviour of an object. The classic example of this is turning the standard library http server into an https server. Unfortunately, although there are hundreds of examples on the web about how to do this - made even more numerous by the move to Python 3 - days of testing here failed to get them to work with CGI scripts. Ordinary pages worked fine, but CGI would fail silently. Apparently these same examples worked on PCs.

The problem turned out to be a quite subtle implementation difference between the Mac and Unix handling of CGI processes and the PC approach in the Python library.
A basic web server in Python 3.3 can be implemented in a trivial few lines:

import os, sys
from http.server import HTTPServer, CGIHTTPRequestHandler
webdir = “.”
port = 8080
os.chdir(webdir)
srvaddr=('', port)
srvobj = HTTPServer(srvaddr, CGIHTTPRequestHandler)
srvobj.serve_forever()


The usual method for making this into an http server is to simply wrap the socket used in the server object (srvobj):

import os, sys
from http.server import HTTPServer, CGIHTTPRequestHandler
import ssl # SSL module
webdir = “.”
port = 8443
os.chdir(webdir)
srvaddr=('', port)
srvobj = HTTPServer(srvaddr, CGIHTTPRequestHandler)
srvobj.socket = ssl.wrap_socket (srvobj.socket, certfile=‘../localhost.pem', server_side=True) # wrap socket
srvobj.serve_forever()


and if you are really lucky some one will tell you that you need the command:

openssl req -new -x509 -keyout localhost.pem -out localhost.pem -days 365 -nodes


to make a new certificate.

Unfortunately, this technique does not work for the Mac and the Unix implementation because, for efficiency reasons, they employ a fork to start the process that executes the CGI rather than creating a subprocess as used by other implementations. In a non-wrapped CGI implementation the fork works fine and the output is sent to the socket correctly, however, when the socket is SSL wrapped things go terribly wrong.

The solution is to force the Unix and Mac implementations to use a subprocess leaving the SSL socket happily working and having the Python Server transfer the output of the CGI script to the client while translating the output into SSL.

import os, sys
from http.server import HTTPServer, CGIHTTPRequestHandler
import ssl
webdir = “.”
port = 8443
os.chdir(webdir)
srvaddr=('', port)
srvobj = HTTPServer(srvaddr, CGIHTTPRequestHandler)
srvobj.socket = ssl.wrap_socket (srvobj.socket, certfile=‘../localhost.pem', server_side=True)
CGIHTTPRequestHandler.have_fork=False # Force the use of a subprocess
srvobj.serve_forever()


So Python is extensible … but you may have to pay close attention to how the underlying modules are implemented.

Tags: , ,