February 2nd, 2010
The amazingly comprehensive open-source quantitative finance library QuantLib supplies a set of Python bindings generated with SWIG. Unfortunately much of QuantLib’s adaptability is made available via C++ templates. With the current SWIG wrapper it’s difficult to expose the entirety of QuantLib’s functionality without compiling every permutation of template parameter.
I discovered this quirk whilst trying to apply cubic spline interpolation to a zero curve. It turns out that by default the SWIG interface only exposes a linearly interpolated zero curve class. Fortunately there are some nice macros within the SWIG interface that ease the exposure of additional interpolation schemes, albeit with a recompile of the Python module.
Zero curves with additional interpolation methods can be added to the end of QuantLib-SWIG-0.9.7/SWIG/zerocurve.i using the export_zero_curve macro as follows:
67
68
| export_zero_curve(ZeroCurve,Linear);
export_zero_curve(CubicZeroCurve,Cubic); |
After recompiling the Python bindings you’ll now have a CubicZeroCurve class that performs cubic spline interpolation between data points.
This approach can be used throughout much of the SWIG interface files to expose template customized QuantLib classes.
Posted in Python, QuantLib, SWIG | No Comments »
December 9th, 2009
At work we use trac for feature planning and defect management. Having recently moved from Subversion to Mercurial (Hg) we needed to install hooks in our Hg repositories so that trac tickets could be modified and closed via special commit messages commands.
Hooks are added to a repository’s configuration file (.hg/hgrc) as follows
[hooks]
incoming = /usr/bin/trac-admin /trac/project/directory added Reponame $HG_NODE
This repository is then served using hgwebdir running via Apache mod_wsgi. hgwebdir.config configures the repositories that are shown, their filesystem locations, permissions, etc.
For the life of me I couldn’t figure out why the trac incoming hook wasn’t running. After a lot of tail chasing and advice from #mercurial we discovered the solution.
It turns out that by default Hg will ignore .hg/hgrc files that aren’t owned by a trusted user. Since our repositories are 2775/664 root:apache and the server process only trusts .hg/hgrc files that it owns, all our repository specific config options are ignored.
Simple adding
[trusted]
users = root
to hgweb.config fixed the problem and our trac hooks now run as desired.
Tags: hg, Mercurial
Posted in Mercurial | No Comments »
December 4th, 2009
SWIG works great out of the box when you’re dealing with basic types. Once you start to wrap more complicated APIs the type conversion magic can only go so far.
I have been writing a Python SWIG interface for a library that accepts stdio FILE* pointers for input/output. Here I’ll present a minimal example where a Python file object is passed as a FILE* to a simple C function. All source is available here if you wish to follow along.
char buffer[1024];
char*
message(FILE *input)
{
memset(buffer, 0, sizeof(buffer));
fread(buffer, sizeof(buffer), 1, input);
return buffer;
}This simply reads from a file stream until it’s closed and returns what was read. I preemptively apologise for it not being thread safe.
We’ll create a module called test wrapping the C function message such that the following code (example.py) will output “Hello World”.
import os
import test
# Create and open pipe as file objects
r,w = os.pipe()
fr, fw = os.fdopen(r, 'r'), os.fdopen(w, 'w')
# Write and close
fw.write('Hello World')
fw.close()
# Read data from pipe using test module
print test.message(fr)The first cut at a SWIG interface file yields
%module test
%{
#include "test.h"
%}
char *message(FILE *input);Unfortunately when we build and run example.py we get a TypeError indicating the argument we passed couldn’t be converted to a FILE*.
$ python setup.py build_ext --inplace
$ python example.py
Traceback (most recent call last):
File "example.py", line 17, in <module>
print test.message(fr)
TypeError: in method 'message', argument 1 of type 'FILE *'How do we make SWIG correctly convert the Python file type to a FILE*? The solution lies in a SWIG feature called a typemap.
Typemaps allow you to write short stubs in C to convert objects from Python to C and vice-versa. Writing them requires you to have some knowledge of the Python API, for simple conversions however you can make good progress simply by digging through the Python include directory.
Adding the following typemap to our SWIG interface file tells squid how to convert incoming Python objects into FILE* pointers. Conveniently the Python C API provides the PyFile_Check function that checks if a PyObject* is of the PyFile type and PyFile_AsFile functions that returns a FILE* given a PyFile instance.
/* Converts a PyFile instance to a stdio FILE* */
%typemap(in) FILE* {
if ( PyFile_Check($input) ){
$1 = PyFile_AsFile($input);
} else {
PyErr_SetString(PyExc_TypeError, "$1_name must be a file type.");
return NULL;
}
}Now when we build test and run example.py we get the desired result.
$ python example.py
Hello World
Notice that if we pass a non-file type to message a TypeError will be raised.
>>> import test
>>> test.message('not a file')
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: input must be a file type.This simple example demonstrates one powerful feature of SWIG that can be expanded to wrap complex C and even C++ APIs.
Tags: cpython, python, SWIG
Posted in Python, SWIG | No Comments »
November 25th, 2009
A decorator is simple syntactic sugar for applying a transformation to a function or method. Both
class A(object):
@decorator
def name(self):
return "foo"and
class A(object):
def name(self):
return "foo"
name = decorator(name)are equivalent. The quintessential decorator example contains a function returning a closure that performs some transformation on the wrapped function’s return value. For example
def upper(wrapped):
def shim(self):
# convert returned value to upper case
return wrapped(self).upper()
return shim
class A(object):
@upper
def name(self):
return "foo"
>>> a = A()
>>> print a.name()
FOOWhen you wish to create a decorator that accepts additional arguments such as
class A(object):
@expose("get_name")
def name(self):
return "foo"the hierarchy of closures can become complicated, and in my opinion unpythonic.
Class based decorators with arguments
Since a decorator can be any callable and we can override a class’s __call__ method to emulate a function call, a “decorator class” can be constructed. This is nice because in the _shim we have access to both the decorator class instance (self) and the instance of the class containing the decorated method (instance).
class expose(object):
def __init__(self, name):
self._name = name
self._wrapped = None
def _shim(self, instance, *args, **kwargs):
print 'Special RPC handling for %s, %r, %r' % \
(self._name, args, kwargs)
return self._wrapped(instance, *args, **kwargs)
def __call__(self, wrapped):
self._wrapped = wrapped
def shim(instance, *args, **kwargs):
return self._shim(instance, *args, **kwargs)
return shim
class A(object):
@expose("get_name")
def name(self):
return "foo"
>>> a = A()
>>> print a.name()
Special RPC handling for get_name, (), {}
fooClass based decorators without arguments
Expose handles the case where the decorator accepts arguments. But what if we just wanted to write
class A(object):
@expose
def name(self):
return "foo"Since expose is now a class, its __init__ method will be called with the method name as an argument. We cannot simply implement a __call__ method on expose because we want to get hold of the decorated method’s containing class instance. The trick here is to change expose into a non-data descriptor by adding a __get__ special method. This __get__ is passed the decorator’s containing class instance, thus we create and return a closure that calls expose’s _shim method with the desired arguments.
class expose(object):
def __init__(self, wrapped):
self._wrapped = wrapped
def _shim(self, instance, *args, **kwargs):
print 'Special RPC handling for %s, %r, %r' % \
(self._wrapped.__name__, args, kwargs)
return self._wrapped(instance, *args, **kwargs)
def __get__(self, instance, owner):
def shim(*args, **kwargs):
return self._shim(instance, *args, **kwargs)
return shim
class A(object):
@expose
def name(self):
return "foo"
>>> a = A()
>>> print a.name()
Special RPC handling for name, (), {}
fooGeneric class based decorator
We can combine the two methods above to create a generic decorator base class that handles both the argument and no argument case. The implementation of decorator_base can be found here. Below is an example where the example decorator class extends decorator_base.
class example(decorator_base):
default_positional_args = ('Default Name',)
def _arg_init(self,
name,
description="Default Description"):
self._name = name
self._description = description
def _shim(self, instance, *args, **kwargs):
print self._name, self._description, args, kwargs
return self._wrapped(instance, *args, **kwargs)
class Test(object):
@example
def upper(self, text):
return text.upper()
@example('Verbose Name', description='Verbose Description')
def lower(self, text):
return text.lower()
>>> t = Test()
>>> print t.upper('hElLo')
Default Name Default Description ('hEllo',) {}
HELLO
>>> print t.lower('HeLlO')
Verbose Name Verbose Description ('HeLlO',) {}
helloAn equivalent class based decorator can be constructed using metaclasses, this will be the subject of further post. I hope this post gives sheds some light on using classes to implement decorators.
Posted in Python | No Comments »
November 23rd, 2009
Metaclasses seem to be an obscure, misunderstood area of Python’s object model. Here is a simple example that I’ve used to both understand and explain what a metaclass is a how it can be used.
This code snippet was spawned from a colleague’s query.
“What would be the most pythonic way to override a class type’s string representation?”
Expressed in code he wanted a new-style class type A to behave like
>>> A
<class '__main__.A'>
>>> str(A)
'Hello World'
Unfotunately overriding A’s __str__ method
>>> class A(object):
... def __str__(self):
... return "Hello World"
...
>>> A
<class '__main__.A'>
>>> str(A)
"<class '__main__.A'>"
>>> str(A())
'Hello World'
>>>
only overrides the string representation of instances of A rather than the type A.
The solution to the problem leads us to metaclasses. My colleague’s conundrum can be solved by creating a new-style class with base type and a __str__ method. This acts to override the __str__ method of A’s type rather than instances of A.
>>> class meta_A(type):
... def __str__(cls):
... return "Hello World"
...
>>> class A(object):
... __metaclass__ = meta_A
...
>>> A
<class '__main__.A'>
>>> str(A)
'Hello World'
>>> str(A())
'<__main__.A object at 0x10049e710>'
Thus we have a pythonic mechanism for overriding the string representation of a class type.
Note that there is new syntax in Python 3.0 for specifying a metaclass.
>>> class A(metaclass=meta_A):
... pass
...
Tags: metaclass, python
Posted in Python | No Comments »
