A quick but dirty way to use the NAG C Library Mark 23 from Python

Posted on
28 Jun 2013

I get requests about using NAG from python from time to
time, and NAG strives to make its algorithms as accessible as possible from a
wide variety of languages and platforms. Python is no exception.

While we don’t yet provide a fully documented python API
replete with examples and so on, there is a quick but dirty way to get
NAG C Library python bindings covering every library algorithm, struct, and

My starting point was the old href="http://www.walkingrandomly.com/?page_id=1662">PyNAG, which is
excellent as far as it goes. It is also for the older Mark 8 library -- which
none of you out there in userland should be using.

PyNAG also looks a bit labor-intensive to produce for the
whole library, which would entirely disrupt my Maynard G. Krebs Cycle. There
has to be better way, I thought.

And there is! At least on Linux. For this exercise I used
our NAG C Library Mark 23 implementation for Linux and gcc, LP64 model,
hereafter abbreviated as href="http://www.nag.com/downloads/impl/cll6a23dgl.tgz">cll6a23dgl.

Here is the method that I used: 

Step 1 --
Get the Tools

For this exercise we'll need two tools. The first is gccxml,
the extension to gcc that produces an xml description of the gcc compiler's
internal representation of a compiled code. The second tool is the python
extension ctypeslib. Both are available from the Ubuntu repositories
(for example.)


Step 2 --
Collect the C Headers

Make a file "nag4py.h" that includes all of the
relevant headers:

echo style='color:#FF25D5'>"#include " > nag4py.h

find style='color:#A42B00'>/opt/NAG style='color:#A42B00'>/cll6a23dgl/include
-name "nag[a-z][0-9][0-9].h" style='color:#A42B00'>| sort | sed -e "s/^.*\//#include
| sed -e style='color:#FF25D5'>"s/$/>/" >> nag4py.h

echo "#include


Step 3 --
Produce the XML

h2xml -I. -I style='color:#A42B00'>/opt/NAG style='color:#A42B00'>/cll6a23dgl/include
nag4py.h -o nag4py.h.xml


Step 4 --
Produce the Python

nag4py.h.xml -o nag4py.py -l/opt style='color:#A42B00'>/
NAG/cll6a23dgl style='color:#A42B00'>/lib/libnagc_nag.so


Step 5
(Optional) -- edits for clear C library concordance / compatibility with PyNAG

Edit the top of nag4py.py to read:

from ctypes style='color:#C604E6'>import *


#human edit

= None

= None

= None

= 0

#end human edit


and edit the bottom of nag4py.py to read:

'Nag_HashTable', style='color:#FF75E5'>'Nag_SparseNsym_CheckData'

#human edit


#end human edit


Then with very minor modifcations the PyNAG examples will
run. For many the only change needed is to change the import, from “PyNAG” to

Another necessary change is that Numpy arrays must be passed
slightly differently than with PyNAG, e.g.

x.ctypes.data_as(POINTER(c_double)), …


instead of

x.ctypes.data_as(c_void_p), …

Nag C Library enum types are translated nicely and may be
used like one normally would, e.g.


and callback function typing is taken care of for us, e.g.


objf[0] = exp(xc[ style='color:#FF75E5'>0
]) * (xc[0]
* 4.0 * (xc[ style='color:#FF75E5'>0]+xc[1])
+ …


= NAG_E04CCC_FUN(py_obj_func)

While the module nag4py.py generated in this way is
immediately usable on Linux, it is specific to the local installation. To get
it working on anything else a small amount of work remains to be done. First,
we have to remove the extraneous debris from gcc that finds its way into
nag4py.py. Second, we must omit some symbols exposed by the NAG C Library on
Linux that are not exposed by all NAG C Library Mark 23 implementations.

Once that’s done, we get a module suitable for Linux,
Windows, and OS X, available href="http://www.nag.com/blog_files/python/nag4py.zip">here. Updated PyNAG
examples are here.

End Notes:

On Linux, if you’re instead using the NAG C Library ILP64
implementation CLL6A23DH,
use this alternative

On Windows, I tested with the href="http://www.continuum.io/downloads">64-bit Anaconda python
distribution, which comes with all of the python bits needed to run the
examples, including the graphical Rosenbrock example. An applicable NAG C
Library implementation is CLW6I23DA.
I found I had to add the folder containing the NAG DLLs to the system PATH
environment variable.

On OS X, I tested with href="http://www.python.org/ftp/python/2.7.5/python-2.7.5-macosx10.6.dmg">python
2.7.5 from python.org. A suitable NAG C Library implementation is href="http://www.nag.com/downloads/cl/clmi623dgl">CLMI623DG. In order to
run the Rosenbrock example, I also needed to install href="http://sourceforge.net/projects/numpy/files/NumPy/1.7.1/numpy-1.7.1-py2.7-python.org-macosx10.6.dmg/download">numpy-1.7.1,
and XQuartz
, since I’m running Snow Leopard.

In all cases I’ve assumed the NAG C Library has been
installed in the default location. If on your system it’s installed elsewhere,
you’ll need to edit nag4py.py accordingly.

Thanks to my colleague Brian Spector for testing on OS X
Lion and for some suggested example changes. Thanks also to my colleague Mike
Hooper, whose gentle questions suborned my inner Krebs.

Jul 17 Update: Some example cleanup, included three
new graphical examples, and changed the module name.