pyCurious

Many times while creating LaTeX documents I’ve thought “Wouldn’t it be great if I could write an inline python script to generate some LaTeX code here.” In this post I describe how to embed python scripts in LaTeX, how to get Vim to correctly syntax highlight the embedded python, and the modifications required to get rubber to compile LaTeX that uses python.sty. All of the code and scripts mentioned in this post are available on bitbucket.

I have already built a system where python generated .tex files are \input into my documents (script execution is managed by makefiles). I want to take this a step further—allowing python source to sit in a .tex file alongside LaTeX source.

In 2007 Martin R. Ehmsen released a LaTeX package python.sty that allows such python embedding. python.sty defines a new python environment in which arbitrary python code can be placed. Everything that is written to stdout by the embedded python code is interpreted as LaTeX in the final document.

For example, this sequence of identity matrices is generated with the mixture of LaTeX and python code below.

$N \times N$ element identity matrices for $N \in \{1,2,\ldots,5\}$:

\begin{python}
def identity(n):
    """
    Generates and returns the LaTeX code for
    a n x n identity matrix.

    """
    return '\n'.join([
        r'\left[',
        r'\begin{array}{%s}' % (n*'c',),
        '\\\\'.join('&'.join('1' if i==j else '0' for j in range(n))
                        for i in range(n)),
        r'\end{array}',
        r'\right]',
    ])

# Displays identity matrices for
# n \in {1, 2, ..., 5}.
print '\n'.join([
    r'\begin{equation*}',
    r'\begin{array}{%s}' % (5*'c',),
    '&'.join(identity(i) for i in range(1,6)),
    r'\end{array}.',
    r'\end{equation*}',
])
\end{python}

What improvements can we make to python.sty?

By default when python.sty encounters \begin{python} the following actions are executed:

1. All text up to the closing \end{python} is output to outfile.py; outfile is the basename of the .tex document being processed;
2. outfile.py is piped to the python executable with the resulting stdout and stderr stored to outfile.py.out and outfile.py.err;
3. outfile.py.out is included in the LaTeX document using \input{outfile.py.out}.

This is great when a document contains a single python script, however since every environment overwrites outfile.py{.out,.err} it is impossible to view the python output and errors of all script executions. An updated python.sty fixes this by writing each environment to a different outfileN.py where N is a counter that is incremented each environment.

The updated python.sty also stores python’s return code to outfileN.rc. If python fails with a return code other than 0 the traceback and any other stderr output is included with red text in the document.

For example, if we deliberately throw an exception the traceback is included in the document body.

\begin{python}
# Create a deliberate exception.
print 'This text will be displayed.'
print 1/0
print 'This text won\'t be displayed.'
\end{python}

Can Vim correctly syntax highlight LaTeX embedded python?

Python syntax-highlighted as LaTeX isn’t pretty. A stackoverflow question and answer describes how to get Vim to correctly highlight .tex as LaTeX while using the python syntax highlighter for \begin{python}\end{python} blocks. A condensed series of steps is

1. Create a new Vim syntax file ~/.vim/syntax/pytex.vim containing:

   " Syntax for LaTeX files with embedded Python
   " environments.
   let b:current_syntax = ''
   unlet b:current_syntax
   runtime! syntax/tex.vim
   
   let b:current_syntax = ''
   unlet b:current_syntax
   syntax include @TeX syntax/tex.vim
   
   let b:current_syntax = ''
   unlet b:current_syntax
   syntax include @Python syntax/python.vim
   syntax region pythonCode matchgroup=Snip start="\\begin{python}" end="\\end{python}" containedin=@TeX contains=@Python
   
   hi link Snip SpecialComment
   let b:current_syntax = 'pytex'
       

2. Add the following line to your .vimrc:

       au BufRead *.tex set syntax=pytex
       

The LaTeX code snippits in this post were generated using this syntax highlighting setup and the :TOhtml command.

Rubber can’t compile documents that use the python.sty package!

Rubber is a python command line tool that greatly simplifies compiling LaTeX documents. Unfortunately it provides no simple way to pass additional command line arguments to the underlying latex and pdflatex executables.

python.sty uses the \write18 command that executes an arbitrary shell command from within LaTeX. This is, by default, restricted for security reasons. Full \write18 functionality can be enabled by passing the –shell-escape argument to latex or pdflatex.

I’ve created a branch of Rubber that accepts and passes on the –shell-escape command line argument. You should download and install this latest version if you wish to use Rubber in concert with python.sty.

With this shell-escape Rubber branch the example.tex file can be compiled using:

mkdir .build
rubber -d --shell-escape --into=.build example.tex
cp .build/example.pdf .

Links of Interest

1. https://bitbucket.org/brotchie/python-sty/ – python.sty source, Vim syntax scripts, and an example .tex file.
2. http://stackoverflow.com/questions/5176972/trouble-using-vims-syn-include-and-syn-region-to-embed-syntax-highlighting – stackoverflow question detailing how to syntax python within LaTeX documents.
3. https://code.launchpad.net/~brotchie/rubber/shell-escape – shell-escape enabled branch of Rubber.
4. http://thewikiblog.appspot.com/blog/python-inside-latex – where I found a copy of the original python.sty.

I heavily use KeepNote for taking random notes, keeping a daily journal, and managing a research pipeline. I’ve recently discovered the power of mind mapping with FreeMind, an open source mind mapping application. Out of curiosity I wanted to see what my KeepNote notebook looked like as a mind map.

KeepNote has an extension system and an easily navigable object hierarchy. I used the existing export_html extension as a base, adjusting the output to the FreeMind file format. The content of KeepNote notes are added as richcontent notes on the FreeMind nodes. Both KeepNote and FreeMind save notes in XHTML making the translation trivial.

The code is available at https://bitbucket.org/brotchie/export_freemind/overview.

Export of my primary KeepNote notebook. KeepNote on left; resultant FreeMind mind map on right.