Tuesday, 26 June 2018

Using C++ code from Python with pybind11

I have recently been working on a large C++ code base and needed to make certain parts of the code available in Python for use by our other teams. We looked at a number of different frameworks for this and eventually decided to go with pybind11. In this blog post I will describe the basic usage of pybind11 and how to call your exported code from python. For a full look at the code check out my example repository from here

Basic Example

The examples below can be considered part of a single library that is being made available. An example cpp file is available here

Method

Consider the following method that you want to make available to python

1
2
3
int add(int x, int y) {
    return x + y;
}

To make this available to python you add the following binding:

1
2
3
4
5
6
7
8
9
PYBIND11_MODULE(pybindings, mymodule) {
    using namespace pybind11::literals; // for _a literal to define arguments
    mymodule.doc() = "example module to export code";
    mymodule.def("add",
          &add,
          "Add 2 numbers together",
          "x"_a,
          "y"_a);
}

To explain the above a little bit we have

PYBIND11_MODULE(pybindings, m) - This defines the module name pybindings, with the variable mymodule used to reference it.

mymodule.doc() - Create a doc string for the module which will be displayed when using the help function in python.

mymodule.def - Define the function you want to export to python. The arguments to this are the name of the function in python, The C++ function to export, The doc string for the function, Any arguments for the function

Once build this function could be used as:

1
2
3
>>> import pybindings
>>> pybindings.add(1, 2)
3

Class

Consider the class:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
class Adder {
public:
    Adder(int x) : x_(x) {};
    int add(int y) {
        return x_ + y;
    }
 
    void setAddition(int x) {
        x_ = x;
    }
    int getAddition() {
        return x_;
    }
private:
    int x_;
};

To export this, under your PYBIND11_MODULE add the following:

1
2
3
4
pybind11::class_<Adder>(mymodule, "Adder")
    .def(pybind11::init<int>())
    .def("add", &Adder::add)
    .def_property("addition", &Adder::getAddition, &Adder::setAddition);

This is similar to the method example above except we now define the class using pybind11::class_. The class type is passed as a template argument to this and functions are defined as part of the class instead of the module.

The constructor is defined by calling pybind11::init, and you can add access to class variables as properties using the def_property function.

Once built it can be run using:

1
2
3
4
5
6
7
>>> import pybindings
>>> a = pybindings.Adder(1)
>>> a.add(2)
3
>>> a.addition = 4
>>> a.add(2)
6

stl

pybind11 seamlessly supports using STL containers within python. To to this you import the pybind11/stl.h header to make the bindings available.

1
2
3
4
5
6
7
std::string join(std::vector<char> tojoin) {
    std::string ret;
    ret.reserve(tojoin.size())
    for(auto c: tojoin) {
        ret += c;
    }
}

To export add the function definition to your module:

1
2
3
4
mymodule.def("join",
      &join,
      "Join a list of strings",
      "tojoin"_a);

This is the very similar to the add function from our first example and shows how pybind11 seamlessly supports STL containers.

To call this from python:

1
2
3
4
5
6
>>> import pybindings
>>> s = pybindings.join(['a', 'b', 'c'])
>>> print(s)
abc
>>> type(s)
<class 'str'>

Building the Example

Pybind11 has good support for CMake and be easily integrated into a CMake project. To do this add the following to your CMakeLists.txt

1
2
find_package(pybind11)
pybind11_add_module(pybindings bindings.cpp)

This will add a target to build a library pybindings.[python-information].so.

NOTE: The name of the library must add match the name of the module you defined in your C++ code.

Using the binding

To use the bindings you add the location of the above shared object into your PYTHONPATH. Assuming your built your code in /data/code/build and have a python file /data/code/bindings.py you can run:

1
PYTHONPATH=/data/code/build/ python3 /data/code/bindings.py

The source of bindings.py can be found here

No comments:

Post a Comment