In my last post, I introduced pybind11 and some basic examples. In this post I want to show how to use python sub-modules with your exported bindings. In particular, I want to show how we structured our bindings in sub-modules when the C++ code was in different libraries in our main project.
Python sub-module
A python sub-module is accessible from python like:
123>>>
from
ork
import
peon
>>> peon.work_work()
I'm
not
that kind of Orc
In this example, ork
is the main module and peon
is the sub-module. In pure Python these might have the folder structure
1234ork
__init__.py
peon
__init__.py
C++ Layout
For our code, we had a project that has multiple C++ libraries and we wanted to expose bindings from each library as a
different sub-module of ork
in Python:
1234567891011121314ork
CMakeLists.txt
peon
CMakeLists.txt
include
/
src
/
grunt
CMakeLists.txt
include
/
src
/
warlock
CMakeLists.txt
include
/
src
/
So we wanted to export some functionality from ork.peon
, ork.grunt
, and ork.warlock
to python.
Exporting the bindings
Single Shared Object
The easiest way to do this is to create a single shared object using pybind11. This will include all exported bindings for all the libraries you want to export.
A simplified example would be to add a new ork_bindings.cpp
after your other libraries:
123456789101112PYBIND11_MODULE(orc, mymodule) {
mymodule.doc()
=
"Orks live here"
;
py::module peon
=
mymodule.def_submodule(
"peon"
,
"A peon is a submodule of 'ork'"
);
peon.
def
(
"work_work"
, &Peon::work_work,
"Do some work"
);
py::module grunt
=
mymodule.def_submodule(
"grunt"
,
"A grunt is a submodule of 'ork'"
);
grunt.
def
(
"work_work"
, &Grunt::work_work,
"Do some work"
);
py::module warlock
=
mymodule.def_submodule(
"warlock"
,
"A warlock is a submodule of 'ork'"
);
warlock.
def
(
"work_work"
, &Warlock::work_work,
"Do some work"
);
}
In your CMakeLists.txt
you export the the module as:
1234567pybind11_add_module(ork, ork_bindings.cpp)
target_add_library(ork
PRIVATE
peon
grunt
warlock
)
This works fine for a small amount of libraries and exported code. However, I didn't like this approach as it moved your export code away from your main code. This would make it easy to forget to add a new function to a binding and allow for consistency issues to creep into the project.
Multiple Shared Objects
Our chosen approach was to use a separate bindings library for each C++ library to be exported as a sub-module. Then use a Python module as the main module.
To do this we added the following to our code:
1234567891011121314151617ork
/
CMakeLists.txt
peon
/
CMakeLists.txt
include
/
src
/
bindings.cpp
grunt
CMakeLists.txt
include
/
src
/
bindings.cpp
warlock
CMakeLists.txt
include
/
src
/
bindings.cpp
An example bindings.cpp
for peon is:
1234567PYBIND11_MODULE(pypeon, m)
{
/
/
rename to a submodule
m.attr(
"__name__"
)
=
"ork.pypeon"
;
m.doc()
=
"A peon is a submodule of 'ork'"
;
m.
def
(
"work_work"
, &Peon::work_work,
"Do some work"
);
}
These bindings were added in each CMakeLists.txt
as:
12345pybind11_add_module(pypeon, src
/
bindings.cpp)
target_link_library(pypeon
PRIVATE
peon
)
When installing your library you then install as:
12345ork
/
__init__.py
pypeon.[python_info].so
pygrunt.[python_info].so
pywarlock.[python_info].so
One of the main problems with this approach is the naming of the sub-modules. As the C++ libraries are called
peon
, grunt
, and warlock
, it is not possible to have another CMake target with the same name. Therefore, you have to
have a slightly different name. In the above example, I have chosen to add py
as a prefix for the sub-module names.
Conclusions
So far we have found our approach to work, even with the downside of having a prefix to the name. This allows us to make sure bindings code lives as close a possible to our C++ code and we can conditionally choose which modules to export using CMake options.
No comments:
Post a Comment