Building a Python Extension

Now that we have checked that VS Code and GCC are working it is time to move on to build a Python extension. Rather than spending time explaining how a Python extension works we will use a very basic “hello world extension” program which simply adds two numbers together and returns the result. Exactly how this extension works is explained in Chapter 4. For the moment we simply use it as a proof that our build process works.

Open a new folder called arithmodule and create a file called arith.c containing the code listed below. The names you give the folder and the file are fairly irrelevant, but it helps to keep it meaningful. You can type the code in as listed or you can copy and paste from the book’s webpage at www.iopress.info:

#define PY_SSIZE_T_CLEAN
#include <Python.h>

static PyObject * add(PyObject *self, PyObject *args)
{
 int x, y, sts;
 if (!PyArg_ParseTuple(args, "ii", &x, &y))
     return NULL;
 sts = x+y;
 return PyLong_FromLong(sts);
}

static PyMethodDef AddMethods[] = {
 {"add", add, METH_VARARGS, "add two numbers"},
 {NULL, NULL, 0, NULL} // sentinel
};

static struct PyModuleDef addmodule = {
 PyModuleDef_HEAD_INIT,
 "arith", 
 "C library for sum",  
  -1,
  AddMethods 
};

PyMODINIT_FUNC PyInit_arith(void) {    
     return PyModule_Create(&addmodule);
}

This creates an extension that has a single function sum(x,y) which returns the sum of x and y. The code sets the module name to arith and the name of the function to sum. Notice that it is the code that determines these names, not the names of the folders and files used for the project.

The first problem we have is that we need to include the header file Python.h and this isn’t stored on the standard header file path. With VS Code there are generally two ways to set an include path – one for the compiler and one for Intellisense. If you set the correct paths for the compiler the code will compile and hopefully work but Intellisense will still show nonexistent errors in the editors if it can’t find the include files. Ideally you need to set both, but only the compiler is essential.

Most versions of Linux will have Python and the development libraries already installed. To make sure that the development libraries are installed use:

sudo apt-get install python-dev 

or

sudo apt-get install pythonx.y-dev

where x and y are the major and minor Python version numbers.

To set the include file for the compiler we have to edit the tasks.json file to indicate where the header is stored. In most cases this is:

usr/include/pythonxy

where x and y are the major and minor Python version numbers. You can set the location of the include files using an environment variable, but it is simpler and more direct to add it as an option to the GCC compiler in tasks.json:

 "-I/usr/include/python3.9",

To generate a shared library.so file we also need to add

-shared

and change the name of the file to arith.so.

The complete tasks.json is:

{
    "tasks": [
        {
            "type": "cppbuild",
            "label": "C/C++: gcc-10 build active file",
            "command": "/usr/bin/gcc-10",
            "args": [
                "-fdiagnostics-color=always",
                "-g",
                "-shared",
                "${file}",
                "-o",
                "arith.so",
                "-I/usr/local/include/python3.11" 
            ],
            "options": {
                "cwd": "${fileDirname}"
            },
            "problemMatcher": [
                "$gcc"
            ],
            "group": {
                "kind": "build",
                "isDefault": true
            },
            "detail": "Task generated by Debugger."
        }
    ],
    "version": "2.0.0"
}

The command line that this generates to build the library is:

/usr/bin/gcc-10 -fdiagnostics-color=always -g -shared
  /home/mike/Documents/PExpansion/Arith/arithmodule.c 
     -o arith.so -I/usr/include/python3.9

This works and finds the necessary include file. You don’t need to specify the location of the Python shared library to link to because it is stored in one of the standard system directories. If for any reason the linker cannot find the library you need to locate:

libpythonx.y.so

where x and y are the major and minor version numbers. Once you have the libraries location you can add:

-lpath/libpythonx.y.so

to the args section of task.json.

This should now all work but the Intellisense prompting will still be showing an error in the editor. The solution to this is to add the line:

"/usr/include/**"
to the include path in c_cpp_properties.json giving:
{
    "configurations": [
        {
            "name": "Linux",
            "includePath": [
                "${workspaceFolder}/**",
                "/usr/include/**"
            ],
            "defines": [],
            "compilerPath": "/usr/bin/gcc",
            "cStandard": "c17",
            "cppStandard": "gnu++14",
            "intelliSenseMode": "linux-gcc-arm"
        }
    ],
    "version": 4
}

Now not only should the program compile, it should also show no errors in the editor. The best way to compile it is to use Terminal, Run Build Task as this doesn’t attempt to start or debug an executable. After the build you should find arith.so in the same folder as the program.