Extending & Embedding Python Using C

Extending & Embedding Python Using C - Embedding

Written by Mike James

Tuesday, 13 August 2024

Article Index
Extending & Embedding Python Using C - Embedding
Embedding Under Linux
Python or C?
Embedded Debugging

Page 4 of 4

Embedded Debugging

There is an advantage in using the embedded approach to developing Python extensions. You can debug your extension by simply running a C program in debug mode. There is no need to separately start a Python program and then attach a C debugger. You can simply allow the C program to load and run the Python interpreter. You can even load the Python program as a file and so make use of an IDE to write it. For example:

#define PY_SSIZE_T_CLEAN
#include <Python.h>
static PyObject *Pi(PyObject *self, PyObject *args)
{
    int m, n;
    double pi, s;
    if (!PyArg_ParseTuple(args, "ii", &m, &n))
        return NULL;
    pi = 0;
    for (int k = m; k < n; k++)
    {
        s = 1;
        if (k % 2 == 0)
            s = -1;
        pi = pi + s / (2 * k - 1);
    }
    return PyFloat_FromDouble(4 * pi);
}

static PyMethodDef AddMethods[] = {
    {"myPi", Pi, METH_VARARGS, "Compute Pi"},
    {NULL, NULL, 0, NULL} // sentinel
};

static struct PyModuleDef addmodule = {
    PyModuleDef_HEAD_INIT,
    "Pi",
    "C library to compute Pi",
    -1,
    AddMethods};

PyMODINIT_FUNC PyInit_Pi(void)
{
    return PyModule_Create(&addmodule);
}

int main(int argc, char *argv[])
{
    PyImport_AppendInittab("Pi", &PyInit_Pi);
    Py_Initialize();
    PyObject *main = PyImport_AddModule("__main__");
    PyObject *mainDict = PyModule_GetDict(main);
    FILE *fp = fopen("Pi.py", "r");   
    PyObject *l = PyDict_New();
    PyObject *result = PyRun_File(fp, "Pi.py", 
                            Py_file_input, mainDict, l);
    Py_FinalizeEx();
    return 0;
}

Using this you can run and debug the Pi.py program saved on disk.

import Pi
import time
N=10000000
t1=time.perf_counter()
pi=Pi.myPi(1,N)
t2=time.perf_counter()
print((t2-t1)*1000)
print(pi)

At the time of writing, this program doesn’t work under Windows due to a difficulty reading in the file using PyRun_File. A solution is to replace PyRun_File by:

char line[1000];
fread(line, 1, 1000, fp);
int res=PyRun_SimpleString(line);

Of course, the buffer has to be large enough to hold the entire file.

Implementing An Embedded API

If you plan to embed Python into an existing program then what you need to do is implement an API which gives the Python code access to the features you want to expose. You can proceed piecemeal and slowly add features, but it is much better to create an object hierarchy which encapsulates the features. Create a module which has objects that correspond to the data in the program and provide methods to access and process the data. A good example of an object hierarchy for embedded languages can be found in the scripting features of almost any spreadsheet.

Summary

Embedding Python in a C program is closely related to writing a Python extension in C as the same C API functions are used.
The key difference is that you have to initialize the Python interpreter before calling any C API functions.
Once you have the interpreter initialized you can make use of it as if it had been invoked and initialized by running a Python program.
You can add a modules to the system and so extend the Python interpreter when it is embedded.
In an embedded system there is usually a choice of doing things in Python or in C and which is better depends on which is easier for the particular task.
The only Python code that returns a result is an expression or a function. You can evaluate an expression using PyRun_String.
Even if you are trying to develop an extension, it can be easier to debug it as an embedded system and load the Python program that makes use of the extension.