Programmer's Python

Programmer's Python - Closure

Written by Mike James

Monday, 19 April 2021

Article Index
Programmer's Python - Closure
Execution Context

Page 2 of 2

It is possible to create more than one inner function at a time. The question is, do they each get a unique closure or do they share the same variable?

The key to understanding this is that both functions share the same execution context and so they share the variables in it.

For example:

def MyFunction():
    myVar=1
    def myInnerFunction1():
        nonlocal myVar
        myVar=myVar+1
        print(myVar)
    def myInnerFunction2():
        nonlocal myVar
        myVar=myVar+1
        print(myVar)     
    return myInnerFunction1,myInnerFunction2

This is just two inner functions each adding one to myVar when called. The only new thing in this example is the use of a tuple to return multiple results – this is explained in detail in the next chapter.

Now we can call both functions:

Closure1,Closure2 = MyFunction()
Closure1()
Closure2()

The result is that you see 1 followed by 2 displayed. The two functions are incrementing the same copy of myVar captured by the closure.

The final detail is to determine what value is captured by the closure. Consider the following example:

def MyFunction():
    myVar=1
    def myInnerFunction1():
        nonlocal myVar
        print(myVar)
    myVar=2
    def myInnerFunction2():
        nonlocal myVar
        print(myVar)     
    return myInnerFunction1,myInnerFunction2

In this case myVar is set to 1 when myInnerFunction1 is defined and to 2 when myInnerFunction2 is defined.

What values will be printed the functions are called:

Closure1,Closure2 = MyFunction()
Closure1()
Closure2()

The answer is you will see 2 displayed twice.

There is only one captured variable per execution context and when the functions are called it has the final value that the containing function assigned to it.

All functions defined within the same execution context share it as their closure.
Function defined in different execution contexts i.e. created by different execution of the containing outer function, have their own execution context.
The values of the variables in a closure are determined by the last values they had in the outer function’s execution context.

A Common Error

Not understanding that there is just one execution context leads to the following common problem with closures. Suppose you want to create ten functions each with its own identifying state variable.

You might try to do it something like:

def MyFunction():
    f=[]
    for i in range(0,10):
        myVar=i
        def myInnerFunction():
            nonlocal myVar
            print(myVar)
        f.append(myInnerFunction)
    return f

This creates a list of ten functions, and you can store references to functions in a list because they are objects just like any other. Each function is created with myVar as a closure and the idea is that each function gets its own closure and own value i.e. the current value of i.

However, you should know that this is not what happens and when the functions are used:

f = MyFunction()
for i in range(0,10):
    f[i]()

what you see is 9 printed ten times because the closure that all of the function share captured the final value of the variable.

If you want to make this work you have to either make sure that each function has a different variable in the closure or make sure that each function is created in a different execution context – neither option is easy.

To create a new execution context you have to define a function to wrap the function you really want to create:

def MyFunction():
    f=[]
    def temp():
        myVar=i
        def myInnerFunction():
            print(myVar)
        f.append(myInnerFunction)
    for i in range(0,10):
        temp()
    return f

The temp function is in the loop to create a new execution context complete with a new local variable myVar.

Notice that temp is called ten times in the loop, each time it creates a separate local variable referenced by myVar and hence a new execution context for myInnerFunction to be defined in. Notice also that temp gets access to the list f via a closure.

This sort of coding is best avoided if possible.

Not included in this extract:

What Use is Closure?
Private State Variables
Function Self Reference
Callbacks
Closures __closure__ and Cells (Advanced)

Summary

A variable that a function assigns to is created as a local variable.
To reference a global variable that the function assigns to it has to be declared as global.
Functions can be defined within other functions and the inner functions have access to the containing function’s local variables but not vice versa.
If an inner function assigns to a variable then it is created as a local variable.
To refer to a local variable of a containing function in assignment you have to declare the variable nonlocal.
Closure is a natural consequence of function objects outliving their local variables. If an inner function exists and can be invoked after its containing function has ended, then it still has access to the same local variable via the closure.
The closure consists of all of the variables in scope when a function is declared – its execution context.
All functions declared within the same execution context share that context as their closure.
The values of variables in the closure are the last values they had before the outer function terminated.
Closures have many uses but the main ones are to provide private state variables, provide a self reference, and to provide context to callback functions.
You can work with the __closure__ magic attribute to access and even change the closure but how this works is platform dependent. Use the inspect module to isolate your code from such changes.