Closure - what's that all about. As it happens it's all surprisingly logical once you know why Python implements closure.  Closure motivated and explained in this extract from my book, Programmer's Python: Everything is an Object.

Programmer's Python
Everything is an Object
Second Edition

Is now available as a print book: Amazon

Contents

1. Get Ready For The Python Difference
2. Variables, Objects and Attributes
3. The Function Object
4. Scope, Lifetime and Closure
     Extract 1: Local and Global ***NEW!
5. Advanced Functions
6. Decorators
7. Class, Methods and Constructors
     Extract 1: Objects Become Classes 
8. Inside Class
9. Meeting Metaclasses
10. Advanced Attributes
11. Custom Attribute Access
12. Single Inheritance
13. Multiple Inheritance
14. Class and Type
15. Type Annotation
16. Operator Overloading
17. Python In Visual Studio Code

 Extracts from the first edition

<ASIN:1871962749>

<ASIN:1871962595>

<ASIN:1871962765>

Not included in this extract:

• Global v Local
• Inner Functions
• Local, Global and Nonlocal 

Closure

Closure is often treated as if it was a mysterious theoretical device when in fact it is a simple consequence of defining functions as objects.

To be more precise, it is a consequence of the fact that a function object can exist even when the function isn’t being executed.

Function objects have a life of their own even when not being used as functions.

When you define a function it has access to the local variables of any containing function. This is fine and causes no problem, but notice that access to the local variables of the containing function occurs when the inner function is executed. This normally isn’t an issue.

Where things become more complicated is if an inner function is executed after its containing function has finished. This is perfectly possible as an inner function can be assigned to a global variable or returned as an object reference by its containing function. In this case the inner function object continues to exist after the outer function has completed – it is an object and it still has a reference to it. Now imagine what happens if you call the function. The local variables of the containing function no longer exist and so the function crashes.

This isn’t reasonable – when you wrote the inner function it was perfectly correct but when you ran the function it crashed. A function shouldn’t depend on when it is run for its correctness.

The solution to the problem is to create a closure. The terminology comes from functional programming and it isn’t entirely meaningful in a wider context.

A closure captures the variables that are in scope at the time the function is defined and makes them available for use when the function is executed.

Another way to say this is that a function’s execution context is the set of variables that are in scope when it is defined and a closure makes its execution context available to it at any time in the future.

An example will make the idea clear:

def MyFunction():
    myVar=1
    def myInnerFunction():
        print(myVar)
    return myInnerFunction

Notice that the only new feature here is the way a reference to the inner function is returned. Keep in mind that myInnerFunction is just a reference to the function object that is created when we use def myInnerFunction.

The function can be used as you might expect:

MyClosure=MyFunction()

MyClosure now contains a reference to the function object created by MyFunction. You can call this using the invocation operator in the usual way:

MyClosure()

You will see 1 displayed. If you didn’t know about closure this would be amazing. The function manages to use a variable, myVar, that is local to MyFunction which finished executing and had all of its local variables destroyed before MyClosure was used to call it.

Any local variables of the containing function that are used by the inner function are stored as part of the closure.

The details of the closure are stored in the __closure__ attribute of the function object – see the section on closure and cells for details of how this works.

Now we come to some finer points of the Python closure.

The first thing that should be obvious is that if the inner function assigns to the variable then it is converted into a local variable and the variable isn’t included in the closure.

For example:

def MyFunction():
    myVar=1
    def myInnerFunction():
        myVar=myVar+1
        print(myVar)
        
    return myInnerFunction

The intent is to add one to myVar to keep a count of how many times the function is called. However, this doesn’t work and the function fails reporting an error of trying to use a variable before it has been assigned to. The point is the assignment creates myVar as a local variable to myInnerFunction and stops the closure.

The solution is to declare myVar nonlocal:

def MyFunction():
    myVar=1
    def myInnerFunction():
        nonlocal myVar
        myVar=myVar+1
        print(myVar)
        
    return myInnerFunction

Now everything works and myVar is available to myInnerFunction as part of a closure. The nonlocal declaration stops the Python system creating a local variable.

If you want to assign to a variable you want to include in the closure then make sure it is declared nonlocal.