Programmer's Python - Closure
Written by Mike James   
Monday, 19 April 2021
Article Index
Programmer's Python - Closure
Execution Context

Closure - what's that all about. As it happens it's all surprisingly logical once you know why Python implements closure.  Closure motivated and explained in this extract from my book, Programmer's Python: Everything is an Object.

Programmer's Python
Everything is an Object

Is now available as a print book: Amazon

pythoncover


Contents

  1. Hello Python World
  2. Variables, Objects and Attributes
  3. The Function Object
      Extract - Function Objects
  4. Scope, Lifetime and Closure
      Extract - Local and Global
      Extract - Closure ***NEW
  5. Advanced Functions
      Extract - Parameters
      Extract - Decorators 
  6. Class Methods and Constructors
      Extract -  Objects Become Classes
  7. Inside Class
  8. Metaclass
  9. Advanced Attributes
      Extract - Properties
  10. Custom Attribute Access
      Extract -  Custom Attributes
      Extract -  Default Methods
  11. Single Inheritance
  12. Multiple Inheritance
  13. Class and Type
      Extract - Class & Type
  14. Type Annotation
      Extract - Type Annotation 
  15. More Magic - Operator Overloading

 

Advanced Attributes

Not included in this extract:

  • Global v Local
  • Inner Functions
  • Local, Global and Nonlocal 

Closure

Closure is often treated as if it was a mysterious theoretical device when in fact it is a simple consequence of defining functions as objects.

To be more precise, it is a consequence of the fact that a function object can exist even when the function isn’t being executed.

Function objects have a life of their own even when not being used as functions.

When you define a function it has access to the local variables of any containing function. This is fine and causes no problem, but notice that access to the local variables of the containing function occurs when the inner function is executed. This normally isn’t an issue.

Where things become more complicated is if an inner function is executed after its containing function has finished. This is perfectly possible as an inner function can be assigned to a global variable or returned as an object reference by its containing function. In this case the inner function object continues to exist after the outer function has completed – it is an object and it still has a reference to it. Now imagine what happens if you call the function. The local variables of the containing function no longer exist and so the function crashes.

This isn’t reasonable – when you wrote the inner function it was perfectly correct but when you ran the function it crashed. A function shouldn’t depend on when it is run for its correctness.

The solution to the problem is to create a closure. The terminology comes from functional programming and it isn’t entirely meaningful in a wider context.

A closure captures the variables that are in scope at the time the function is defined and makes them available for use when the function is executed.

Another way to say this is that a function’s execution context is the set of variables that are in scope when it is defined and a closure makes its execution context available to it at any time in the future.

An example will make the idea clear:

def MyFunction():
    myVar=1
    def myInnerFunction():
        print(myVar)
    return myInnerFunction

Notice that the only new feature here is the way a reference to the inner function is returned. Keep in mind that myInnerFunction is just a reference to the function object that is created when we use def myInnerFunction.

The function can be used as you might expect:

MyClosure=MyFunction()

MyClosure now contains a reference to the function object created by MyFunction. You can call this using the invocation operator in the usual way:

MyClosure()

You will see 1 displayed. If you didn’t know about closure this would be amazing. The function manages to use a variable, myVar, that is local to MyFunction which finished executing and had all of its local variables destroyed before MyClosure was used to call it.

Any local variables of the containing function that are used by the inner function are stored as part of the closure.

The details of the closure are stored in the __closure__ attribute of the function object – see the section on closure and cells for details of how this works.

Now we come to some finer points of the Python closure.

The first thing that should be obvious is that if the inner function assigns to the variable then it is converted into a local variable and the variable isn’t included in the closure.

For example:

def MyFunction():
    myVar=1
    def myInnerFunction():
        myVar=myVar+1
        print(myVar)
        
    return myInnerFunction

The intent is to add one to myVar to keep a count of how many times the function is called. However, this doesn’t work and the function fails reporting an error of trying to use a variable before it has been assigned to. The point is the assignment creates myVar as a local variable to myInnerFunction and stops the closure.

The solution is to declare myVar nonlocal:

def MyFunction():
    myVar=1
    def myInnerFunction():
        nonlocal myVar
        myVar=myVar+1
        print(myVar)
        
    return myInnerFunction

Now everything works and myVar is available to myInnerFunction as part of a closure. The nonlocal declaration stops the Python system creating a local variable.

If you want to assign to a variable you want to include in the closure then make sure it is declared nonlocal.



Last Updated ( Monday, 19 April 2021 )