The Python decorator is one of its most powerful features and it is used to implement many of Python's own internals. It is a simple idea and yet it can be presented in a way that hides its simplicity. In this extract from Mike James' forthcoming book, we take a completely different look at decorators.

Programmer's Python
Everything is an Object
Second Edition

Is now available as a print book: Amazon

Contents

1. Get Ready For The Python Difference
2. Variables, Objects and Attributes
3. The Function Object
4. Scope, Lifetime and Closure
     Extract 1: Local and Global ***NEW!
5. Advanced Functions
6. Decorators
7. Class, Methods and Constructors
     Extract 1: Objects Become Classes 
8. Inside Class
9. Meeting Metaclasses
10. Advanced Attributes
11. Custom Attribute Access
12. Single Inheritance
13. Multiple Inheritance
14. Class and Type
15. Type Annotation
16. Operator Overloading
17. Python In Visual Studio Code

 Extracts from the first edition

<ASIN:1871962749>

<ASIN:1871962595>

<ASIN:1871962765>

Decorators

The Python decorator is an interesting and innovative idea, but once you understand it then you can appreciate how very simple the idea is.

As functions are first class objects you can pass a function into another function and it can also return a function as its result. This means that you can write functions which transform other functions.

For example, if we have an arith function:

def arith(a, b):
   total = a + b
   product = a * b
   return (total, product)

then we can turn it into a function that prints a message just before the computation begins and one just after:

def decorateMark(f):
    def wrapper(a,b):
        print("start")
        result=f(a, b)
        print("end")
    return result
return wrapper

Notice that this function accepts a function and constructs another function wrapper that prints start, calls the function and then prints end. Because of the nested functions it can be difficult to actually write such functions. One technique to make it easier is to write the wrapper on its own and then place it within the decorator.

 We can use this to modify the arith function:

arith=decorateMark(arith)

Now when we call arith we see start and end printed. Notice that decorateMark takes in the arith function, wraps it in a new function, complete with new actions, and then returns this new function. It effectively transforms the original function.

Python doesn’t use the term "transform" preferring "decorator" instead.

Notice that we had to save the result of the function call and return it as the result of the new wrapper function. Also notice that this wrapper will only work with a function that has two parameters. However, we can easily make it work with any number of parameters:

def decorateMark(f):
    def wrapper(*pos,**names):
        print("start")
        result=f(*pos, **names)
        print("end")
        return result
    return wrapper

The wrapper has a catch-all for positional parameters and keyword parameters and these are simply unpacked in the call to f.

This is a standard way of passing all of the parameters of one function on to another.

Decorators in this form were used in Python 2 as a way of implementing some aspects of creating objects. The problem was that it was untidy because you had to write:

def arith(a, b):
    total = a + b
    product = a * b
    return (total, product)
arith=decorateMark(arith)

The decorator function was called after the function was defined and as decorators are mostly stating something about the function – in this case that it will print start and end – they logically should come before the definition.

With a Python decorator the decorator function can be written before the definition:

@decorateMark
def arith(a, b):
    total = a + b
    product = a * b
    return (total, product)

All you do is write @ followed by the name of the decorator function and this is converted to a call to the decorator with the function being defined as the first and only parameter.

Notice that you do not call the decorater using the invocation operator (). That is you write @decorateMark and not @decorateMark(f).

Of course the decorator can be used on any function to add the start-end markers.

For a more realistic example consider the following decorator:

def benchMark(f):
    from timeit import default_timer as timer
    def wrapper(*pos, **names):
        start = timer()
        result = f(*pos, **names)
        end = timer()
        print(end - start)
        return result
    return wrapper

In this case we use the default_timer in the timeit module to find out how long a function takes to execute.

For example, to time our arith function:

@benchMark
def arith(a, b):
    total = a + b
    product = a * b
    return (total, product)
print(arith(1, 2))

Making the Wrap Transparent - functools

Although the decorator appears to recycle the original function there are some changes that you might not want because it actually wraps it with a new function object.

For example, if you try:

print(arith.__name__)
print(arith.__qualname__)

you will discover that arith thinks its name is wrapper and that it is within benchMark. You can, however, restore arith’s name and any other details by simply transferring them to the new function object.

def benchMark(f):
    from timeit import default_timer as timer
    def wrapper(*pos, **names):
        start = timer()
        result = f(*pos, **names)
        end = timer()
        print(end - start)
        return result
    wrapper.__name__=f.__name__
    wrapper.__qualname__= f.__qualname__
    return wrapper

This is obviously a tedious and common operation. A simple solution is to use the wraps decorator from the functools module.

from functools import wraps
from timeit import default_timer as timer
def benchMark(f):
    @wraps(f)
    def wrapper(*pos, **names):
        start = timer()
        result = f(*pos, **names)
        end = timer()
        print(end - start)
        return result
    return wrapper

The transfers the original __module__, __name__, __qualname__, __annotations__ and __doc__ to the wrapper. It also sets __wrapped__ to reference the wrapped function and updates the wrapper’s __dict__ with the entries in original function. This means the wrapper has any attributes that the original function has. You can add additional parameters to control what is set on the wrapper, but usually you want the wrapper to mimic the original.

For example:

@benchMark def arith(a, b):
    "My arith function"
    arith.myAttribute=10
    total = a + b
    product = a * b
    return (total, product)
print(arith(1, 2)) #(3, 2)
print(arith.__name__) #arith
print(arith.__qualname__) #arith
print(arith.__doc__) #My arith function
print(arith.myAttribute) #10

The wrapper now has all of the characteristics of the original function.

Also notice that any attributes you may create will be created on the new function object, i.e. after the decorator has been applied:

@benchMark def arith(a, b):
    total = a + b
    product = a * b
    return (total, product)
arith.myAttribute2=4

In this case the decorator is applied and then the attribute is added to the new function object returned by the decorator. That is, the wrapper has the attribute, but the original function object as referenced by __wrapped__ doesn’t.