Meta __new__ and __init__

So after the namespace has been constructed, the class object is constructed by the system making an implicit call:

MyClass=MyMeta(“MyClass,(object,),ns)

Notice that this is a call to MyMeta which is a class, so what happens is exactly what always happens – first __new__ is called and then __init__. This is also what happens when you make a direct call to:

type("MyClass",(object,),ns)

if MyMeta doesn’t have a definition for __new__ or __init__. The calls are passed on to the same methods defined on type or the metaclass next up in the inheritance chain – see Chapter 11.

If you want to customize the metaclass then you have to provide implementations of __new__ or __init__. The purpose of both methods is exactly the same as in the case of a class creating an instance and they are called with the same parameters as passed to the metaclass i.e. "MyClass",(object,),ns. Notice that metaclass __new__ and __init__ are called with more parameters than class __new__ and __init__.

Use __new__ to create the instance of the class and __init__ to customize it.

This isn’t strictly the case because you can do your initialization in __new__ after constructing the class object and before returning it, but it is a good division of labor and worth trying to keep to.

So the default behavior provided by type can be implemented using explicit __new__ and __init__ definitions:

class MyMeta(type):
    def __new__(self,name,base,ns):
        return type.__new__(self,name,base,ns)
    def __init__(self,name,base,ns):
        type.__init__(self,name,base,ns)

It is more usual to use cls for class rather than self, but using self does make sure that it is clear that there is nothing new here.

You can use this default code to start customizing the metaclass:

class MyMeta(type):
    def __new__(self,name,base,ns):
        print("meta new")
        return type.__new__(self,name,base,ns)
    def __init__(self,name,base,ns):
        print("meta init") self.cloned = False
        type.__init__(self,name,base,ns) class

MyClass(metaclass=MyMeta):
    myAttribute=21*2

If you run this you will see:

meta new
meta init

which are generated as soon as the system reaches the class definition i.e. not when the class is called. Notice that it would be best practice to use super().__new__ and super().__init__ but in this case we need it to be clear that it is type that is doing the work.

Notice that you can also pass additional keyword parameters to the metaclass when it is used in a class definition.

For example:

class MyClass(metaclass=MyMeta,myParameter=10):

means that the __init__ and __new__ will be called with parameters:

self,name,base,ns,myParameter=10

You can use any parameters you care to invent but make sure that the metaclass’s methods accept them even if they don’t use them.

For example, suppose you want all MyClass to always have a cloned attribute that is set to true or false to indicate that it has been cloned:

class MyMeta(type):
    def __init__(self,name,base,ns):
        self.cloned = False
        type.__init__(self,name,base,ns)

Notice that this leaves type.__new__ to create the class object, __init__ just customizes it.

Now you can use MyClass to create an instance:

class MyClass(metaclass=MyMeta):
    myAttribute=21*2

myObject=MyClass()

It is important that you don’t confuse the metaclass __init__ with the class __init__. The first is called when the class is implemented i.e. when the Python system reaches the class definition. The second is called when you use to the class to create an instance.

If you understand this you can immediately see that the metaclass __init__ is only called once but the class __init__ is called once for each instance. It should also be obvious that the cloned attribute is an attribute of the class and not an instance – it is converted into an instance attribute when assigned to:

myObject.cloned=True

You can also override the __new__ metaclass method to modify how the class instance is created but this is less common a requirement.

__prepare__

A well as __new__ and __init__, metaclasses also have a __prepare__ magic method which is automatically called to create the namespace data structure. It is called with the name and bases and it returns an empty ordered mapping.

It is called before the evaluation of the class body and its sole purpose is to provide an empty data structure to store the namespace. The __prepare__ method isn’t often needed and its main purpose is to allow the use of a modified dictionary to order the namespace in some way.

The example given in the standard Python documentation shows how both __prepare__ and __new__ can be used:

import collections
class OrderedClass(type):
  def __prepare__(name, bases, **kwds):
    return collections.OrderedDict()
  def __new__(cls, name, bases, namespace, **kwds):
    result = type.__new__(cls, name, bases,
                                 dict(namespace))
    result.members = tuple(namespace) return result

The __prepare__ is called before the class body is executed and it simply returns an OrderedDict data structure rather than a dictionary. An OrderedDict works exactly like a dictionary but it keeps the order of the key/values as they are entered. Next the body of the class is executed and the results are stored in the OrderedDict returned by __prepare__. Next __new__ is called in the usual way but now the namespace is an OrderedDict and not just a dictionary. The call to type.__new__ creates the class instance but notice that we need to convert namespace into a dictionary. Finally, the OrderedDict is converted to a tuple and assigned to an attribute.

Now any class that uses OrderedClass as its metaclass:

class A(metaclass=OrderedClass):
    def one(self): pass
    def two(self): pass
    def three(self): pass
    def four(self): pass
print(A.members)

will have a members attributes that gives the attributes in the order that they were defined:

('__module__', 'one', 'two', 'three', 'four')

Notice that in this case the __new__ method was used to modify the class instance, something we usually expect __init__ to do.

So to summarize: 

• Every class object has a metaclass which creates it.

• The metaclass behaves just like a class in that it has a __new__ which creates the instance of the class and an __init__ which customizes the instance. It also has a __prepare__ method which can be used to supply a data structure to receive the namespace.

• The metaclass is called when the system encounters the class definition. 

1. The __prepare__ method is called, if there is one, to supply the data structure for the namespace.

2. The class body is evaluated and stored in the namespace.

3. __new__ is called to create the class instance usually delegating to super().__new__.

4. __init__ is called to initialize the class.

An Example – Final Classes

Many languages have the concept of a final class – one that cannot be inherited. Python does not have such a facility as any class can be inherited, but you can use a metaclass to make the equivalent of final.

What is special about the metaclass mechanism is that it is the only Python construct that can come into effect before the class has been created. You can use decorators to modify a class after it has been created, but only a metaclass can intervene in the creation of a class.

Even this doesn’t seem to be enough to stop a class being inherited as this is something that occurs after the class has been created, but it is not the original class that concerns us. When a class inherits from a class it also inherits the same metaclass. All we have to do is check that there isn’t a class in base that isn’t an instance of the metaclass.

This has to be done before the class is created:

class Final(type):
    def __new__(self, name, base, ns):
        for b in base: if isinstance(b, Final):
            raise TypeError("Class isfinal: " + b.__name__)

return type.__new__(self, name, base, ns)

When you try:

class A(metaclass=Final):
    pass

and __new__ is called, the class is created because there isn’t anything in the base list.

However, following this if you try:

class B(A):
   pass

then you will generate the exception because A is in Base and it is an instance of Final. In short, if any class in the base list is marked as final by using the metaclass Final then the new class isn’t created.

Notice that many Python programmers would find this use of metaclasses not idiomatic Python because the feeling is that classes should always be inheritable and the most you should do is put a polite warning in the form of comments if you don’t want a class used in that way.

Meta Attributes and Methods

Final version in book

How Classes become Functions – Metaclass __run__

Final version in book 

Singleton Revisited

Final version in book

Abstract Base Class

Final version in book

Decorators Revisited

Final version in book

Using Metaclass

It is difficult to present any convincing use cases for metaclass, largely because many obvious uses can be implemented more directly using decorators.

The key facts about metaclass is that it gives you a way of intervening in the class before the class object has been constructed. It also gives you a way of working with and organizing sets of classes. Think of a metaclass’s relationship to classes as being similar to the relationship between class and instance.

This is not to say that you should underestimate what can be done with a metaclass. For example, the slowly developing gradual typing for Python makes a lot of use of metaclass and is an example of how you can create sophisticated systems using it.

Summary 

• A class is an object that creates instances.

• A metaclass is an object that creates classes.

• The base metaclass is type.

• Type can also be used as a function to dynamically create classes.

• A metaclass has to inherit from type.

• A metaclass goes though the usual stages of calling __new__ to create an instance of the class and __init__ to initialize it.

• In addition, a metaclass also has __prepare__ which can supply a data structure suitable for storing the namespace.

• Metaclasses can be used for many things but they are mostly sophisticated. Typical uses are implementing a final class and singleton.

• The metaclass __call__ is used to convert a class object into a callable. You can override this to control how the class constructs an instance.

• As a class is a callable it can be used as a decorator with its behavior set by its metaclass’s __call__.

• As a class is a callable it can also be decorated by either a class or a function.

• There are many cases where the job of a metaclass can be taken on by a decorator.

Programmer's Python
Everything is an Object
Second Edition

Is now available as a print book: Amazon

Contents

1. Get Ready For The Python Difference
2. Variables, Objects and Attributes
3. The Function Object
4. Scope, Lifetime and Closure
     Extract 1: Local and Global ***NEW!
5. Advanced Functions
6. Decorators
7. Class, Methods and Constructors
     Extract 1: Objects Become Classes 
8. Inside Class
9. Meeting Metaclasses
10. Advanced Attributes
11. Custom Attribute Access
12. Single Inheritance
13. Multiple Inheritance
14. Class and Type
15. Type Annotation
16. Operator Overloading
17. Python In Visual Studio Code

 Extracts from the first edition

<ASIN:1871962749>

<ASIN:1871962595>

<ASIN:1871962765>

Related Articles

Creating The Python UI With Tkinter

Creating The Python UI With Tkinter - The Canvas Widget

The Python Dictionary

Arrays in Python

Advanced Python Arrays - Introducing NumPy