In the previous chapter we looked at how Python provides the basic facilities needed to implement what looks like a classical approach to objects – that is class and methods. In this chapter we look more closely at how this works. It is slightly more complicated than you might imagine.This extract is from my new book with the subtitle "Something Completely Different".

Programmer's Python
Everything is an Object
Second Edition

Is now available as a print book: Amazon

Contents

1. Get Ready For The Python Difference
2. Variables, Objects and Attributes
3. The Function Object
4. Scope, Lifetime and Closure
     Extract 1: Local and Global ***NEW!
5. Advanced Functions
6. Decorators
7. Class, Methods and Constructors
     Extract 1: Objects Become Classes 
8. Inside Class
9. Meeting Metaclasses
10. Advanced Attributes
11. Custom Attribute Access
12. Single Inheritance
13. Multiple Inheritance
14. Class and Type
15. Type Annotation
16. Operator Overloading
17. Python In Visual Studio Code

 Extracts from the first edition

<ASIN:1871962749>

<ASIN:1871962595>

<ASIN:1871962765>

You don’t have to know how class does its magic generally and to create objects to use object-oriented programming but if you do know how it works there many more possibilities open to you. Python exposes much of the mechanism of objects so that you can modify the way that it works.

This is often called meta-programming but where ordinary programming ends and meta-programming begins is a fairly arbitrary line and in Python you can do meta-programming in Python.

Most of the work of the class is done by a set of magic methods. In this chapter the focus is on those methods that are involved in creating an object without the complication of inheritance – for that see Chapter 11.

__new__ and __init__

We have already met __init__ as a way of creating instance attributes and generally initializing the instance. When you call a class as a function __init__ is run just after another magic method, __new__.

It is confusing at first to have two methods involved in creating an instance but it is perfectly logical.

• The __new__ class method is responsible for creating an object of the class. It brings the instance object into existence.

• Then __init__ is called to initialize the instance by adding instance attributes and assigning values.

This is all fine but what exactly does it mean that __new__ is used to create an instance of the class?

The class actually creates an instance of its superclass.

We haven’t gone into detail about inheritance in Python, but the basic idea is that every class has a superclass – i.e. the class that it is based on. The class adds attributes and methods to the object that it derives from its superclass. By default every class has the class object as its superclass.

So to construct an instance of the class you need an instance of the superclass, which is by default object.

How do you get an instance of the superclass?

Easy, just explicitly call its __new__ method and you will get an initialized instance.

Consider the class:

class MyClass:
    MyAttribute=20

what happens when you write:

obj=MyClass()

The first thing that happens is that __new__ is called and the class definition is essentially:

class MyClass:
    def __new__(cls):
        return object.__new__(cls)
    MyAttribute=20
obj=MyClass()

The usual way of calling __new__ on the superclass is to use the super function which is described in detail in Chapters 11 and 12, but we know in this simple situation that the superclass of MyClass is object.

When you create an instance __new__ is called and this returns an instance of object i.e. a minimal object with no attributes other than a set of magic methods which are standard in all Python objects. Of course this is exactly what you need as the attributes set by MyClass are class methods and not created on the instance. The variable, obj in this case, is then set to reference the new instance.

Recall that at first all of the attributes that an instance seems to have are hosted by its class object.

The parameter that you pass to __new__ has to be a type object which for the moment we can take to be a class or a built-in type. It plays a very minor role in the creation of the instance but you will find that it is stored in the __class__ attribute of the instance. It is used to determine what the instance is an instance of. In most cases you simply pass the default parameter, cls in this case, which is automatically set to the class i.e. MyClass in this case.

If you don’t want to do this you don’t have to.

For example:

class MyClass2:
    pass
class MyClass:
    def __new__(cls):
        return object.__new__(MyClass2)
    MyAttribute=20
myObj=MyClass()
print(myObj.__class__)

Notice that now we have passed MyClass2 to the call to __new__ which results in __class__ being set to MyClass2 even though the instance was created by MyClass.

In Python type is a matter of keeping track of what created the instance and it is largely based on an honor system – if you want to break the rules and lie about what created the instance you can. The system does, however, check that the cls you specify is a subtype of the type you are creating. Notice that when you use object.__new__(MyClass) what you get is a new instance of object that claims to be an instance of MyClass. Recall everything is a subtype of object.

The type that is assigned doesn’t actually change very much.

The one thing it does change is whether or not __init__ is called. As long as __new__ returns an object of the indicated type, or subtype, then __init__ is called but if it returns anything else __init__ is not called.

That is, the system only calls __init__ if the __class__ of the instance matches the class used to create it. So in the previous example where object.__new__(MyClass2) was used they don’t match and __init__ isn’t called.

However, in:

class MyClass:
    def __new__(cls):
        return object.__new__(cls)
   
    def __init__(self):
        self.MyAttribute = 20
        print("__init__ called")
myObj=MyClass()

You will see the __init__ called and MyAttribute will exist in the instance’s __dict__ and be set to 20.

Finally, both __new__ and __init__ are passed any arguments used when the class is called as a function. This means you have to include the parameters even if you don’t use them.

For example:

class MyClass:
    def __new__(cls,value):
        return object.__new__(cls)
       
    def __init__(self,value):
        self.MyAttribute = value
        
myObj=MyClass(42)