And Now for Something New

What happens when you first create an instance of a class? The special method you'll (almost) never need to use: `.__new__()`

Dear reader,

You're in one of three groups. Your response to the question "What happens when you first create an instance of a class" is one of the following:

1. "Hmmm. I'm not quite sure what happens when you create an object. Actually, what's a class, again?" • No worries, I wrote a lot about Object-Oriented Programming. You can start with Chapter 7 of The Python Coding Book, and then the seven-part Harry Potter-themed OOP series here on The Python Coding Stack or its video course counterpart (currently available for free).

2. "Python calls the .__init__() special method, of course, to initialise the object" • Read on—this article is for you.

3. "Why don't you sit down. How much time do you have?" • You probably already know the stuff in this article, possibly even better than I do. But do read on, anyway, you may still find the article entertaining.

Let's go…

Creating a Hogwarts Class

I mentioned the Harry Potter-themed series about classes and object-oriented programming, and recently I was recording this series into a course for The Python Coding Place, so I'll use it as inspiration.

Here's a Hogwarts class. Note that this is a much-simplified version of the code in the seven-part series or the course!

All code blocks are available in text format at the end of this article • #1

Let's go through the code method by method:

• .__init__() is the special method used to initialise an instance of a class when you create the object. Group 2 readers are feeling smug right now—but read on. This particular .__init__() special method creates a .houses data attribute, which is a dictionary with four items. The keys in this dictionary are the Hogwarts house names. The values are lists that will eventually contain the students.

• .add_student() does what it says on the tin (as all functions and methods should!) A student is added to the appropriate list in the .houses dictionary if the house name is valid. You could add more checks in this method to ensure the student is not already in the house, but I'll keep this version simple.

• .get_students_by_house() returns the list of students for a given house.

• .get_all_students() returns a list of all the school's students, using a list comprehension that iterates through the four houses and through the lists associated with each house.

• .__repr__() is the special method that creates the string representation of the object designed for the programmer working on this class. This particular .__repr__() shows the name of the class and the state of the object by showing the .houses data attribute.

• .__str__() is the special method that creates the user-friendly string representation. This one creates a neatly-formatted display of the houses and the students within them.

And let's try out the Hogwarts class and its methods:

#2

You create an instance of the class, which you call hogwarts (with all lowercase letters), and you add a few calls to .add_student(). Finally you pass hogwarts as an argument to print(). The print() function uses an object's .__str__() special method when it's present, as it is in the Hogwarts class. Here's the output:

 Gryffindor: Harry Potter, Hermione Granger, Ron Weasley
 Hufflepuff: Cedric Diggory
 Ravenclaw: Luna Lovegood
 Slytherin: Draco Malfoy

You can also look at the other string representation, the one intended for development and debugging:

#3

The output of the built-in repr() function is the string returned by the object's .__repr__() special method. Note that I've formatted the output below manually to make it easier to read. The code's actual output shows the result in a single line unless you use the pretty print module pprint:

 Hogwarts(
     houses={
         'Gryffindor': ['Harry Potter', 'Hermione Granger', 'Ron Weasley'],
         'Hufflepuff': ['Cedric Diggory'],
         'Ravenclaw': ['Luna Lovegood'],
         'Slytherin': ['Draco Malfoy']
     }
 )

And let's try out the remaining two methods, starting with .get_students_by_house():

#4

Here are all the Gryffindors:

 ['Harry Potter', 'Hermione Granger', 'Ron Weasley']

And now, let's get a list of all the school's students:

#5

This returns a list with all the students:

 ['Harry Potter', 'Hermione Granger', 'Ron Weasley',
  'Cedric Diggory', 'Luna Lovegood', 'Draco Malfoy']

Right, Back to Our Main Topic Now We've Introduced the Hogwarts Class

A class is a template to create objects that have the same attributes. When you create an instance of a class with the expression Hogwarts(), you initialise the object using the code in .__init__(). In the Hogwarts class, this means adding the .houses data attribute. The .__init__() special method sets the initial state of the object. This state can then change during the object's lifetime, such as by adding students to houses.

But before the object can be initialised, it needs to be created. The .__init__() special method does not do this.

Python calls another method before .__init__(). This is the .__new__() special method, which creates the new instance of the class.

Typically, this is how it works:

• .__new__() creates a "blank" object that has no features. The object has no attributes or methods other than those that every Python object has (more later). This is the basic building block for all objects.

• .__init__() picks up where .__new__() leaves off. It initialises the object typically by adding data attributes and performing any other operations included in the .__init__() method definition.

But let's dive a bit deeper with the help of the Hogwarts class.

Every Class is a Subclass • Inheriting From object

You learnt about inheritance when you learnt about classes. But—I hear you say—the definition of the class Hogwarts isn't followed by parentheses to show that it inherits from a parent class (or base class or superclass—call it whatever you want.)

However, every class in Python is a child class (or derived class or subclass—call it whatever you want.) Every Python class inherits from the class object, which is the base class for every other class. Defining a class such as Hogwarts in the code above is equivalent to defining the class using the following syntax:

#6

This is valid Python syntax—and indeed, this was the syntax used in Python 2. However, this notation is no longer required in Python 3, and it's best to avoid explicitly adding the object base class, as I've done above, when defining a class.

Still, all classes inherit from object, even when you don't add object in parentheses. You can confirm this using the .__base__ attribute:

#7

This confirms that Hogwarts inherits from object. This is the case even when you don't add (object) after the class name:

 <class 'object'>

You can also create an instance of the object class directly if you want. However, this object can't do much on its own. Let's explore this in a REPL session:

#8

You can see what attributes come included with this "blank" object, and therefore, also included in every object in Python:

#9

Note that .__new__() is included in the methods present in every object, including instances of object.

Every Class Creates an Instance of object Before Customising It

You hardly ever need to write your own .__new__() special method when defining a class since the default version is called automatically.

But let's add it anyway to our Hogwarts class to help us understand what it does:

#10

The .__new__() special method is not an instance method—it doesn't have self as its first parameter. How can it be an instance method when the instance doesn't exist yet? It's a static method, but one that doesn't need the static method decorator.

The first parameter in .__new__() refers to the class. The method also includes *args and **kwargs as parameters to allow any other arguments included when creating an instance to be passed on. More on this in a bit.

The method calls the base class's .__new__() method. The base class is object. The variable cls, which refers to the class you're defining, is passed to object.__new__(). The object.__new__() call creates a new instance. This is the "blank" object.

Finally, Hogwarts.__new__() returns this new instance. And now that the instance exists, Python calls the instance method .__init__(). You can tell it's an instance method since its first parameter is self, which is the parameter used to refer to the instance itself. Python passes any *args and **kwargs in .__new__() to .__init__().

The two methods also have print() calls in this example. I included these only for demonstration purposes. Their outputs show that both special methods are called:

 Calling the __new__ method
 Calling the __init__ method

The program calls the .__new__() method before .__init__().

And just out of interest…

You won't need this in this article's example, but here's another bit of information that can give you more insight into this process.

If .__new__() doesn't return an instance of the class you're defining, the class's .__init__() method is never called:

#11

The output from this code (which is now useless!) shows that it only calls .__new__():

 Calling the __new__ method

But let's go back to the sensible version that returns the instance, and let's also replace object.__new__() with super().__new__(). Python's built-in super() returns the base class object in this example, and it's the common approach since it can deal with more complex cases of inheritance:

#12

There Can Only Be One Hogwarts

Let's look at a use case where overriding the default .__new__() method is useful. The class in this article creates an instance of Hogwarts, but there can only be one Hogwarts School. You can ensure that your code only ever creates one instance of the class, even if you try to create more than one. This is known as a singleton class.

Now, before I proceed, imagine 100 programmers in a room. Ask them about creating singleton classes in Python and you'll end up with 200 different views. If you want to know about those views, go and read about them elsewhere! In this article, I'm creating a singleton to showcase .__new__(), and I'm not making the case either for or against creating singleton classes…

Here's how you can ensure there can only be one instance of Hogwarts:

#13

Let's look at the changes from the previous version:

• You added a class attribute ._instance and set it to None initially. The leading underscore is just a convention to show that this attribute shouldn't be accessed from outside the class definition.

• The .__new__() method is called each time you try to create an instance of the class. The if statement checks whether the class attribute ._instance is still None.

• A new instance of the class is created only if ._instance is None. You then assign this new instance to the class attribute ._instance.

  • If you try to create another instance of the class, the code in the if block won't be executed again. Therefore, there can only be a single instance of the class.

• Finally, .__new__() returns the instance stored in the class attribute ._instance. This is either the new instance just created if this is the first time .__new__() is called or the existing instance created previously.

You can confirm that now you can only have a single instance of Hogwarts:

#14

The output from this code is the following:

 Calling the __new__ method
 Calling the __init__ method
 Calling the __new__ method
 Calling the __init__ method
 True

The methods .__new__() and .__init__() are called each time you try to create an instance of Hogwarts. However, the variable names hogwarts and another_hogwarts refer to the same instance of the class, as the True value at the end of the code's output confirms.

This technique assumes you're not working with multiple threads in your code.

But do you want to call .__init__() every time?

The outputs from the print() functions in the .__new__() and .__init__() methods raise another question. When you have a singleton class such as Hogwarts, do you want to call .__init__() each time you try to create an instance or just the first time when the singleton is created?

In this example, .__init__() will overwrite the existing .houses data attribute, removing any students you may have already added. It's unlikely this is the behaviour you need. Therefore, you only want to create and initialise .houses once.

One option is to move the code that creates .houses into .__new__(). This works. However, .__new__() is intended to create a new instance of the class, and .__init__() is there to initialise the instance. Therefore, it's best to stick to the primary intentions for these methods and keep the creation of .houses in .__init__(). Here's how you can resolve this:

#15

You use another class attribute, ._initialised, which you set to False. The .__init__() method checks whether this attribute is False before creating the .houses data attribute and then sets the flag to True to ensure the initialisation doesn't occur a second time.

Note how the print() call in .__init__() is also in the if block. Here's the output from this code:

 Calling the __new__ method
 Calling the __init__ method
 Calling the __new__ method
 True

The code only calls .__init__() once when creating the instance the first time.

Final Words

There are other use cases when you may need to define the .__new__() special method explicitly. These may include working with immutable types or metaclasses. But this is a good point to bring this insight into Python's .__new__() method to an end.

You may never need to write your own version of .__new__() when creating classes, but it's always a good idea to know what's going on a few inches beneath the surface!

Apologies for repeating myself, but I don't want you to miss out on this: I've used the Harry Potter world as the theme for a seven-part course—one for each year at Hogwarts—about Python's Object-Oriented Programming. The code goes much further than the simplified example used in this article and explores both early and intermediate topics in object-oriented programming.

This course is part of The Python Coding Place and is normally available only for members. However, I've made it available for free for a short period. Enrol for free now at https://thepythoncodingplace.thinkific.com/courses/magical-tour-through-oop

Code in this article uses Python 3.12

---

Stop Stack

#71

• Tempted to become a member of The Python Coding Place? Now, there's a monthly subscription option that's only $15 / month. Get access to all the video courses and the members' forum…

• Thank you to all those who supported me with one-off donations. This means a lot and helps me focus on writing more articles and keeping more of these articles free for everyone. Here's the link again for anyone who wants to make a one-off donation to support The Python Coding Stack

• The Python Coding Book is available (Ebook and paperback). This is the First Edition, which follows from the "Zeroth" Edition that has been available online for a while—Just ask Google for "python book"!

• And if you read the book already, I'd appreciate a review on Amazon. These things matter so much for individual authors!

• Any questions? Just ask…

---

Appendix: Code Blocks

Code Block #1

class Hogwarts:
    def __init__(self):
        self.houses = {
            "Gryffindor": [],
            "Hufflepuff": [],
            "Ravenclaw": [],
            "Slytherin": [],
        }

    def add_student(self, name, house):
        if house not in self.houses:
            raise ValueError(f"{house} does not exist")
        self.houses[house].append(name)

    def get_students_by_house(self, house):
        if house not in self.houses:
            raise ValueError(f"{house} does not exist")
        return self.houses[house]

    def get_all_students(self):
        return [
            student
            for house in self.houses.values()
            for student in house
        ]

    def __repr__(self):
        return f"Hogwarts(houses={self.houses})"

    def __str__(self):
        return "\n".join(
            f"{house}: {', '.join(students)}"
            for house, students in self.houses.items()
        )

Code Block #2

# ...

hogwarts = Hogwarts()

hogwarts.add_student("Harry Potter", "Gryffindor")
hogwarts.add_student("Hermione Granger", "Gryffindor")
hogwarts.add_student("Ron Weasley", "Gryffindor")
hogwarts.add_student("Draco Malfoy", "Slytherin")
hogwarts.add_student("Luna Lovegood", "Ravenclaw")
hogwarts.add_student("Cedric Diggory", "Hufflepuff")

print(hogwarts)

Code Block #3

# ...

print(repr(hogwarts))

Code Block #4

# ...

print(hogwarts.get_students_by_house("Gryffindor"))

Code Block #5

print(hogwarts.get_all_students())

Code Block #6

class Hogwarts(object):
    ...

Code Block #7

print(Hogwarts.__base__)

Code Block #8

blank_obj = object()

Code Block #9

blank_obj.__dir__()
# ['__new__', '__repr__', '__hash__', '__str__', '__getattribute__', '__setattr__',
#  '__delattr__', '__lt__', '__le__', '__eq__', '__ne__', '__gt__', '__ge__',
#  '__init__', '__reduce_ex__', '__reduce__', '__subclasshook__',
#  '__init_subclass__', '__format__', '__sizeof__', '__dir__', '__class__',
#  '__doc__']

Code Block #10

class Hogwarts:
    def __new__(cls, *args, **kwargs):
        print("Calling the __new__ method")
        instance = object.__new__(cls)
        return instance

    def __init__(self):
        print("Calling the __init__ method")
        self.houses = {
            "Gryffindor": [],
            "Hufflepuff": [],
            "Ravenclaw": [],
            "Slytherin": [],
        }

    # ... rest of the methods ...

hogwarts = Hogwarts()

Code Block #11

class Hogwarts:
    def __new__(cls, *args, **kwargs):
        print("Calling the __new__ method")
        instance = object.__new__(cls)
        return 42

    def __init__(self):
        print("Calling the __init__ method")
        self.houses = {
            "Gryffindor": [],
            "Hufflepuff": [],
            "Ravenclaw": [],
            "Slytherin": [],
        }

    # ... rest of the methods ...

hogwarts = Hogwarts()

Code Block #12

class Hogwarts:
    def __new__(cls, *args, **kwargs):
        print("Calling the __new__ method")
        instance = super().__new__(cls)
        return instance

    # ...

Code Block #13

class Hogwarts:
    _instance = None

    def __new__(cls, *args, **kwargs):
        print("Calling the __new__ method")
        if cls._instance is None:
            cls._instance = super().__new__(cls)
        return cls._instance
      
    # ...

Code Block #14

# ...

hogwarts = Hogwarts()
another_hogwarts = Hogwarts()

print(hogwarts is another_hogwarts)

Code Block #15

class Hogwarts:
    _instance = None
    _initialised = False

    def __new__(cls, *args, **kwargs):
        print("Calling the __new__ method")
        if cls._instance is None:
            cls._instance = super().__new__(cls)
        return cls._instance

    def __init__(self):
        if not self._initialised:
            print("Calling the __init__ method")
            self.houses = {
                "Gryffindor": [],
                "Hufflepuff": [],
                "Ravenclaw": [],
                "Slytherin": [],
            }
            Hogwarts._initialised = True

    # ...

hogwarts = Hogwarts()
another_hogwarts = Hogwarts()

print(hogwarts is another_hogwarts)