But today, I’ll write a short post about some tools I came across that may be interesting. I’ll keep the post short since, if you’re interested, you can easily explore the packages independently – you won’t need my explanatory efforts.

Remember some of the earliest code you ever wrote? It may have looked like this:

first = input(”Enter the first number: “)
second = input(”Enter the second number: “)
print(
    f”The sum of the two numbers is {float(first) + float(second)}”
)

Those days are long gone. But what if you wanted this code to work for any number of inputs, not just two?

I’m not talking about the code to work out the sum itself. You’d probably use a list to collect all the numbers and the sum() built-in function, or just use a running total. Whatever.

The annoying part is making user-friendly strings when asking for the input – it’s fine to write "first" and "second" when there are only two prompts – and then when showing the result, which currently says "two numbers". But what if you have more?

I came across this need several times, but my solution has always been to change the string so that it’s general enough to work in all cases. I’m lazy, I know.

In any case, recently I decided to look for solutions. And of course, they exist! Everything seems to exist in the Python ecosystem. Several solutions…

num2words

The first solution is probably the simplest: the num2words package doesn’t do much beyond converting numbers to words, as its name suggests. You’ll need to install num2words using pip, uv, or whatever you use to install packages.

>>> import num2words
>>> num2words.num2words(34)
‘thirty-four’

The main function in the num2words module is also called num2words(), as often happens with such modules!

How far can it go?

>>> num2words.num2words(5468.23)
‘five thousand, four hundred and sixty-eight point two three’

Working in Spanish?

>>> num2words.num2words(5468.23, lang=”es”)
‘cinco mil cuatrocientos sesenta y ocho punto dos tres’

Or Japanese?

>>> num2words.num2words(5468.23, lang=”ja”)
‘五千四百六十八点二三’

Read more

And in my 10+ years of teaching Python, I also preferred teaching LBYL to beginners and delaying EAFP until later.

But over the years, as I came to understand Python’s psyche better, I gradually shifted my programming style—and then, my teaching style, too.

So, what are LBYL and EAFP? And which one is more suited to Python?

Look Both Sides Before Crossing the Road

You should definitely look before you leap across a busy road…or any road, really. And programming also has a Look Before You Leap concept—that’s LBYL—when handling potential failure points in your code.

Let’s start by considering this basic example. You define a function that accepts a value and a list. The function adds the value to the list if the value is above a user-supplied threshold:

def add_value_above_threshold(value, threshold, data):
    if value >= threshold:
        data.append(value)

You can confirm this short code works as intended:

# ...
prices = []
add_value_above_threshold(12, 5, prices)
add_value_above_threshold(3, 5, prices)
add_value_above_threshold(9, 5, prices)
print(prices)

This code outputs the list with the two prices above the threshold:

[12, 9]

However, you want to ensure this can’t happen:

# ...
products = {}
add_value_above_threshold(12, 5, products)

Now, products is a dictionary, but add_value_above_threshold() was designed to work with lists and not dictionaries:

Traceback (most recent call last):
  ...
    add_value_above_threshold(12, 5, products)
    ~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  ...
    data.append(value)
    ^^^^^^^^^^^
AttributeError: ‘dict’ object has no attribute ‘append’

One option is the look before you leap (LBYL):

def add_value_above_threshold(value, threshold, data):
    if not isinstance(data, list):
        print(”Invalid format. ‘data’ must be a list”)
        return
    if value >= threshold:
        data.append(value)

Now, the function prints a warning when you pass a dictionary, and it doesn’t crash the program!

But this is too restrictive.

Let’s assume you decide to use a deque instead of a list:

from collections import deque
​
# ...
​
prices = deque()
add_value_above_threshold(12, 5, prices)
add_value_above_threshold(3, 5, prices)
add_value_above_threshold(9, 5, prices)
print(prices)

This code still complains that it wants a list and doesn’t play ball:

Invalid format. ‘data’ must be a list
Invalid format. ‘data’ must be a list
Invalid format. ‘data’ must be a list
deque([])

But there’s no reason why this code shouldn’t work since deque also has an .append() method.

You could change the call to isinstance() to include the deque data type—isinstance(data, list | deque)—but then there may be other data structures that are valid and can be used in this function. You don’t want to have to write them all.

If you’re well-versed with the categories of data structures—perhaps because you devoured the The Python Data Structure Categories Series—then you might conclude you need to check whether the object is a MutableSequence since all mutable sequences have an .append() method. You can import MutableSequence from collections.abc and use isinstance(data, MutableSequence). Now you’re fine to use lists, deques, or any other mutable sequence.

This version fits better with Python’s duck-typing philosophy. It doesn’t restrict the function to a limited number of data types but to a category of data types. This category is defined by what the data types can do. In duck typing, you care about what an object can do rather than what it is. You can read more about duck typing in Python in this post: When a Duck Calls Out • On Duck Typing and Callables in Python

However, you could still have other data types that have an .append() method but may not fully fit into the MutableSequence category. There’s no reason you should exclude those data types from working with your function.

Perhaps, you could use Python’s built-in hasattr() to check whether the object you pass has an .append() attribute. You’re now checking whether the object has the required attribute rather than what the object is.

But if you’re going through all this trouble, you can go a step further.

Just Go For It and See What Happens

Why not just run the line of code that includes data.append() and see what happens? Ah, but you don’t want the code to fail if you use the wrong data type—you only want to print a warning, say.

That’s where the try..except construct comes in:

def add_value_above_threshold(value, threshold, data):
    if value < threshold:  # inequality flipped to avoid nesting
        return
    try:
        data.append(value)
    except AttributeError:
        print(
            “Provided data structure does not support appending values.”
        )

This is the Easier to Ask for Forgiveness than Permission (EAFP) philosophy. Just try the code. If it doesn’t work, you can then deal with it in the except block. Now, this fits even more nicely with Python’s duck typing philosophy. You’re asking the program whether data can append a value. It doesn’t matter what data is–can it append a value?

You don’t have to think about all the valid data types or which category they fall into. And rather than checking whether the data type has the .append() attribute first, you just try to run the code and deal with the consequences later. That’s why it’s easier to ask for forgiveness than permission.

But don’t use this philosophy when crossing a busy road. Stick with “look before you leap” there!

Another Example Comparing LBYL and EAFP

Read more

And my reaction at first was: “So what?! This tool doesn’t do anything I couldn’t already do with other features in Python.”

So, why does itemgetter() exist?

But, before answering this question, let’s answer a more basic one…

What Does itemgetter() Do?

Read more

And many people who comment about programming and AI know as much about programming as I know about music…and probably even less about AI.

But the orchestra conductor analogy seems a good one. Let me explore it further.

Read more

And here’s how they deal with the finish line logistics. Runners don’t wear bibs with numbers. When they cross the finish line, they enter a “funnel” marked by plastic cones and are handed a token with their position number. They then proceed to another official, who scans their personal barcode, which runners carry in their pockets or on a wristband, and the position token they received a few seconds earlier. This process matches the runner with their finishing position.

What’s this got to do with Python loops? And how does it help us understand why Python does loops differently from other languages?

First step, let’s create the Parkrun funnel. I’ll just put the first five finishers in this example:

>>> funnel = [”Jonathan”, “Michael”, “Samantha”, “Jessica”, “Daniel”]

Now, here’s something you definitely know already because it’s always one of the first things you’re taught when learning Python: Don’t loop through this list like this:

# Avoid this when coding in Python
>>> i = 0
>>> while i < len(funnel):
...     name = funnel[i]
...     print(name)
...     i += 1
...    
Jonathan
Michael
Samantha
Jessica
Daniel

This style mimics how other languages may work: you manually define and increment the index. To be fair, most people who shift from other languages are more likely to write the following version at some point:

# Also best to avoid this in Python
>>> for i in range(len(funnel)):
...     name = funnel[i]
...     print(name)
...    
Jonathan
Michael
Samantha
Jessica
Daniel

This version may seem more Pythonic since it uses Python tools such as range(), but still fails to make the most of Python’s iteration protocol. The Pythonic way of looping through this list is the following:

>>> for name in funnel:
...     print(name)
...    
Jonathan
Michael
Samantha
Jessica
Daniel

A question that’s often asked but rarely answered is: Why is this version preferred over the other two? I’ll write another short post to answer this question soon as I want to keep these The Club posts short whenever possible. So, let me state just a few reasons (there are more) and then I’ll move on to my main topic for today.

• It’s more readable

• It’s more efficient (try timing the versions above using timeit—remove the print() calls first)

• It’s less prone to errors and bugs

• It works with a broader selection of data structures, not just sequences

While you wait for my follow-up post on this, you can read more about Python’s Iterator Protocol, iterables, and iterators here:

• The Anatomy of a for Loop

• Iterable: Python’s Stepping Stones

• A One-Way Stream of Data • Iterators in Python

But let’s move on.

Let’s say you want to print out the names alongside each runner’s position. You’d like the following output:

1. Jonathan
2. Michael
3. Samantha
4. Jessica
5. Daniel

“Aha!” I’m often told by some learners, “This is when you need to use the for i in range(len(funnel)) idiom, since you need the index!”

Python’s for loop doesn’t explicitly use the index, so you don’t have access to the index within the for loop. Many revert to the non-Pythonic idioms for this.

But Python provides tools that let you stay within the pure Pythonic style. Python’s for loop needs an iterator—it will create one from the iterable you provide. All Python iteration needs iterators, not just for loops. Iterators are Python’s tool for any iteration.

And there are some bespoke iterators in Python that handle most of your iteration needs. I recently wrote a series about the itertools module. The itertools module contains many such tools. Here’s the series: The itertools Series.

But there are also two built-in tools that many forget, but are extremely useful. The first one is enumerate().

Here’s how you can use it to display the Parkrun results:

>>> for index, name in enumerate(funnel, start=1):
...     print(f”{index}. {name}”)
...    
1. Jonathan
2. Michael
3. Samantha
4. Jessica
5. Daniel

Read more

Gold customers get served first. When no Gold customers are waiting, you serve Silver customers. Bronze customers get served when there’s no one in the upper tiers waiting.

How do you set up this queue in your Python program?

You need to consider which data structure to use to keep track of the waiting customers and what code you’ll need to write to keep track of the complex queuing rules.

Sure, you could keep three separate lists (or better still, three `deque` objects). But that’s not fun! And what if you had more than three priority categories? Perhaps a continuous range of priorities rather than a discrete number?

There’s a Python tool for this!

So let’s start coding. First, create the data structure to hold the customer names in the queue:

“You told me there’s a special tool for this? But this is just a bog-standard list, Stephen!!”

Don’t send your complaints just yet. Yes, that’s a list, but bear with me. We’ll use the list just as the structure to hold the data, but we’ll rely on another tool for the fun stuff. It’s time to import the heapq module, which is part of the Python standard library:

This module contains the tools to create and manage a heap queue, which is also known as a priority queue. I’ll use the terms ‘heap queue’ and ‘priority queue’ interchangeably in this post. If you did a computer science degree, you’d have studied this at some point in your course. But if, like me and many others, you came to programming through a different route, then read on…

Let’s bundle the customer’s name and priority level into a single item. Jim is the first person to join the queue. He’s a Silver-tier member. Here’s what his entry would look like:

It’s a tuple with two elements. The integer 2 refers to the Silver tier, which has the second priority level. Gold members get a 1 and Bronze members—you guessed it—a 3.

But don’t use .append() to add Jim to service_queue. Instead, let’s use heapq.heappush() to push an item onto the heap:

Note that heapq is the name of a module. It’s not a data type—you don’t create an instance of type heapq as you would with data structures. You use a list as the data structure, which is why you pass the list service_queue as the first argument to .heappush(). The second argument is the item you want to push to the heap. In this case, it’s the tuple (2, “Jim”). You’ll see later on why you need to put the integer 2 first in this tuple.

The heapq module doesn’t provide a new data structure. Instead, it provides algorithms for creating and managing a priority queue using a list.

Here’s the list service_queue:

“So what!” I hear you say. You would have got the same result if you had used .append(). Bear with me.

Pam comes in next. She’s a Gold-tier member:

OK, cool, Pam was added at the beginning of the list since she’s a Gold member. What’s all the fuss?

Let’s see what happens after Dwight and Michael join the queue. Dwight is a Bronze-tier member. He’s followed in the queue by Michael, who’s a Silver-tier member:

OK, this is what you’d expect once Dwight joins the queue, right? Dwight is a low-priority customer, so he’s last. Is this just a way of automatically ordering the list, then? Not so fast…

The fourth customer to walk in is Michael, who’s a Silver-tier customer. But he ends up in the last position in the list. What’s happening here?

It’s time to start understanding the heap queue algorithm.

Heap Queue • What’s Going On?

Let’s go back to when the queue was empty. The first person to join the queue was Jim (Silver tier). Let’s place Jim in a node:

So far, there’s nothing too exciting. But let’s start defining some of the rules in the heap queue algorithm:

• Each node can have at most two child nodes—that’s two nodes connected to it.

So let’s add more nodes as more customers join the queue.

Pam joined next. So Pam’s node starts as a child node linked to the only node you have so far:

However, here’s the second rule for dealing with a heap queue:

• A child cannot have a higher priority than its parent. If it does, swap places between child and parent.

Recall that 1 represents the highest priority:

Pam (Gold tier / 1) is now the parent node, and Jim (Silver tier / 2) is now the child node and lies in the second layer in the hierarchy.

Bronze-tier member Dwight joined next. Recall that each parent node can have at most two child nodes. Since Pam’s node still has an empty slot, you add Dwight as a child node to Pam’s node:

Let’s apply the second rule: the child node cannot have a higher priority than its parent. Dwight is a Bronze-tier member, and so he has a lower priority than Pam. All fine. No swaps needed.

Michael joined the queue next. He’s a Silver-tier member. Since Pam’s node already has two child nodes, you can’t add more child nodes to Pam. The second layer of the hierarchy is full. So, you take the first node in the second layer, and this now becomes a parent node. So you can add a child node to Jim:

Time to apply the second rule. But Michael, who’s in the child node, has the same membership tier as Jim, who’s in the parent node. Python doesn’t stop here to resolve the tie. But you’ll explore this later in this post. For now, just take my word that no swap is needed.

Let’s look at the list service_queue again. Recall that this list is hosting the priority queue:

The priority queue has one node in the top layer. So the first item in the list represents the only node in the top layer. That’s (1, Pam).

The second and third items in the list represent the second layer. There can only be at most two items in this second layer. The fourth item in the list is therefore the start of the third layer. That’s why it’s fine for Michael to come after Dwight in the order in the list. It’s not the actual order in the list that matters, but the relationship between nodes in the heap tree.

But there’s more fun to come as we add more customers and start serving them—and therefore remove them from the priority queue! Let’s add some more customers first.

Angela, a Bronze-tier member, joins the queue next. Let’s add the new node to the tree first:

The relationship between parent and child doesn’t violate the heap queue rule. Angela (Bronze) has a lower priority than the person in the parent node, Jim (Silver):

One more client comes in. It’s Kevin, and he’s a Gold-tier member:

There are no more free slots linked to Jim’s node, so you add Kevin as a child node linked to Dwight. But Kevin has a higher priority than Dwight, so you swap the nodes:

But now you need to compare Kevin’s node with its parent. Pam and Kevin both have the same membership level. They’re Gold-tier members.

But how does Python decide priority in this case?

Python thinks that (1, “Kevin”) has a higher priority than (1, “Pam”)—in Python’s heap queue algorithm, an item takes priority if it’s less than another item. Python is comparing tuples. It doesn’t know anything about your multi-tier queuing system.

When Python compares tuples, it first compares the first element of each tuple and determines which is smaller. The whole tuple is considered smaller than the other if the first element is smaller than the matching first element in the other tuple. However, if there’s a tie, Python looks at the second element from each tuple.

Let’s briefly assume there’s a Gold-tier member called Adam:

Python now considers (1, “Adam”) as the item with a higher priority.

The second element of each tuple is a string. Therefore, Python sorts these out using alphabetical order (lexicographic order, technically).

That’s why Kevin takes priority over Pam even though they’re both Gold-tier members. ‘K’ comes before ‘P’ in the alphabet! You must swap Kevin and Pam:

Note that the algorithm only needs to consider items along one branch of the tree hierarchy. Jim, Michael, and Angela weren’t disturbed to figure out where Kevin should go. This technique makes this algorithm efficient, especially as the number of items in the heap increases.

Incidentally, you can go back to when you added Michael to the queue and see why he didn’t leapfrog Jim even though they were both members of the same tier. ‘M’ comes after ‘J’ in the alphabet.

Now, we can argue that it’s not fair to give priority to someone just because their name comes first in alphabetical order. We’ll add timestamps later in this code to act as tie-breakers. But for now, let’s keep it simple and stick with this setup, where clients’ names are used to break ties.

Let’s check that the service_queue list matches the diagram above:

Kevin is in the first slot in the list, which represents the node at the top of the hierarchy. Jim and Pam are in the second layer, and Michael, Angela, and Dwight are the third generation of nodes. There’s still one more space in this layer. So, the next client would be added to this layer initially. But we’ll stop adding clients here in this post.

And How Does the Heap Queue Work When Removing Items?

It’s time to start serving these clients and removing them from the priority queue.

Read more

That’s the kind of magic I don’t like. I want to know how things work. So let me take you back to when I first learnt about named tuples—the NamedTuple in the typing module, not the other one—and data classes. They share a similar syntax, and it’s this shared syntax that confused me at first. I found these topics harder to understand because of this.

Their syntax is different from other stuff I had learnt up to that point. And I could not reconcile it with the stuff I knew. That bothered me. It also made me doubt the stuff I already knew. Here’s what I mean. Let’s look at a standard class first:

class Person:
    classification = “Human”
​
    def __init__(self, name, age, profession):
        self.name = name
        self.age = age
        self.profession = profession

You define a class attribute, .classification, inside the class block, but outside any of the special methods. All instances will share this class attribute. Then you define the .__init__() special method and create three instance attributes: .name, .age, and .profession. Each instance will have its own versions of these instance attributes. If you’re not familiar with class attributes and instance attributes, you can read my seven-part series on object-oriented programming: A Magical Tour Through Object-Oriented Programming in Python • Hogwarts School of Codecraft and Algorithmancy

Now, let’s assume you don’t actually need the class attribute and that this class will only store data. It won’t have any additional methods. You decide to use a data class instead:

from dataclasses import dataclass
​
@dataclass
class Person:
    name: str
    age: int
    profession: str

Or you prefer to use a named tuple, and you reach out for typing.NamedTuple:

from typing import NamedTuple
​
class Person(NamedTuple):
    name: str
    age: int
    profession: str

The syntax is similar. I’ll tell you why I used to find this confusing soon.

Whichever option you choose, you can create an instance using Person(”Matthew”, 30, “Python Programmer”). And each instance you create will have its own instance attributes .name, .age, and .profession.

But wait a minute! The data class and the named tuple use syntax that’s similar to creating class attributes. You define these just inside the class block and not in an .__init__() method. How come they create instance attributes? “That’s just how they work” is not good enough for me.

These aren’t class attributes. Not yet. There’s no value associated with these identifiers. Therefore, they can’t be class attributes, even though you write them where you’d normally add class attributes in a standard class. However, they can be class attributes if you include a default value:

@dataclass
class Person:
    name: str
    age: int
    profession: str = “Python Programmer”

The .profession attribute now has a string assigned to it. In a data class, this represents the default value. But if this weren’t a data class, you’d look at .profession and recognise it as a class attribute. But in a data class, it’s not a class attribute, it’s an instance attribute, as are .name and .age, which look like…what do they look like, really? They’re just type hints. Yes, type hints without any object assigned. Python type hints allow you to do this:

>>> first_name: str

This line is valid in Python. It does not create the variable name. You can confirm this:

>>> first_name
Traceback (most recent call last):
  File “<input>”, line 1, in <module>
NameError: name ‘first_name’ is not defined

Although you cannot just write first_name if the identifier doesn’t exist, you can use first_name: str. This creates an annotation which serves as the type hint. Third-party tools now know that when you create the variable first_name and assign it a value, it ought to be a string.

So, let’s go back to the latest version of the Person data class with the default value for one of the attributes:

@dataclass
class Person:
    name: str
    age: int
    profession: str = “Python Programmer”

But let’s ignore the @dataclass decorator for now. Indeed, let’s remove this decorator:

class Person:
    name: str
    age: int
    profession: str = “Python Programmer”

You define a class with one class attribute, .profession and three type hints:

• name: str

• age: int

• profession: str

How can we convert this information into instance attributes when creating an instance of the class? I won’t try to reverse engineer NamedTuple or data classes here. Instead, I’ll explore my own path to get a sense of what might be happening in those tools.

Let’s start hacking away…

Read more

I’ll wait…

Done? It probably didn’t take you too long to find zymology. Probably it took you longer to find the dictionary on your bookshelf, the one you haven’t used in years!

You know z is the last letter of the alphabet, so you opened the dictionary on a page towards the end of the book. Then you looked at a random word on the page or maybe the word listed in the header. Does it start with z? Yes? Look at the word’s second letter. No? Open the book to another page further in the dictionary.

You get the idea. I know you know how to find a word in a dictionary.

But now, imagine that you didn’t know the alphabet. Bear with me. Just assume you can recognise letters, so you can read words, but you don’t know the order of the letters in the alphabet. You don’t know that A comes first and D is the fourth one, and so on.

How would you find zymology now? And how long do you reckon it will take you to find the word?

There you go, now you understand the purpose of using hash values for hashable objects. Or you can read on…

Today, I also explore the topic in a short video where I can communicate differently. But if you prefer to just read, then go ahead and skip the video.

Let’s continue exploring dictionaries and hashable objects.

Read more

Two functions that enable you to work effectively with loops in Python are zip() and enumerate(). Along with the range() function, they’re some of the most common tools you’ll see in for loops. And when you master them, you can start exploring the functions in the itertools module.

The correct number of errors in this text is either four or zero. Confused? I don’t blame you. And here’s a bit more confusion for you. It doesn’t matter either way.

Let’s talk about impostors in Python.

It’s likely that one of the first examples you saw when learning about for loops used the range() function as part of the for statement.

But you were fooled. That example, whichever one it was, didn’t do such a thing.

Read more

Tuples are usually introduced to Python learners after lists and strings. They’re almost always described as “like lists, but a bit different”.

And that’s not a bad thing. I say similar things when I introduce tuples to students. But then, fast-forward a few levels of proficiency, and I’m having different discussions with students. Are tuples like lists? Are they like strings? But the point is that those questions don’t really make sense. Other questions matter a lot more in Python. Let’s explore.

This is the first post in The Club, so allow me a word or three before getting back to tuples. I sent out an email last week about the new chapter in The Python Coding Stack. Paid subscribers are now members of The Club. I’ll write more frequent, shorter posts for The Club, sometimes accompanied by short videos (videos will complement the text, they won’t replace it). And all the other stuff: forum, Q&As, code reviews, and more.

I recall when I first started The Python Coding Stack, I agonised over what the first article should be about. It’s the first. It should be special. But then I just published about whatever topic I was thinking about at the time.

I took the same route when writing the first post on The Club – this one. And I’ll do this every week. I’ll write about whatever comes up in my coding, in my teaching, whatever I think you’ll find interesting. Or send me your questions, and they may inspire me to write about them. Don’t be shy.

Now back to tuples.

A long time ago, on a social media platform (now) far, far away, I wrote a series of threads exploring real-world objects from the perspective of Python data types. I recall one of them was about a row of houses. Would a dictionary, or a list, or a tuple, or something else be the “ideal data structure” if I wanted to represent this row of houses as a data type? But there was another question I needed to ask first.

Read more