If You Find if..else in List Comprehensions Confusing, Read This, Else…
The apparent confusion of where to place if and if..else in list comprehensions
For too long, I relied on trial and error to get this right. When writing list comprehensions, I could never remember whether the lone if
comes before the for
and the if..else
combo after it or the other way round (spoiler alert: it's the other way round). Not anymore. Now, it makes perfect sense. Here's my epiphany.
What Am I Talking About?
Here's what's confused me for so long. I'll start with a short example. Let's say you have a list of names, and you only want to keep names that have more than five letters. You can achieve this with the following list comprehension:
The list comp includes the following three parts, in this order:
The variable called
name
The
for
bit to loop through the listnames
The
if
bit to decide which names to keep
But what if instead of discarding the shorter names entirely, you want to replace them with a placeholder:
The order of the parts in the list comp is now different:
The variable called
name
[Some things don't change]The
if..else
bit to decide which names to keep and which to replaceThe
for
bit to loop through the listnames
Did you notice the switch? It's like a magician's sleight of hand. Here are both list comps shown together, with some extra non-PEP 8 spaces for display purposes only:
The if..else
bit came before the for
bit in the second case.
Surely, this doesn't make sense, right? You have two options:
Accept it and memorise the two versions
Read on
Yesterday, I published The Python Coding Book • a relaxed and friendly programming textbook for beginners. It’s for beginners, so possibly not suitable for you. But perhaps, you know someone starting their journey. Or maybe, you still want to read my perspective on Python programming. It’s the First Edition, but it follows from the “Zeroth” Edition which has been live online for a while–just ask Google for “python book”.
Ebook and paperback available (Ebook available now, paperback in a few hours or so!)
The Anatomy of a List Comprehension (The Abridged Version)
Here's what the fount of wisdom has to say–a.k.a. the Python documentation: "The comprehension consists of a single expression followed by at least one for
clause and zero or more for
or if
clauses."
Let's start with the first list comp, still with the non-conforming spaces:
Reread the phrase from the docs. Here are the three parts:
Start with a single expression–The identifier
name
, which we often refer to as the variable, fits this description....followed by at least one
for
clause–This example has onefor
clause, satisfying the "at least one" part.…and zero or more
for
orif
clauses–There's oneif
clause to finish this list comp, satisfying this part of the description.
How about the second list comprehension, the one with the if..else
?
Take your time. Can you somehow fit this with the description of a list comprehension from the docs?
...
No?
That's because the extra spaces I put in are wrong. (A reminder that I'm adding these extra spaces to aid me with the discussion–don't do this in actual code, or you'll get a telling-off from PEP 8!)
Here's the same list comp with different spacing:
There are only two parts to this list comprehension:
Start with a single expression–Yep, this is one expression:
name if len(name) > 5 else "?"
More on this soon....followed by at least one
for
clause–There is still onefor
clause, so we're good.…and zero or more
for
orif
clauses–There's nothing left in the list comp, which still satisfies the condition thanks to the "zero" within it!
You had no issues with accepting 2 and 3 above as correct. But do you need more convincing to accept the first point?
This is a single expression:
This is the conditional expression. No, no, I didn't mean conditional statement, that's something else.
Confused? It's only quite recently that I stopped looking up the difference between statement and expression each time I needed to use the terms. So, a small diversion before returning to the list comps.
Statements and Expressions
Definitions are annoying, I know. (Confession: I like definitions, but don't tell anyone.) I'll keep this brief.
An expression can be evaluated as a value. Here's the easy way to figure out if something is an expression: Write it on a line in the REPL and press Enter/Return. Does it output a value? If it does, it's an expression. Here are some examples of expressions:
They all return a value. Therefore, they're all expressions. If you can put it after the equals sign in an assignment, then it's an expression.
A statement is a more general description of a line of code since statements may not evaluate to any value. For example, for _ in range(10):
doesn't evaluate to a value, neither does while True:
or if name == "Stephen"
.
The assignment statement is a statement:
The part after the equals is an expression, but the whole assignment line is a statement.
And in recent Python versions, there's also an assignment expression, often referred to as the walrus operator:
This does evaluate to a value:
All expressions are statements. But not all statements are expressions.
But I'm digressing. Back to the main topic for this article.
The Conditional Expression (A Ternary Operator)
If you're reading this, you're familiar with conditional statements. They're one of the first things you learn, such as if name == "Stephen:"
, and you can construct blocks with if
, elif
, and else
clauses, say. But you know all that already.
These are statements. They don't evaluate to a value. You cannot place these statements on the right-hand side of an equals sign in an assignment.
But there's also a conditional expression:
This is an expression since it evaluates to a value. The value returned is either whatever is before the if
or whatever comes after the else
, depending on whether the bit in the middle is True
or False
(or truthy or falsy).
And since it's an expression, you can assign the value to a variable name if you wish:
This conditional expression is often referred to as the ternary operator. But it's more accurately called a ternary operator, even though, as far as I'm aware, it's the only ternary operator in Python. A ternary operator is any operator that requires three operands in the same way that a binary operator requires two operands.
There are many binary operators in Python, such as ==
, >
, or
, and
, is
, and more. But there's only one ternary operator, the conditional expression:
<first operand> if <second operand> else <third operand>
So, back to the list comprehension:
The single expression required by the list comprehension description is the conditional expression (a ternary operator!) Here it is again:
name if len(name) > 5 else "?"
The three operands in this ternary operator are:
name
len(name) > 5
"?"
All three are expressions in their own right!
The list comprehension description then needs at least one for
clause, which you have. The last part of the list comp description requires zero or more for
or if
clauses, and there are zero of these in this list comp.
So it does make sense after all. [Top tip: if you think something doesn't make sense in Python, you probably ought to dig deeper. You'll find some sense when you do.]
Now, you'll never need trial and error again to remember what order everything goes in a list comprehension.
Code in this article uses Python 3.12
Stop Stack
#55
Two announcements:
• The Python Coding Book is out (Ebook available now, paperback in a few hours from Amazon) This is the First Edition, which follows from the "Zeroth" Edition that has been available online for a while—Just ask Google for ""python book"!
• The first hybrid cohort course starts in April: Beyond the Basics. This is included with membership of The Python Coding Place.
If you read my articles often, and perhaps my posts on social media, too, you've heard me talk about The Python Coding Place several times. But you haven't heard me talk a lot about is Codetoday Unlimited, a platform for teenagers to learn to code in Python. The beginner levels are free so everyone can start their Python journey. If you have teenage daughters or sons, or a bit younger, too, or nephews and nieces, or neighbours' children, or any teenager you know, really, send them to Codetoday Unlimited so they can start learning Python or take their Python to the next level if they've already covered some of the basics.
Each article is the result of years of experience and many hours of work. Hope you enjoy each one and find them useful. If you're in a position to do so, you can support this Substack further with a paid subscription. In addition to supporting this work, you'll get access to the full archive of articles. Alternatively, if you become a member of The Python Coding Place, you'll get access to all articles on The Stack as part of that membership. Of course, there's plenty more at The Place, too.
Appendix: Code Blocks
Code Block #1
names = ["Jim", "Audrey", "Clara", "Aaron", "Ishaan", "Amelia"]
long_names = [name for name in names if len(name) > 5]
print(long_names)
# OUTPUT: ['Audrey', 'Ishaan', 'Amelia']
Code Block #2
names = ["Jim", "Audrey", "Clara", "Aaron", "Ishaan", "Amelia"]
long_names = [name if len(name) > 5 else "?" for name in names]
print(long_names)
# OUTPUT: ['?', 'Audrey', '?', '?', 'Ishaan', 'Amelia']
Code Block #3
[ name for name in names if len(name) > 5 ]
[ name if len(name) > 5 else "?" for name in names ]
Code Block #4
[ name for name in names if len(name) > 5 ]
Code Block #5
[ name if len(name) > 5 else "?" for name in names ]
Code Block #6
[ name if len(name) > 5 else "?" for name in names ]
Code Block #7
name if len(name) > 5 else "?"
Code Block #8
5
# 5
1 + 1
# 2
int("5")
# 5
max(2, 5, 10)
# 10
10 == 5 + 5
# True
# # The next one assumes an import 'statement' first
random.randint(1, 10)
# 7
"hello" or []
# 'hello'
Code Block #9
my_name = "Stephen"
Code Block #10
my_name := "Stephen"
Code Block #11
final_var = (my_name := "Stephen")
my_name
# 'Stephen'
final_var
# 'Stephen'
Code Block #12
name = "Stephen"
name if len(name) > 5 else "?"
# 'Stephen'
name = "Bob"
name if len(name) > 5 else "?"
# '?'
Code Block #13
name = "Stephen"
final_result = name if len(name) > 5 else "?"
final_result
# 'Stephen'
name = "Bob"
final_result = name if len(name) > 5 else "?"
final_result
# '?'
Code Block #14
[ name if len(name) > 5 else "?" for name in names ]
This is great! Comprehensions can be confusing at first, but they come up quite often in Python.
One of the few drawbacks of formatters like black and ruff is how they handle comprehensions with conditionals. Here's how I've seen comprehensions formatted sometimes to make them more readable:
[
name
for name in names
if len(name) > 5
]
Both black and ruff collapse this into a single line. You can insert directives to skip reformatting on these lines, but that adds its own kind of clutter. Even if our comprehensions end up reformatted on a single line, knowing they can be broken up like this can help you make sense of them.
If you come across a comprehension you're struggling to make sense of, try writing it with the spacing that Stephen showed in this post, or breaking it up across lines. Even if a formatter undoes your work, your comprehension will remain. :)
Finally! I understand why list comprehensions behave like that