Using Comprehensions

Comprehensions are simple to write and mostly easy to understand, however, they can be used in many “creative” ways. The key is generally the expression you use which you need to think of as transforming the iterable. For example to create a zeroed list you could use:

myList = [0 for i in range(1,11)]

sets the list elements to 0. In this case the transformation of the iteration is to map every element to zero. If you want to set the list to random numbers you could use:

myList = [random.random() for i in range(1,11)]

In the case where the expression doesn’t depend on the variable all the iterable does is determine the number of elements in the result.

In most cases the expression will depend on the iterable and this will result in a true transformation of the elements. For example:

myList = [i*2 for i in range(1,11)]

multiplies each element by 2 to convert them into even values 2, 4,6 and so on.

In addition to the transformation caused by the expression you can also use a conditional to filter the elements, for example:

myList=[i for i in range(1,11) if i%2 == 0]

In this case i%2 is zero only if i is even and so the final list is:

[2, 4, 6, 8, 10]

That is, we have picked out the even elements from the iteration.

So far we have only used range as an example of an iterable, but you can use any iterable and this means you can transform text strings. For example:

s = set(("mike","lucy","harry"))
myList = [str.upper(w) for w in s]

In this case the iteration is provided by a set and each element is converted to upper case, displaying:

['MIKE','LUCY', 'HARRY']

Also notice that we started with a set and transformed it into a list.

Dictionary comprehensions are just as easy, but now we have the possibility of constructing two expressions, one for the key and one for the value. For example:

myList = ["mike","lucy","harry"]
myDict = {str.upper(w):0 for w in myList}

creates a dictionary set to:

{'MIKE': 0, 'LUCY': 0, 'HARRY': 0}

The usual problem in using a dictionary comprehension is deriving two expressions for the key and value. You can make use of data stored in other data structures for either part of the new element. For example:

myList = ["mike","lucy","harry"]
ages = [18,16,23]
myDict = {w[1]:ages[w[0]] for w in enumerate(myList)}

In this case the key is derived from each item in myList, but the value is derived from the ages list. Notice the way that enumerate is used to derive an index that can be used to select the appropriate element of ages. The result is that myDict contains:

{'mike': 18, 'lucy': 16, 'harry': 23}

Finally a generator comprehension can involve infinite iterations, for example:

def genSquared():
    x = 0
    while True:
        x = x+1
        yield x**2
genOddSquare = (s for s in genSquared() if s%2 != 0)
for i in genOddSquare:
    print(i)

In this case the generator function genSquared produces the squares of the integers and it runs for as long as its __next__ method is called. The generator comprehension transforms genSquared into an iteration of just the squares that are odd. The work in this case is done by the conditional which selects only the odd elements generated by genSquare. The final for loop that uses the generator object runs until you hit the Break key. Notice that in the case of a generator comprehension it is possible to write a condition that will never be satisfied and hence the call to the generator object will never return.

Finally it is worth noting that comprehensions can occur in unexpected places. In fact, anywhere a for loop occurs there is the possibility of using a comprehension. For example, if you want to print a row of numbers formatted with two digits you might use:

for i in range(20):

    print(f"{i:2}",end = " ")

which displays:

0  1  2  3  4  5  6  7  8  9 10 11 12 13 14 15 16 17 18 19

This can be written as a single print using a generator comprehension:

print(*(f"{i:2}" for i in range(20)))

Notice that we need the unpacking operator to both evaluate the comprehension and to provide the results as parameters to the print function. As there is only a single print function, we also don’t need the end = " ". Another difference is that as the numbers are passed to the print function as separate parameters they are displayed with a single space between them.

Is this better? As long as you are very happy about reading comprehensions, it is more compact, but it is arguably more difficult to understand.

Nested Comprehensions

The expression in a comprehension can be any valid expression and this includes another comprehension. This gives rise to the idea of a nested comprehension and these can be difficult to understand. The key idea is that the expression in a comprehension is fully evaluated to an object and this object is added to the data structure being created. What this means is that when the expression is itself a comprehension it is fully evaluated and it returns an object of the appropriate type, for example:

m = [[j for j in range(3)] for i in range(5)]

This is a nested list comprehensions, as you can tell from the fact that the expression for the outer comprehension is itself a comprehension.

The way to understand this is to first look at the inner comprehension:

[j for j in range(3)]

This creates a list [0,1,2] and it does this every time the outer comprehension evaluates its expression which it does for i = 0,1,2,3,4. You can see that the result is the list:

[[0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2], [0, 1, 2]]

You can see that this is a list of five repeats of [0, 1, 2], which is what you would expect if you understand the basic idea of nesting.

In the more general case, the inner expression can involve the result of the outer comprehension, for example:

m=[[j*i for j in range(3)] for i in range(5)]

In this case the inner comprehension uses the current value of i when it is evaluated. The result is the list:

[[0, 0, 0], [0, 1, 2], [0, 2, 4], [0, 3, 6], [0, 4, 8]]

In book but not in this extract

• Python Matrices
• Function-Oriented Programming – Map, Filter and Zip

 Summary

• Comprehensions are easy and compact ways of writing iterations and are especially useful for creating initialized lists, dictionaries, sets and generators.

• Generator comprehensions create iterators which are lazy-evaluated.

• The range of applications of comprehensions is more than you might expect and whenever you are thinking of using an iterative for loop you should consider whether a comprehension would be simpler.

• Comprehensions can be nested to create double iterations which are particularly relevant to lists used as matrices.

• Python is not a functional language but it does have some useful function-oriented features. It supports map, filter and zip, although map and filter can be replaced by comprehensions.

• The zip function allows comprehensions to iterate through multiple iterables.

Programmer's Python
Everything is Data

Is now available as a print book: Amazon

Contents

1. Python – A Lightning Tour
2. The Basic Data Type – Numbers
      Extract: Bignum
   
3. Truthy & Falsey
4. Dates & Times
      Extract Naive Dates ***NEW!!!
5. Sequences, Lists & Tuples
      Extract Sequences 
6. Strings
      Extract Unicode Strings
7. Regular Expressions
8. The Dictionary
      Extract The Dictionary 
9. Iterables, Sets & Generators
      Extract  Iterables 
10. Comprehensions
       Extract  Comprehensions 
11. Data Structures & Collections
       Extract Stacks, Queues and Deques
       Extract Named Tuples and Counters
12. Bits & Bit Manipulation
       Extract Bits and BigNum 
13. Bytes
       Extract Bytes And Strings
       Extract Byte Manipulation 
14. Binary Files
15. Text Files
16. Creating Custom Data Classes
        Extract A Custom Data Class 
17. Python and Native Code
        Extract   Native Code
    Appendix I Python in Visual Studio Code
    Appendix II C Programming Using Visual Studio Code

<ASIN:1871962765>

<ASIN:1871962749>

<ASIN:1871962595>

<ASIN:B0CK71TQ17>

<ASIN:187196265X>

Related Articles

Creating The Python UI With Tkinter

Creating The Python UI With Tkinter - The Canvas Widget

The Python Dictionary

Arrays in Python

Advanced Python Arrays - Introducing NumPy

Comments

Make a Comment or View Existing Comments Using Disqus

or email your comment to: comments@i-programmer.info

To be informed about new articles on I Programmer, sign up for our weekly newsletter, subscribe to the RSS feed and follow us on Twitter, Facebook or Linkedin.