Iterators are one of the great simplifications of design that make Python so productive. Find out how it all works in this extract from my new book Programmer's Python: Everything is Data.

Programmer's Python
Everything is Data

Is now available as a print book: Amazon

Contents

1. Python – A Lightning Tour
2. The Basic Data Type – Numbers
      Extract: Bignum
   
3. Truthy & Falsey
4. Dates & Times
5. Sequences, Lists & Tuples
      Extract Sequences 
6. Strings
      Extract Unicode Strings
7. Regular Expressions
8. The Dictionary
      Extract The Dictionary 
9. Iterables, Sets & Generators
      Extract  Iterables 
10. Comprehensions
       Extract  Comprehensions 
11. Data Structures & Collections
12. Bits & Bit Manipulation
         Extract Bits and BigNum ***NEW!!!
13. Bytes
        Extract Bytes And Strings
        Extract Byte Manipulation 
14. Binary Files
15. Text Files
16. Creating Custom Data Classes
        Extract A Custom Data Class 
17. Python and Native Code
        Extract   Native Code
    Appendix I Python in Visual Studio Code
    Appendix II C Programming Using Visual Studio Code

<ASIN:1871962765>

<ASIN:1871962749>

<ASIN:1871962595>

<ASIN:187196265X>

Although iterables are a very basic data type and very simple, many Python data types are iterables. Sequences and mappings which include lists, tuples and dictionaries are all iterables. An iterable is a mixin that you can add to almost any data class without having to worry about an inheritance hierarchy. They are so ubiquitous that it makes it difficult to focus on the idea in isolation. It is also difficult to find a “pure” example of an iterable, but the set comes closest. Finally iterables provide a way of generating sets, sequences and dictionaries via the powerful idea of a comprehension, which is the subject of the next chapter.

Iterables, Enumerables and Sequences

An iteration is simply the act of taking each item stored in a container in turn. It is typified by a for loop. A container is a class that, as its name suggests, contains other classes. Whenever you write something like:

for x in container:

you are performing an iteration. Iteration is simple but we tend to give it more properties than are absolutely necessary.

Python has three distinct types – iterable, enumerable and sequence.

• Iterable – a container that you can access one element at a time in an arbitrary order. An iterable only has to have the ability to give you the next element.

• Enumerable – an iterable that also has an integer associated with each element. That is, you can access each element in the container by asking for the first, second, third and so on. Notice that this doesn’t mean that you can go to any particular element without accessing all of the elements that come before it. You can think of an enumerable as a container that only allows sequential access.

• Sequence – an enumerable that also allows you to access any element in any order using indexing. A sequence is a container that allows random access to any element.

A sequence is necessarily an enumerable, which in turn is necessarily an iterable, but the reverse isn’t true. An iterable doesn’t have to be an enumerable and neither does it have to be a sequence. For example, a list is a sequence and you can write:

print(myList[42])

to display element number 42.

You can iterate through a sequence as if it was enumerable using the index:

for i in range(0,42):
	print(myList[i])

and this is the way it is done in other languages. Notice that you are iterating through a range and in turn using this to index the list.

Enumeration implies an order, first, second and so on, but iteration doesn’t – it just means next. Of course, you can iterate through containers that do have an order such as a list:

for x in myList:
	print(x)

but there are containers that you can iterate through that don’t support indexing or enumeration.

Iteration is more basic than enumeration or sequencing.

Iterators

Any Python object can be made into an iterable. All it has to do is support the __iter__ method, which returns an iterator object. An iterator object has to have a __next__ method, which returns elements one after another until there are no more when it throws a StopIteration exception. An iterator doesn’t promise to deliver the items in any particular order and indeed might return them in a different order each time. All an iterator promises is to provide the next element until it runs out of elements. The reason that the iterator has to be an object rather than just a function that gives you the next element is explained later.

An iterator object should also have an __iter__ method, which returns a reference to the iterator object. This allows the iterator object to be used as if it was an iterable in its own right.

For example, we can create a random iterable which simply returns a random number for each of its elements:

class randIt(object):
    def __iter__(self):
        return randItIterator()

In this case the __iter__ method returns an iterator object which provides the random numbers each time it is used.

To make this work we need to define the randItIterator class:

class randItIterator():
    def __next__(self):
        return random.random()

The iterator class simply returns a random number as the result of __next__. Putting this together we can now write a for loop that uses an instance of randIt:

numbers=randIt()
for x in numbers:
    print(x)

This will print random numbers until you stop it by pressing the Break key.

The numbers iterable behaves as if it was a container with an infinite series of random numbers stored within it. You can explicitly call __next__ to get the next item in the iteration or you can use the next built-in function to do the same job:

numbers=randIt()
print(numbers.__next()__)
print(next(numbers))

displays two random numbers or, more generally, the first two elements in the iteration.

You can also specify a default as a second parameter to next, which will be used if the iteration is complete and there isn’t a next element. Notice that next simply calls __next__ and substitutes the default if specified and there is no pending item.

If you define a simple __iter__ method within the iterator it too can be used as an iterable, in a for loop for example. That is, if we add:

def __iter__(self):
    return self

to randItItertor it can be used in a for loop in place of the iterable:

numbers=randIt()
numberIterator=numbers.__iter__()
for x in numberIterator:
    print(x)

In this case we explicitly return the iterator and then use it in the for loop in place of the iterable. This is really the only use for the __iter_ method within the iterator, but it is still advisable to include it.

The full program is:

import random
class randItIterator():
    def __next__(self):
        return random.random()
    def __iter__(self):
        return self
class randIt(object):
    def __iter__(self):
        return randItIterator()  
      
numbers=randIt()
for x in numbers:
    print(x)