Iterators

Overview#

Iterators are the mechanism behind every for loop in Python. Any object you can loop over — lists, dictionaries, files, strings, generators — does so by exposing an iterator. Understanding this protocol deeply lets you build your own custom iterable objects, work confidently with the itertools module, and understand exactly why generators (covered in the previous notebook) behave the way they do. This notebook covers the iterator protocol from first principles, the difference between an iterable and an iterator, building custom iterator classes, the built-in iter()/next() functions, and the most useful tools in itertools.

1. Iterable vs Iterator: The Key Distinction#

These two terms are related but distinct, and mixing them up is one of the most common sources of confusion for newcomers.

• An iterable is any object you can loop over — it implements __iter__(), which returns an iterator.
• An iterator is the object that actually produces values one at a time — it implements both __iter__() (returning itself) and __next__() (returning the next value, or raising StopIteration when done).

Python 3.12 (WASM)
EditCopyAsk AIRun Code
numbers = [1, 2, 3]        # numbers is an ITERABLE
iterator = iter(numbers)   # iter() asks the iterable for an ITERATOR

print(next(iterator))   # 1
print(next(iterator))   # 2
print(next(iterator))   # 3
print(next(iterator))   # raises StopIteration

A for loop does exactly this behind the scenes:

Python 3.12 (WASM)
EditCopyAsk AIRun Code
# What Python does internally for:
# for item in numbers:
#     print(item)

iterator = iter(numbers)
while True:
    try:
        item = next(iterator)
    except StopIteration:
        break
    print(item)

Why the distinction matters#

A list is iterable, but calling iter() on it every time gives you a fresh iterator starting from the beginning. An iterator, once exhausted, stays exhausted.

Python 3.12 (WASM)
EditCopyAsk AIRun Code
numbers = [1, 2, 3]

for x in numbers:
    print(x)         # 1, 2, 3

for x in numbers:     # works again — a NEW iterator is created each time
    print(x)          # 1, 2, 3

it = iter(numbers)
list(it)              # [1, 2, 3] — consumes the iterator
list(it)              # [] — already exhausted, same iterator object

2. Building a Custom Iterator (Class-Based)#

Implementing __iter__ and __next__ lets any custom class support for loops, list(), sum(), and every other iteration-based built-in.

Python 3.12 (WASM)
EditCopyAsk AIRun Code
class Countdown:
    def __init__(self, start):
        self.current = start

    def __iter__(self):
        return self               # the object is its own iterator

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value


for n in Countdown(5):
    print(n)   # 5, 4, 3, 2, 1

print(list(Countdown(3)))   # [3, 2, 1]

Separating iterable and iterator (a more robust pattern)#

The pattern above works, but it means Countdown(5) can only be iterated once — the object's own state (self.current) gets consumed. For a reusable iterable that supports multiple independent loops at once, separate the two roles into different classes.

Python 3.12 (WASM)
EditCopyAsk AIRun Code
class CountdownIterable:
    def __init__(self, start):
        self.start = start

    def __iter__(self):
        return CountdownIterator(self.start)   # new iterator each time


class CountdownIterator:
    def __init__(self, current):
        self.current = current

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= 0:
            raise StopIteration
        value = self.current
        self.current -= 1
        return value


countdown = CountdownIterable(3)
print(list(countdown))   # [3, 2, 1]
print(list(countdown))   # [3, 2, 1] — works again! Each iter() call is independent

This separation is exactly what built-in types like list do internally: the list itself is iterable, but each for loop over it gets its own independent iterator.

3. iter() with a Sentinel Value#

A lesser-known form of iter() takes two arguments: a callable, and a "sentinel" value that signals when to stop. This is useful for reading until a specific marker is reached.

Python 3.12 (WASM)
EditCopyAsk AIRun Code
# Keep reading lines from a file until an empty line is found
with open("data.txt") as f:
    for line in iter(f.readline, ""):
        print(line.strip())

iter(callable, sentinel) repeatedly calls callable(), yielding each result, and stops as soon as a result equals sentinel.

4. Generators Are Iterators#

Every generator object automatically satisfies the iterator protocol — it has __iter__ (returning itself) and __next__ (resuming execution) built in for free. This means the Countdown class above can be written far more simply:

Python 3.12 (WASM)
EditCopyAsk AIRun Code
def countdown(start):
    current = start
    while current > 0:
        yield current
        current -= 1

for n in countdown(5):
    print(n)   # 5, 4, 3, 2, 1

Whenever a custom iterator's logic can be expressed as a straightforward loop with yield, prefer the generator function — it's shorter, has no manual state-tracking bugs to introduce, and is easier to read.

5. The itertools Module#

The standard library's itertools module provides a large set of fast, memory-efficient building blocks for working with iterators.

Infinite iterators#

Python 3.12 (WASM)
EditCopyAsk AIRun Code
from itertools import count, cycle, repeat

for n in count(10, step=2):     # 10, 12, 14, ... forever
    if n > 16:
        break
    print(n)

colors = cycle(["red", "green", "blue"])
for _ in range(5):
    print(next(colors))          # red, green, blue, red, green

for _ in repeat("hi", 3):
    print(_)                     # hi, hi, hi

Combining and slicing iterators#

Python 3.12 (WASM)
EditCopyAsk AIRun Code
from itertools import chain, islice, zip_longest

a = [1, 2, 3]
b = [4, 5, 6]
print(list(chain(a, b)))              # [1, 2, 3, 4, 5, 6] — concatenate lazily

print(list(islice(count(0), 5)))       # [0, 1, 2, 3, 4] — slice an iterator

names = ["Alice", "Bob"]
scores = [90, 85, 70]
print(list(zip_longest(names, scores, fillvalue="N/A")))
# [('Alice', 90), ('Bob', 85), ('N/A', 70)]

Grouping and combinations#

Python 3.12 (WASM)
EditCopyAsk AIRun Code
from itertools import groupby, permutations, combinations

# groupby requires the input to already be sorted by the grouping key
words = ["apple", "avocado", "banana", "blueberry", "cherry"]
for letter, group in groupby(words, key=lambda w: w[0]):
    print(letter, list(group))
# a ['apple', 'avocado']
# b ['banana', 'blueberry']
# c ['cherry']

print(list(permutations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 1), (2, 3), (3, 1), (3, 2)]

print(list(combinations([1, 2, 3], 2)))
# [(1, 2), (1, 3), (2, 3)]

Filtering#

Python 3.12 (WASM)
EditCopyAsk AIRun Code
from itertools import filterfalse, takewhile, dropwhile

numbers = [1, 3, 5, 4, 7, 8, 9]

print(list(takewhile(lambda n: n % 2 != 0, numbers)))   # [1, 3, 5]
print(list(dropwhile(lambda n: n % 2 != 0, numbers)))    # [4, 7, 8, 9]
print(list(filterfalse(lambda n: n % 2 == 0, numbers)))  # [1, 3, 5, 7, 9]

6. Built-in Functions That Rely on the Iterator Protocol#

Many everyday built-ins work with any iterable precisely because they're written against this protocol, not against specific types like list.

Python 3.12 (WASM)
EditCopyAsk AIRun Code
numbers = (n for n in range(5))   # a generator, still just an iterable

print(sum(numbers))       # 10 — works on any iterable, not just lists
print(max([3, 7, 2]))      # 7
print(sorted("dcba"))       # ['a', 'b', 'c', 'd']
print(any(n > 3 for n in [1, 2, 3]))   # False
print(all(n > 0 for n in [1, 2, 3]))    # True
print(list(map(str, [1, 2, 3])))         # ['1', '2', '3']
print(list(filter(lambda n: n > 1, [1, 2, 3])))  # [2, 3]

7. Checking Iterability#

Python 3.12 (WASM)
EditCopyAsk AIRun Code
from collections.abc import Iterable, Iterator

print(isinstance([1, 2, 3], Iterable))    # True
print(isinstance([1, 2, 3], Iterator))    # False — a list is iterable but not itself an iterator

it = iter([1, 2, 3])
print(isinstance(it, Iterator))           # True

A quick manual check: an object is iterable if hasattr(obj, "__iter__"), and an iterator additionally has __next__.

Notes#

• Every for loop, list/dict/set comprehension, * unpacking, and function like sum()/sorted()/zip() relies on the iterator protocol under the hood.
• An iterator is exhausted after one full pass; if you need to loop over the same data multiple times, keep the original iterable around and call iter() again rather than trying to reuse a consumed iterator.
• Most itertools functions return iterators, not lists — wrap the result in list(...) when you need to inspect or reuse the values, but leave it as an iterator in a pipeline for memory efficiency.
• itertools.groupby only groups consecutive matching items — sort the input first if you need groups across the entire dataset.
• When a custom iterator's logic is a simple loop, a generator function is almost always simpler and less error-prone than a hand-written __iter__/__next__ class.

Best Practices Checklist#

• Understand the difference between an iterable (has __iter__) and an iterator (has __iter__ and __next__)
• Separate iterable and iterator classes when an object needs to support multiple independent loops over the same data
• Prefer a generator function over a hand-written iterator class whenever the logic is a simple loop with yield
• Reach for itertools before writing manual loops for chaining, slicing, grouping, or combining iterables
• Remember itertools.groupby requires pre-sorted input by the same key to group correctly
• Convert an iterator to a list only when you actually need to reuse or inspect all values more than once
• Use isinstance(obj, collections.abc.Iterable) rather than duck-typing checks when you need to verify iterability explicitly