Iterators & Iterables

1. Overview#

Every for loop in Python relies on a simple, consistent protocol under the hood. Understanding it demystifies how for x in my_list, for line in file, and for item in generator all work through the same mechanism — and lets you build your own custom iterable objects.

A generator (covered in functions.md) is simply the easiest way to create an iterator — this note covers the underlying protocol generators are built on.

---

2. Iterable vs Iterator — The Key Distinction#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
numbers = [1, 2, 3] # a list is Iterable
iterator = iter(numbers) # calling iter() on it gives you an Iterator

print(next(iterator)) # 1
print(next(iterator)) # 2
print(next(iterator)) # 3
print(next(iterator)) # raises StopIteration

A for loop is essentially syntax sugar for repeatedly calling iter() then next() until StopIteration is raised — it catches that exception for you automatically.

---

3. How for Actually Works#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
numbers = [10, 20, 30]

# This for loop...
for n in numbers:
 print(n)

# ...is roughly equivalent to:
iterator = iter(numbers)
while True:
 try:
 n = next(iterator)
 except StopIteration:
 break
 print(n)

---

4. Building a Custom Iterator#

Implement __iter__ (returns the iterator object, usually self) and __next__ (returns the next value or raises StopIteration).

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
class CountUp:
 def __init__(self, start, end):
 self.current = start
 self.end = end

 def __iter__(self):
 return self # the object is its own iterator

 def __next__(self):
 if self.current > self.end:
 raise StopIteration
 value = self.current
 self.current += 1
 return value

for n in CountUp(1, 5):
 print(n) # 1 2 3 4 5

4.1 Separating Iterable and Iterator#

It's often cleaner to keep the "container" (Iterable) separate from the object doing the iterating (Iterator) — this allows multiple independent loops over the same data at once.

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
class NumberRange:
 def __init__(self, start, end):
 self.start = start
 self.end = end

 def __iter__(self):
 return NumberRangeIterator(self.start, self.end)

class NumberRangeIterator:
 def __init__(self, current, end):
 self.current = current
 self.end = end

 def __iter__(self):
 return self

 def __next__(self):
 if self.current > self.end:
 raise StopIteration
 value = self.current
 self.current += 1
 return value

numbers = NumberRange(1, 3)
print(list(numbers)) # [1, 2, 3]
print(list(numbers)) # [1, 2, 3] — works again, unlike a single-use generator

---

5. Making a Class Iterable with __getitem__#

Older-style iterables can also work by implementing __getitem__ — Python falls back to calling it with increasing indices (0, 1, 2, ...) until an IndexError is raised.

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
class Squares:
 def __init__(self, n):
 self.n = n

 def __getitem__(self, index):
 if index >= self.n:
 raise IndexError
 return index ** 2

for sq in Squares(5):
 print(sq) # 0 1 4 9 16

---

6. Generators as Iterators (Recap)#

A generator function automatically implements the iterator protocol for you — no need to write __iter__/__next__ by hand.

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
def count_up(start, end):
 current = start
 while current <= end:
 yield current
 current += 1

gen = count_up(1, 5)
print(next(gen)) # 1
print(next(gen)) # 2
print(list(gen)) # [3, 4, 5] — continues from where it left off

---

7. The itertools Module#

The standard library's toolkit for combining and transforming iterators efficiently, without building intermediate lists.

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
import itertools

# chain — combine multiple iterables into one
combined = list(itertools.chain([1, 2], [3, 4], [5]))
print(combined) # [1, 2, 3, 4, 5]

# count — infinite counter
counter = itertools.count(start=10, step=5)
print([next(counter) for _ in range(3)]) # [10, 15, 20]

# cycle — repeat a sequence forever
colors = itertools.cycle(["red", "green", "blue"])
print([next(colors) for _ in range(5)]) # ['red', 'green', 'blue', 'red', 'green']

# islice — slice an iterator without loading it all into memory
first_three = list(itertools.islice(itertools.count(1), 3))
print(first_three) # [1, 2, 3]

# groupby — group consecutive items by a key
data = [("A", 1), ("A", 2), ("B", 3), ("B", 4)]
for key, group in itertools.groupby(data, key=lambda x: x[0]):
 print(key, list(group))
# A [('A', 1), ('A', 2)]
# B [('B', 3), ('B', 4)]

# permutations & combinations
print(list(itertools.permutations([1, 2, 3], 2)))
print(list(itertools.combinations([1, 2, 3], 2)))

---

8. Common Pitfalls#

INCORRECT: Exhausting an Iterator and Reusing It#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
numbers = iter([1, 2, 3])
print(list(numbers)) # [1, 2, 3]
print(list(numbers)) # [] — already exhausted!

CORRECT: Fix — Rebuild the Iterator, or Use an Iterable Instead#

🐍 PythonInteractive WebAssembly
EditCopyAsk AIRun Code
numbers = [1, 2, 3] # a list can be iterated over repeatedly
print(list(numbers))
print(list(numbers))

INCORRECT: Forgetting StopIteration in a Custom __next__#

Without raising StopIteration, a custom iterator will loop forever in a for loop.

---

9. Summary & Best Practices Checklist#

• Know the distinction: Iterable has __iter__, Iterator has __iter__ and __next__.
• Prefer a generator function over a hand-written class-based iterator whenever possible — far less code.
• Use a class-based iterator only when you need the sequence to be re-iterated fresh each time.
• Reach for itertools before writing manual loops for chaining, slicing, or grouping iterators.
• Remember an exhausted iterator/generator can't be reused — rebuild it if you need another pass.