Python Interview Question: Iterables vs Iterators

An interviewer recently asked me the following question during an interview ‘What is an iterable?” I told him that data types that can be used in for in loops in Python such as lists and sets are called iterable. He accepted my answer but I recently learned that my answer was rather cursory. If you are not pythonically inclined, feel free to skip this post. Otherwise, in this post, I am going to elaborate on what is iterable and what is an iterator. This post is technically focused so watch out for code (gasp).

Iterable vs Iterator

Lets get some quick definitions out of the way. In python speak, any class that implements an __iter__ dunder/magic method is called an iterable. This means that it is possible for anyone to create an iterable class as long as the class implements an __iter__ method. Let’s go ahead and do that now. I will comment the code in line.

import ctypes # Imports the ctypes module. 
# https://docs.python.org/3/library/ctypes.html

class myArray:
    def __init__(self,size):
        assert size > 0 , 'Size must be greater than 0' # Ensures size > 0 because object will be immutable
        self._size=size
        # Create a data structure using the ctypes module of the provided size.
        self._array = (ctypes.py_object * size)() #this is simply an array that can store any python object
        # Initialize each element to None.
        for i in range(size):
            self._array[i] = None
    def __len__ (self):
        return self._size
    def clear(self, val):
        for i in range(self._size):
            self._array[i] = val
        return self._array
    def __str__(self):
        __printable = [None] * self._size
        for i in range(self._size):
            __printable[i] = self._array[i]
        return str(__printable)

Howoever, this class is not very useful. We have not implemented getter and setter methods yet so lets do that.

import ctypes # Imports the ctypes module. 
# https://docs.python.org/3/library/ctypes.html

class myArray:
    def __init__(self,size):
        assert size > 0 , 'Size must be greater than 0' # Ensures size > 0 because object will be immutable
        self._size=size
        # Create a data structure using the ctypes module of the provided size.
        self._array = (ctypes.py_object * size)() #this is simply an array that can store any python object
        # Initialize each element to None.
        for i in range(size):
            self._array[i] = None
    def __len__ (self):
        return self._size
    def __getitem__(self,ndx):
        assert ndx &gt;= 0 and ndx < self._size , 'index not in range'
        return self._array[ndx]
    def __setitem__(self,ndx, val):
        assert ndx &gt;= 0 and ndx < self._size , 'index not in range'
        self._array[ndx] = val
        return self._array[ndx]
    def clear(self, val):
        for i in range(self._size):
            self._array[i] = val
        return self._array
    def __str__(self):
        __printable = [None] * self._size
        for i in range(self._size):
            __printable[i] = self._array[i]
        return str(__printable)

faisalClass = myArray(3) #initializes an object of  size 3
print(len(faisalClass)) # Prints the size of the object
faisalClass[0] = 'testing' # Set the item at index 0 to the string 'testing'
print(faisalClass[0])  # Print the item at index 0

Even though this class implements a getter method, it is not technically iterable. We must implements an __iter__ method for this class to be iterable. Let’s do that

import ctypes # Imports the ctypes module. 
# https://docs.python.org/3/library/ctypes.html

class myArray:
    def __init__(self,size):
        assert size &gt; 0 , 'Size must be greater than 0' # Ensures size &gt; 0 because object will be immutable
        self._size=size
        # Create a data structure using the ctypes module of the provided size.
        self._array = (ctypes.py_object * size)() #this is simply an array that can store any python object
        # Initialize each element to None.
        for i in range(size):
            self._array[i] = None
    def __len__ (self):
        return self._size
    def __getitem__(self,ndx):
        assert ndx &gt;= 0 and ndx < self._size , 'index not in range'
        return self._array[ndx]
    def __setitem__(self,ndx, val):
        assert ndx &gt;= 0 and ndx < self._size , 'index not in range'
        self._array[ndx] = val
        return self._array[ndx]
    def clear(self, val):
        for i in range(self._size):
            self._array[i] = val
        return self._array
    def __str__(self):
        __printable = [None] * self._size
        for i in range(self._size):
            __printable[i] = self._array[i]
        return str(__printable)
    def __iter__(self):
        return FaisalArrayIterator(self._array)

class FaisalArrayIterator():
    def __init__(self,array):
        self._curndx = 0
        self._array = array
    def __iter__(self):
        return self
    def __next__(self):
        if self._curndx < len(self._array) :
            entry = self._array[ self._curndx ]
            self._curndx += 1
            return entry
        else :
            raise StopIteration

faisalClass = myArray(3) #initializes an object of  size 3
print(len(faisalClass)) # Prints the size of the object
faisalClass[0] = 'testing' # Set the item at index 0 to the string 'testing'
print(faisalClass[0])  # Print the item at index 0
for item in faisalClass:
    print(item)

So what’s going on here? Go ahead and put a breakpoint on line 44 and run the code and step through it. You will notice two things. First, the iter method is called when the for loop is run on the object. Since this class implements the __iter__ dunder method, the class is iterable. However, the iteration actually happens in a different class called FaisalArrayIterator. This class is the one that actually does the iteration. It tracks the current state of the iteration and returns the current item.

Long story short: any class that implements an __iter__ dunder/magic method is iterable while any class that implements that __next__ method is the iterator. We could have used the class iteself to track its own internal state and the class would have become its own iterator (the class would have been an iterable and an iterator) but that is not considered Pythonic.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s