Python collections

last modified October 29, 2026 Python 3.15

In this article we show how to use the collections module. The module provides specialized container types that extend the built-in dict, list, set, and tuple.

The module contains the following types:

• Counter – a dictionary for counting hashable objects
• defaultdict – a dictionary that creates missing values
• OrderedDict – a dictionary with order-related operations
• ChainMap – a view of several mappings as one
• UserDict, UserList, UserString – base classes for custom containers
• deque – a double-ended queue
• namedtuple – a factory for tuples with named fields

The deque and namedtuple types have their own articles: Python deque and Python namedtuple. Here we cover the rest of the module, including the collections.abc submodule.

Counting with Counter

A Counter is a dict subclass that maps elements to their counts.

import re
from collections import Counter

text = """
The sky is blue. The sea is blue too. The grass is green
and the sun is yellow.
"""

words = re.findall(r'\w+', text.lower())
counts = Counter(words)

print(counts.most_common(3))
print(counts['blue'], counts['red'])
print(counts.total(), len(counts))

We count the words of a text. The Counter constructor accepts any iterable and counts its elements.

print(counts.most_common(3))
print(counts['blue'], counts['red'])

The most_common method returns the most frequent elements with their counts. Elements with equal counts keep the order in which they were first encountered. Looking up a missing element returns 0 instead of raising a KeyError.

print(counts.total(), len(counts))

The total method returns the sum of all counts, that is, the number of words. The len function returns the number of distinct words.

$ python counter_basic.py
[('the', 4), ('is', 4), ('blue', 2)]
2 0
18 11

Counter arithmetic

Counters support arithmetic and set-like operators. The results keep only positive counts.

from collections import Counter

monday = Counter(apples=3, pears=1, plums=4)
tuesday = Counter(apples=1, pears=2, cherries=5)

print(monday + tuesday)
print(monday - tuesday)
print(monday & tuesday)
print(monday | tuesday)
print(monday ^ tuesday)

We have the fruit sales of two days.

print(monday + tuesday)
print(monday - tuesday)

Addition adds the counts. Subtraction subtracts them and drops the elements whose count is zero or negative.

print(monday & tuesday)
print(monday | tuesday)

The intersection takes the minimum of each count and keeps only elements found in both counters. The union takes the maximum.

print(monday ^ tuesday)

The symmetric difference operator was added in Python 3.15. It returns the absolute difference of the counts: how much the two days differ for each fruit.

$ python counter_math.py
Counter({'cherries': 5, 'apples': 4, 'plums': 4, 'pears': 3})
Counter({'plums': 4, 'apples': 2})
Counter({'apples': 1, 'pears': 1})
Counter({'cherries': 5, 'plums': 4, 'apples': 3, 'pears': 2})
Counter({'cherries': 5, 'plums': 4, 'apples': 2, 'pears': 1})

Updating counters

The update and subtract methods change a counter in place.

from collections import Counter

stock = Counter(apples=4, pears=2)

stock.update(['apples', 'plums', 'plums'])
print(stock)

stock.subtract(apples=5, pears=1)
print(stock)

print(+stock)
print(sorted(stock.elements()))

Unlike dict.update, the update method of a counter adds counts instead of replacing them. It accepts an iterable of elements or a mapping of counts.

stock.subtract(apples=5, pears=1)

The in-place methods keep zero and negative counts. Here the count of apples drops to 0.

print(+stock)
print(sorted(stock.elements()))

The unary plus operator removes elements with zero or negative counts. The elements method returns an iterator that repeats each element as many times as its count.

$ python counter_update.py
Counter({'apples': 5, 'pears': 2, 'plums': 2})
Counter({'plums': 2, 'pears': 1, 'apples': 0})
Counter({'plums': 2, 'pears': 1})
['pears', 'plums', 'plums']

Grouping with defaultdict

A defaultdict calls a factory function to create a value for a missing key. The factory is passed to the constructor.

from collections import defaultdict

words = ['sky', 'forest', 'sun', 'falcon', 'storm', 'fog', 'cloud']

by_letter = defaultdict(list)
for word in words:
    by_letter[word[0]].append(word)

print(by_letter)
print(dict(by_letter))

lengths = defaultdict(int)
for word in words:
    lengths[len(word)] += 1

print(dict(lengths))

We group words by their first letter and count words by length.

by_letter = defaultdict(list)
for word in words:
    by_letter[word[0]].append(word)

When a letter is seen for the first time, list is called to create an empty list, which is stored under the key and then appended to. Without defaultdict, we would have to check the key or call setdefault first.

lengths = defaultdict(int)
for word in words:
    lengths[len(word)] += 1

Calling int without arguments returns 0, which makes it a natural factory for counting. Converting the result with dict gives a cleaner representation.

$ python defaultdict_group.py
defaultdict(<class 'list'>, {'s': ['sky', 'sun', 'storm'], 'f': ['forest', 'falcon', 'fog'], 'c': ['cloud']})
{'s': ['sky', 'sun', 'storm'], 'f': ['forest', 'falcon', 'fog'], 'c': ['cloud']}
{3: 3, 6: 2, 5: 2}

When default values are created

The factory is called only by the square bracket lookup. Other ways of accessing the dictionary behave as with a plain dict.

from collections import defaultdict

scores = defaultdict(list)

print(scores.get('john'))
print('john' in scores)

print(scores['john'])
print('john' in scores)

scores.default_factory = None

try:
    scores['paul']
except KeyError as e:
    print('KeyError:', e)

The get method and the in operator do not create values. Merely reading scores['john'] inserts an empty list, which is easy to do by accident, for example in a condition.

scores.default_factory = None

The factory is stored in the default_factory attribute. Setting it to None turns the dictionary back into one that raises KeyError, which is useful after the data has been built.

$ python defaultdict_missing.py
None
False
[]
True
KeyError: 'paul'

A tree of defaultdicts

A factory function that returns a new defaultdict of the same kind creates nested dictionaries of any depth on demand.

import json
from collections import defaultdict


def tree():
    return defaultdict(tree)


config = tree()
config['database']['primary']['host'] = 'db1.example.com'
config['database']['primary']['port'] = 5432
config['database']['replica']['host'] = 'db2.example.com'
config['cache']['ttl'] = 300

print(json.dumps(config, indent=2))

The tree function is its own factory. Each missing key creates another tree, so we can assign to deeply nested keys without creating the intermediate levels. Since defaultdict is a dict subclass, the result can be serialized with json.dumps.

$ python defaultdict_tree.py
{
  "database": {
    "primary": {
      "host": "db1.example.com",
      "port": 5432
    },
    "replica": {
      "host": "db2.example.com"
    }
  },
  "cache": {
    "ttl": 300
  }
}

OrderedDict

Since Python 3.7, a plain dict keeps the insertion order. The OrderedDict class is still useful for its reordering operations and its order-sensitive equality.

from collections import OrderedDict

tasks = OrderedDict(write='todo', test='todo', deploy='todo')

tasks.move_to_end('write')
print(list(tasks))

tasks.move_to_end('deploy', last=False)
print(list(tasks))

print(tasks.popitem())
print(tasks.popitem(last=False))

a = OrderedDict(x=1, y=2)
b = OrderedDict(y=2, x=1)
print(a == b, dict(a) == dict(b))

The move_to_end method moves a key to the end, or to the beginning with last=False.

print(tasks.popitem())
print(tasks.popitem(last=False))

The popitem method removes the last item, or the first one with last=False. A plain dictionary can remove only the last item.

print(a == b, dict(a) == dict(b))

Two ordered dictionaries are equal only if their items are in the same order. Plain dictionaries ignore the order.

$ python ordereddict_ops.py
['test', 'deploy', 'write']
['deploy', 'test', 'write']
('write', 'todo')
('deploy', 'todo')
False True

An LRU cache

The reordering operations make OrderedDict a good basis for a least recently used cache.

from collections import OrderedDict


class LRUCache:

    def __init__(self, capacity):
        self.capacity = capacity
        self.data = OrderedDict()

    def get(self, key):
        if key not in self.data:
            return None
        self.data.move_to_end(key)
        return self.data[key]

    def put(self, key, value):
        self.data[key] = value
        self.data.move_to_end(key)
        if len(self.data) > self.capacity:
            evicted, _ = self.data.popitem(last=False)
            print(f'evicted {evicted}')


cache = LRUCache(2)
cache.put('a', 1)
cache.put('b', 2)
cache.get('a')
cache.put('c', 3)
cache.put('d', 4)

print(list(cache.data.items()))

The cache keeps the most recently used keys at the end.

self.data.move_to_end(key)

Each access moves the key to the end of the dictionary.

if len(self.data) > self.capacity:
    evicted, _ = self.data.popitem(last=False)

When the cache is full, the first item, the least recently used one, is removed. Because we read a before adding c, the key b is evicted first. For caching function results, the standard library offers the ready-made functools.lru_cache decorator.

$ python lru_cache.py
evicted b
evicted a
[('c', 3), ('d', 4)]

Layered settings with ChainMap

A ChainMap groups several mappings into a single view. Lookups search the mappings in order and return the first match.

from collections import ChainMap

defaults = {'theme': 'light', 'language': 'en', 'autosave': True}
user = {'theme': 'dark'}
cli = {'language': 'sk'}

settings = ChainMap(cli, user, defaults)

print(settings['theme'], settings['language'], settings['autosave'])
print(dict(settings))

settings['autosave'] = False
print(cli)
print(defaults)

The settings of a program come from command line arguments, user configuration, and defaults, in this order of priority.

settings = ChainMap(cli, user, defaults)

The theme comes from the user configuration, the language from the command line, and the autosave value from the defaults. No data is copied; the chain only refers to the original dictionaries, so later changes to them are visible through the chain.

settings['autosave'] = False

Writes and deletions affect only the first mapping. The defaults stay unchanged.

$ python chainmap_config.py
dark sk True
{'theme': 'dark', 'language': 'sk', 'autosave': True}
{'language': 'sk', 'autosave': False}
{'theme': 'light', 'language': 'en', 'autosave': True}

Nested scopes with ChainMap

A chain of mappings can model nested scopes, such as variables in a programming language.

from collections import ChainMap

module = ChainMap({'x': 1, 'y': 2})

function = module.new_child({'x': 10})
block = function.new_child({'z': 30})

print(block['x'], block['y'], block['z'])
print(block.maps)
print(block.parents['x'])

block['y'] = 99
print(block.maps[0], function['y'])

The new_child method returns a new chain with an additional mapping in front. The original chain is not changed.

print(block.maps)
print(block.parents['x'])

The maps attribute is the list of mappings, from the innermost to the outermost. The parents property returns the chain without the first mapping, so it looks up x in the enclosing scope.

block['y'] = 99

The assignment creates a new y in the innermost scope, which shadows the outer value. The outer scopes still see the original y.

$ python chainmap_scopes.py
10 2 30
[{'z': 30}, {'x': 10}, {'x': 1, 'y': 2}]
10
{'z': 30, 'y': 99} 2

UserDict

The UserDict class is a wrapper around a regular dictionary, which is stored in its data attribute. It is meant to be subclassed.

from collections import UserDict


class LowerDict(dict):

    def __setitem__(self, key, value):
        super().__setitem__(key.lower(), value)


class LowerUserDict(UserDict):

    def __setitem__(self, key, value):
        super().__setitem__(key.lower(), value)


headers = {'Content-Type': 'text/html', 'Server': 'Python'}

d1 = LowerDict(headers)
d2 = LowerUserDict(headers)

print(d1)
print(d2)

d1.update(Host='example.com')
d2.update(Host='example.com')

print(list(d1))
print(list(d2))

Both classes try to store all keys in lower case by overriding __setitem__.

class LowerDict(dict):

    def __setitem__(self, key, value):
        super().__setitem__(key.lower(), value)

The methods of the built-in dict, including the constructor and update, are implemented in C and do not call an overridden __setitem__. So the keys of LowerDict keep their case.

class LowerUserDict(UserDict):

The methods of UserDict are written in terms of __setitem__ and the other basic operations, so overriding one method changes the behaviour of the whole class. The UserList and UserString classes work in the same way for lists and strings.

$ python userdict.py
{'Content-Type': 'text/html', 'Server': 'Python'}
{'content-type': 'text/html', 'server': 'Python'}
['Content-Type', 'Server', 'Host']
['content-type', 'server', 'host']

Abstract base classes

The collections.abc module defines abstract base classes for containers. They are used for type checks and as mixins that provide methods for free.

from collections.abc import Sequence


class Squares(Sequence):

    def __init__(self, n):
        self.n = n

    def __len__(self):
        return self.n

    def __getitem__(self, i):
        if not 0 <= i < self.n:
            raise IndexError(i)
        return i * i


sq = Squares(6)

print(list(sq))
print(25 in sq, sq.index(16), sq.count(9))
print(list(reversed(sq)))
print(isinstance(sq, Sequence), isinstance([], Sequence), isinstance({}, Sequence))

The Squares class represents the first n square numbers without storing them.

def __len__(self):
    return self.n

def __getitem__(self, i):

The Sequence base class requires only __len__ and __getitem__. From them it derives __contains__, __iter__, __reversed__, index, and count.

print(isinstance(sq, Sequence), isinstance([], Sequence), isinstance({}, Sequence))

Checks against the abstract classes accept any type that fits the interface. Lists are sequences, while dictionaries are not; a dictionary would match Mapping.

$ python abc_sequence.py
[0, 1, 4, 9, 16, 25]
True 4 1
[25, 16, 9, 4, 1, 0]
True True False

Practical example

In the last example, we analyze a web server log with a Counter and a defaultdict.

from collections import Counter, defaultdict

log = """\
192.168.0.4 GET /index.html 200
192.168.0.7 GET /about.html 200
192.168.0.4 POST /login 302
192.168.0.9 GET /admin 403
192.168.0.4 GET /index.html 200
192.168.0.7 GET /missing 404
192.168.0.9 GET /admin 403
"""

statuses = Counter()
pages_by_ip = defaultdict(set)

for line in log.splitlines():
    ip, method, path, status = line.split()
    statuses[status] += 1
    pages_by_ip[ip].add(path)

print(statuses.most_common())

for ip, pages in sorted(pages_by_ip.items()):
    print(ip, sorted(pages))

errors = Counter({k: v for k, v in statuses.items() if k.startswith(('4', '5'))})
print(f'{errors.total()} error responses: {dict(errors)}')

Each line contains a client address, a method, a path, and a status code.

statuses = Counter()
pages_by_ip = defaultdict(set)

We count the status codes and collect the distinct pages requested by each client. A set factory removes duplicate pages.

errors = Counter({k: v for k, v in statuses.items() if k.startswith(('4', '5'))})

Finally, we build a counter of client and server errors and print their total.

$ python log_report.py
[('200', 3), ('403', 2), ('302', 1), ('404', 1)]
192.168.0.4 ['/index.html', '/login']
192.168.0.7 ['/about.html', '/missing']
192.168.0.9 ['/admin']
3 error responses: {'403': 2, '404': 1}

Source

collections – Container datatypes

collections.abc – Abstract Base Classes for Containers

In this article we have worked with the container types of the Python collections module.

Author

My name is Jan Bodnar, and I am a passionate programmer with extensive programming experience. I have been writing programming articles since 2007. To date, I have authored over 1,400 articles and 8 e-books. I possess more than ten years of experience in teaching programming.

List all Python tutorials.