Skip to content

Expert · Advanced Python · topic 42 of 50 · 30 min

Context managers and the with statement

Explanation

A context manager sets something up when a with block starts and always cleans it up when the block ends, even if an error happens inside. Files, locks and database connections use it so nothing is left open. You can write your own with __enter__ and __exit__, or more simply with contextlib.contextmanager and one yield.

The problem: forgetting to clean up

Many things must be closed or undone after use: a file, a network connection, a lock that stops two threads from clashing, a setting you changed for a moment. If an error happens in the middle, the code that closes it may never run. Over time, open files and connections pile up and the program slows down or crashes.

The usual fix is try/finally, because a finally block always runs. A context manager packs that try/finally into a reusable object, and the with statement uses it.

Think of it like this: Think of borrowing a library book with a strict librarian. When you enter the reading room (with), the book is handed to you (__enter__). When you leave the room for any reason, finished or in a hurry, the librarian takes the book back at the door (__exit__). You cannot walk out with it by mistake.

The with statement

output
with open("marks.csv") as f:      # the pattern you know from files
    data = f.read()
# ...is roughly the same as:
f = open("marks.csv")
try:
    data = f.read()
finally:
    f.close()                       # runs even if read() fails
with calls the set-up step, runs the indented block, then always runs the clean-up step.

__enter__ and __exit__

Any object with two dunder methods is a context manager. __enter__ runs at the start; whatever it returns is given to the name after "as". __exit__ runs at the end and receives three values about any exception (its type, the exception itself and the traceback), or three Nones if all went well.

python
class FakeConnection:
    def __init__(self, log):
        self.log = log

    def __enter__(self):
        self.log.append("open")
        return self                      # this becomes "conn" below

    def __exit__(self, exc_type, exc, tb):
        self.log.append("close")
        return False                     # False: do not hide errors

    def send(self, text):
        self.log.append("send " + text)

log = []
try:
    with FakeConnection(log) as conn:
        conn.send("hello")
        raise ValueError("network dropped")
except ValueError:
    pass
print(log)   # ['open', 'send hello', 'close']
Even though an error happened, "close" was still recorded.
Careful: If __exit__ returns True, Python treats the exception as handled and hides it. That is almost never what you want by accident. Return False (or nothing, which is None) unless you really mean to swallow the error.

The easy way: contextlib.contextmanager

Writing a class is a lot of typing. The contextmanager decorator from the contextlib module turns a generator function into a context manager. Code before yield is the set-up, the yielded value goes to "as", and code after yield is the clean-up. Put the yield inside try/finally so the clean-up runs even on errors.

python
from contextlib import contextmanager
import time

@contextmanager
def timer(label):
    start = time.perf_counter()
    try:
        yield
    finally:
        seconds = time.perf_counter() - start
        print(label, "took", round(seconds * 1000, 2), "ms")

with timer("adding numbers"):
    total = sum(range(100_000))
A reusable timer: wrap any block to see how long it took.

Changing a setting for a moment

python
from contextlib import contextmanager

settings = {"language": "en"}

@contextmanager
def temporary(key, value):
    old = settings[key]
    settings[key] = value
    try:
        yield
    finally:
        settings[key] = old        # always put it back

with temporary("language", "hi"):
    print(settings["language"])    # hi
print(settings["language"])        # en
Tests use this pattern a lot: change something, run a check, restore it.

Ignoring one expected error with contextlib.suppress

contextlib.suppress(SomeError) is a ready-made context manager that silently ignores that one kind of exception inside the block. It is a short, clear replacement for try/except/pass.

python
from contextlib import suppress

stock = {"rice": 10}
with suppress(KeyError):
    del stock["sugar"]      # no sugar key: KeyError is ignored
print(stock)
Only KeyError is ignored. Any other error still stops the program.
Tip: You can open several context managers in one with statement: with timer("job"), temporary("language", "hi"): ... They are cleaned up in reverse order.
NeedTool
Open and close a file safelywith open(...) as f
Your own set-up and clean-up, quickly@contextlib.contextmanager with one yield
Your own context manager with state and methodsA class with __enter__ and __exit__
Ignore one expected exceptionwith contextlib.suppress(ErrorType)

Watch the lesson

Context managers and the with statement · 1/6

Context managers

In this lesson you will learn the with statement and how to write your own context managers.

Prefer a video by a teacher? Find free videos on “python context managers with statement contextlib tutorial” on YouTube ↗

In real life: A UPI payment that always releases its lock

When a payment app debits a bank account, it first locks that account so two payments cannot spend the same money at the same moment. If the code crashes after locking but before unlocking, the account stays stuck. Wrapping the work in a context manager guarantees the lock is released, whatever happens.

python
from contextlib import contextmanager

locked = set()

@contextmanager
def account_lock(account):
    locked.add(account)
    try:
        yield
    finally:
        locked.discard(account)

try:
    with account_lock("ACC-001"):
        print("locked:", locked)
        raise RuntimeError("bank server did not reply")
except RuntimeError as err:
    print("payment failed:", err)
print("locked after:", locked)
The account is unlocked even though the payment failed.

Practice (0/3 done)

Task 1 · Code · Python

Write a class Recorder that is a context manager. Recorder(log) receives a list. __enter__ appends "open" to the list and __exit__ appends "close". __exit__ must not hide errors (return False).

Task 2 · Code · Python

Using contextlib.contextmanager, write temp_setting(settings, key, value). Inside the with block, settings[key] must equal value. After the block, the old value must be back, even if an error happened inside.

Task 3 · Code · Python

Write safe_int(text) that returns int(text), or None when the text is not a whole number. Use contextlib.suppress(ValueError) instead of try/except.

Quick quiz

Question 1

1.When does the clean-up part of a context manager run?

Question 2

2.In "with FakeConnection() as conn:", what becomes conn?

Question 3

3.With @contextmanager, where should the clean-up code go?

Question 4

4.What happens if __exit__ returns True?

Question 5

5.Which is the shortest clear way to ignore a KeyError while deleting a dict key?

Read more (free)

Pass the quiz (60% or more) or mark this topic as done when you are ready.

Your progress is saved on this device only.

Next: Type hints →

← Previous: Decorators