I keep seeing people write try/finally blocks for file handles, database connections, and locks. Every. Single. Time. Python has context managers for this, and they are not complicated ( honestly, they are the cleanest abstraction in the language ).
If you have ever used with open('file.txt') as f: you have already used a context manager. You just did not write one yourself.
The Basics
A context manager is any object that implements __enter__ and __exit__. That is it. When you write with x as y: Python calls x.__enter__() and assigns the result to y. When the block ends, x.__exit__() runs no matter what ( even if an exception was thrown ).
Here is the simplest one:
class MyContext:
def __enter__(self):
print('Entering')
return self
def __exit__(self, exc_type, exc_val, exc_tb):
print('Exiting')
return False
with MyContext() as ctx:
print('Inside')
# Entering
# Inside
# ExitingReturning False from __exit__ means exceptions propagate normally. Return True and you just swallowed the exception. Do not do that unless you really mean it.
Use contextlib Instead of Writing a Class
Most of the time you do not need a class. contextlib.contextmanager turns a generator into a context manager. This is what I use 90% of the time.
Here is a timer:
import time
from contextlib import contextmanager
@contextmanager
def timer(label):
start = time.perf_counter()
yield
elapsed = time.perf_counter() - start
print(f'{label}: {elapsed:.3f}s')
with timer('database query'):
# do expensive stuff here
time.sleep(0.5)
# database query: 0.500sClean, no class, no boilerplate. The yield is where your with block runs. Everything before yield is __enter__, everything after is __exit__.
Handling Errors in __exit__
The __exit__ method gets three arguments: exc_type, exc_val, exc_tb. If no exception happened, all three are None.
A database connection handler that rolls back on error:
@contextmanager
def db_transaction(conn):
cursor = conn.cursor()
try:
yield cursor
conn.commit()
except Exception:
conn.rollback()
raise # re-raise, do not swallow
# Usage
with db_transaction(conn) as cur:
cur.execute('INSERT INTO users ...')
cur.execute('INSERT INTO logs ...')
# committed, or rolled back if anything blew upThe raise at the end is important. contextlib.contextmanager catches exceptions from the with block and re-raises them after your except runs. If you do not re-raise, the exception disappears silently. That is a bug factory.
contextlib.suppress: The Silent Exception Killer
You know this pattern:
# Ugly
def maybe_delete(path):
try:
os.remove(path)
except FileNotFoundError:
passReplace with:
from contextlib import suppress
with suppress(FileNotFoundError):
os.remove(path)Same result, less noise. suppress also accepts multiple exception types: suppress(FileNotFoundError, PermissionError).
contextlib.closing for Things Without __exit__
Some objects have a close() method but no __exit__. Like urllib.request.urlopen() or a selenium driver. closing() wraps them:
from contextlib import closing
from urllib.request import urlopen
with closing(urlopen('https://example.com')) as response:
html = response.read()
# response.close() called automaticallyWithout closing() you would need a try/finally just to call close(). Now you do not.
ExitStack: Managing Multiple Contexts
When you need to open 3 files, a database connection, and a lock, you get nested with blocks that make your code look like a staircase. ExitStack fixes that:
from contextlib import ExitStack
with ExitStack() as stack:
f1 = stack.enter_context(open('input.txt'))
f2 = stack.enter_context(open('output.txt', 'w'))
f3 = stack.enter_context(open('log.txt', 'a'))
conn = stack.enter_context(db_connection())
lock = stack.enter_context(redis_lock('my-key'))
# all the work
for line in f1:
f2.write(line.upper())
# everything closed in reverse order, guaranteedIf the second open() fails, ExitStack still closes f1 first. That is the whole point ( you never leak resources ).
redirect_stdout and redirect_stderr
Quick way to silence noisy libraries or capture output:
from contextlib import redirect_stdout
import io
# Silence everything in this block
with redirect_stdout(io.StringIO()):
noisy_library_function()
another_noisy_call()
# Capture output
f = io.StringIO()
with redirect_stdout(f):
print('this goes to the buffer')
captured = f.getvalue()Yes, you could monkey-patch sys.stdout manually. This is just cleaner and restores it automatically when the block ends.
Async Context Managers
Same idea, but with __aenter__ and __aexit__. The asyncwith library has asynccontextmanager too:
from contextlib import asynccontextmanager
@asynccontextmanager
async def db_pool(dsn):
pool = await create_pool(dsn)
try:
yield pool
finally:
await pool.close()
async with db_pool('postgres://localhost/mydb') as pool:
async with pool.acquire() as conn:
await conn.execute('SELECT 1')Nothing revolutionary here, just the async version of what you already know.
When to Write Your Own
Write a context manager when:
- You are opening resources ( files, connections, locks ) that need cleanup
- You find yourself writing the same try/finally block more than twice
- You need to temporarily change state ( redirect output, set environment variables, change working directory )
- You need transaction-like behavior ( commit on success, rollback on error )
Do not write one for everything. If it is a one-off try/finally that you do not reuse, keep it simple. A context manager is for when you need the pattern more than once, or when the setup/cleanup is complex enough to warrant it.
Thanks for reading.