Everyone Gets Python Logging Wrong ( And How to Fix It )

Python's logging module is one of those things everyone uses and almost nobody uses well. I've seen production codebases with print() scattered everywhere, logging.basicConfig() called five times in the same module, and log files that are 2GB of useless INFO noise. It doesn't have to be like this.

I spent years getting logging wrong. Here's what I finally figured out.

Stop Using print() for Logging

Every time I see print(f"Processing {item_id}") in production code, I die a little inside. Print goes to stdout. It has no level. No timestamp. No module name. No way to turn it off selectively. It's not logging. It's debugging left behind.

# Bad - you know who you are
print(f"Processing {item_id}")
print(f"Error: {err}")
print("======")

# Good - this is logging
import logging
logger = logging.getLogger(__name__)

logger.info("Processing item", extra={"item_id": item_id})
logger.error("Failed to process item", extra={"item_id": item_id}, exc_info=True)

The difference is structure. With logging you get levels, timestamps, module context, and the ability to filter. With print you get a string in a sea of strings.

Use __name__ for Your Logger

This is the single most common mistake I see:

# Bad - root logger, no namespace
logging.warning("Something went wrong")

# Also bad - hardcoded name
logger = logging.getLogger("myapp")

# Good - per-module logger
logger = logging.getLogger(__name__)

When you use __name__, your logger inherits its name from the module ( like app.api.users ). This means you can configure log levels per module, filter by namespace, and actually tell where a log message came from. If you use the root logger, every message says root: and you're back to guessing.

Configure Once, Not Everywhere

I've seen this more times than I can count:

# config.py - THE ONLY PLACE logging should be configured
import logging
import logging.config

def setup_logging():
    logging.config.dictConfig({
        "version": 1,
        "disable_existing_loggers": False,
        "formatters": {
            "default": {
                "format": "%(asctime)s [%(levelname)s] %(name)s - %(message)s",
                "datefmt": "%Y-%m-%d %H:%M:%S",
            },
        },
        "handlers": {
            "console": {
                "class": "logging.StreamHandler",
                "formatter": "default",
                "level": "INFO",
            },
            "file": {
                "class": "logging.handlers.RotatingFileHandler",
                "filename": "/var/log/app.log",
                "maxBytes": 10485760,  # 10MB
                "backupCount": 5,
                "formatter": "default",
                "level": "DEBUG",
            },
        },
        "root": {
            "handlers": ["console", "file"],
            "level": "DEBUG",
        },
    })

One function. One place. Call it once at app startup. Every module just does logger = logging.getLogger(__name__) and moves on. No basicConfig() scattered across 15 files. No custom handlers per module. One config to rule them all.

Use the Right Level

Most codebases have two levels: INFO for everything and ERROR for when things blow up. That's not logging. That's screaming into the void.

Here's my rule:

DEBUG: stuff I need when debugging a specific issue. Off in production.

INFO: stuff I want to see in production. Startup, shutdown, key business events.

WARNING: something's wrong but the app can continue. Degraded performance, retries, fallback paths.

ERROR: something failed. The operation didn't complete. Needs attention.

CRITICAL: the app is on fire. Database connection gone, can't serve requests, time to page someone.

If your INFO logs are more than a few lines per minute, you're logging too much. INFO is for milestones, not for every HTTP request parameter.

Structured Logging Is Non-Negotiable

Plain text logs are fine for reading in a terminal. They're useless for anything else. You need structured logs ( JSON or key-value pairs ) the moment you have more than one service or need to grep through more than a day's worth of data.

import logging
import json

class JSONFormatter(logging.Formatter):
    def format(self, record):
        log_entry = {
            "timestamp": self.formatTime(record, self.datefmt),
            "level": record.levelname,
            "logger": record.name,
            "message": record.getMessage(),
        }
        if record.exc_info and record.exc_info[0]:
            log_entry["exception"] = self.formatException(record.exc_info)
        if hasattr(record, "extra_data"):
            log_entry.update(record.extra_data)
        return json.dumps(log_entry)

With this, your logs become searchable. You can filter by level, logger, or any custom field. You can pipe them into jq or feed them into Loki or ELK. Try doing that with free-form text.

Use extra for Context ( Not String Formatting )

Stop doing this:

# Bad - all context locked in a string
logger.info(f"User {user_id} purchased item {item_id} for ${price}")

# Good - context is structured data
logger.info(
    "Purchase completed",
    extra={
        "user_id": user_id,
        "item_id": item_id,
        "price": price,
        "currency": "USD",
    },
)

The f-string version is fine for reading in a terminal. It's garbage for everything else. Can you filter all purchases for a specific user? Can you graph average price over time? Not without regex. With extra ( and a JSON formatter ) ( like this ) you get that for free.

Rotate Your Log Files

If you're writing logs to a file ( and you should be, not just stdout ), use RotatingFileHandler. Not FileHandler. Not logrotate on the side. Python handles it natively.

from logging.handlers import RotatingFileHandler

handler = RotatingFileHandler(
    "/var/log/app.log",
    maxBytes=10 * 1024 * 1024,  # 10MB per file
    backupCount=5,                # keep 5 old files
)
handler.setFormatter(logging.Formatter(
    "%(asctime)s [%(levelname)s] %(name)s - %(message)s"
))
logger.addHandler(handler)

This gives you at most ~60MB of logs ( 6 files x 10MB ). Set the size and count to whatever makes sense for your app. Just don't let logs grow without bounds. I've seen a single log file fill a 50GB disk. Not fun at 3am.

Never Log in Hot Loops

If your code processes 10,000 items a second, and you log INFO for each one, you're generating 10,000 log lines per second. That's not logging. That's a DoS attack on your own disk.

# Bad - 10k log lines per second
for item in items:
    logger.info(f"Processing {item.id}")
    process(item)

# Good - log the summary
logger.info(f"Processing {len(items)} items")
for item in items:
    logger.debug(f"Processing {item.id}")  # DEBUG, off in production
    process(item)
logger.info(f"Processed {len(items)} items in {elapsed:.2f}s")

Log the milestones. Log the errors. Log the summary. Don't log every iteration.

Conclusion

Python logging isn't complicated. It's just that most people learn basicConfig() and stop there. Use __name__ loggers. Configure once. Use the right levels. Structure your output. Rotate your files. Skip the hot loops.

That's it. No frameworks needed. No third-party libraries. Just the stdlib, configured properly.

Stop making it harder than it is. :)