141 posts tagged with "python"

🪵 Structured Logging in Python: The Key to Observability​

While the logging module is powerful, its default output is unstructured plain text, making it difficult for tools like ELK Stack or Splunk to search, aggregate, and analyze messages efficiently. Structured logging solves this by outputting logs in a standard format (usually JSON) where every event detail is an easily parsable key-value pair.

This article details how to integrate structured logging into a Python application using the popular library python-json-logger and how to leverage it for operational insight.

While printing logs to the console (stdout or stderr) is useful during development, directing logs to a file is mandatory for production environments. File logging provides a persistent record of application events, crucial for debugging, auditing, and long-term monitoring.

The Python logging module manages file output through File Handlers. This guide covers the simplest setup, advanced rotation, and common configuration patterns.

The logging.basicConfig() function is the easiest way to perform basic setup for the Python logging module. It sets the configuration for the root logger, which is the parent of all other loggers in your application.

This function is ideal for simple scripts, development environments, and applications where you only need a single, global logging configuration.

While Python's built-in logging module uses integer constants (logging.DEBUG, logging.INFO, etc.) for log levels, modern Python practice encourages using the enum.Enum class for defining symbolic names, especially for configurations and custom values.

Using an Enum to wrap or reference standard logging levels significantly enhances code readability, prevents hard-to-debug typos, and aids type checking when passing levels as function arguments.

Logging is a critical tool for debugging, monitoring, and auditing applications. Unlike simple print() statements, Python's built-in logging module allows you to categorize messages by severity. This categorization is done using Logging Levels, which are numerical constants that determine which messages are recorded and which are ignored.

Understanding and correctly configuring logging levels is essential for running a robust production system.

While Pyright, particularly its integration via Pylance in VS Code, offers superior speed and a responsive developer experience, it is not without its drawbacks. For certain projects, teams, or development philosophies, Pyright's characteristics can lead to friction, complexity, or outright incompatibility.

Here is a critical assessment of the reasons why an experienced developer might choose not to rely on Pyright, or use it only in a secondary role.

MyPy and Pyright are the two most powerful static type checkers currently used in the Python ecosystem. While both aim to enforce type hints and catch errors before runtime, they differ significantly in their implementation, philosophy, speed, and feature set.

Choosing between them—or deciding how to use them together—depends heavily on your priorities: speed, strictness, or integration with development environments.

Ignoring errors with MyPy should be a precise, surgical process, not a broad suppression. The most effective way to handle known, persistent issues (like dynamically assigned attributes or known type inconsistencies in third-party libraries) is by using the # type: ignore comment combined with the exact MyPy Error Code.

An experienced developer always aims for the narrowest possible scope of suppression to maintain maximum type-checking coverage across the rest of the codebase.

When integrating MyPy into a large or legacy Python codebase, or when dealing with highly dynamic code, you will inevitably encounter situations where MyPy raises valid errors that you cannot (or should not) fix immediately. In these cases, selectively ignoring errors becomes a vital skill.

A super experienced developer never ignores errors blindly; they use these mechanisms strategically to maintain the highest possible type-checking quality in the rest of the codebase.

Here is a deep dive into the various methods MyPy provides for ignoring errors, from the narrowest to the broadest scope.

The AttributeError is one of the most common exceptions in Python, indicating an attempt to access or set a class attribute that simply doesn't exist. When it occurs within the context of a @dataclass, it often points to a misunderstanding of how the decorator automatically generates methods like __init__ and __setattr__.

Here is a breakdown of the most frequent AttributeError scenarios involving dataclasses and the high-level solutions to resolve them.