Basic and Advanced Logging Tutorial in Python

Basic Logging Tutorial

Logging is a method for tracking events that occur while software runs. Developers add logging calls to their code, providing descriptive messages that may include variable data and an importance level (also called severity).

When to Use Logging

Python’s logging module offers convenience functions such as debug() , info() , warning() , error() and critical() . The table below shows the best tool for common tasks:

Task

Best Tool

Show console output for scripts

print()

Report normal runtime events (e.g., status monitoring)

logging.info()

or

logging.debug()

for very detailed output

Issue a warning about a specific runtime event

warnings.warn()

(if the client can avoid the issue) or

logging.warning()

(if the client cannot handle it)

Report an error that should raise an exception

Raise an exception

Report an error without raising an exception (e.g., long‑running server)

logging.error()

,

logging.exception()

or

logging.critical()

Log functions are named after the severity of the events they track. The standard levels (in increasing severity) are:

Level

When to Use

DEBUG

Detailed information useful only for diagnosing problems.

INFO

Confirm that things are working as expected.

WARNING

Indicates an unexpected situation or a future problem (e.g., low disk space) while the software still works.

ERROR

The software cannot perform some function due to a more serious problem.

CRITICAL

Severe error indicating that the program itself may not continue.

The default level is WARNING , meaning only events at this level or higher are tracked unless the logging package is configured otherwise.

A Simple Example

Very simple example:

<code>import logging
logging.warning('Watch out!')  # prints to console
logging.info('I told you so')  # nothing is printed</code>

Running the script prints:

<code>WARNING:root:Watch out!</code>

Because the default level is WARNING , the INFO message does not appear.

Logging to a File

Logging to a file is a common scenario. Run the following in a fresh interpreter:

<code>import logging
logging.basicConfig(filename='example.log', level=logging.DEBUG)
logging.debug('This message should go to the log file')
logging.info('So should this')
logging.warning('And this, too')</code>

Opening example.log shows:

<code>DEBUG:root:This message should go to the log file
INFO:root:So should this
WARNING:root:And this, too</code>

Since the threshold is set to DEBUG , all messages are recorded.

Multiple‑Module Logging

When a program contains several modules, you can organise logging as follows:

<code># myapp.py
import logging, mylib

def main():
    logging.basicConfig(filename='myapp.log', level=logging.INFO)
    logging.info('Started')
    mylib.do_something()
    logging.info('Finished')

if __name__ == '__main__':
    main()</code>

<code># mylib.py
import logging

def do_something():
    logging.info('Doing something')</code>

Running myapp.py produces myapp.log containing:

<code>INFO:root:Started
INFO:root:Doing something
INFO:root:Finished</code>

Logging Variable Data

Use format strings to include variable data:

<code>import logging
logging.warning('%s before you %s', 'Look', 'leap!')</code>

Output:

<code>WARNING:root:Look before you leap!</code>

Older % formatting is shown for backward compatibility; newer str.format() and string.Template styles are also supported.

Changing Message Format

Specify a format string via basicConfig :

<code>import logging
logging.basicConfig(format='%(levelname)s:%(message)s', level=logging.DEBUG)
logging.debug('This message should appear on the console')
logging.info('So should this')
logging.warning('And this, too')</code>

Result:

<code>DEBUG:This message should appear on the console
INFO:So should this
WARNING:And this, too</code>

Displaying Date/Time

Include %(asctime)s in the format string:

<code>import logging
logging.basicConfig(format='%(asctime)s %(message)s')
logging.warning('is when this event was logged.')</code>

Typical output:

<code>2010-12-12 11:41:42,612 is when this event was logged.</code>

The datefmt argument can customise the timestamp format.

Next Steps

The basic tutorial ends here; the logging package offers many more features. Continue reading the advanced sections for deeper insight.

Advanced Logging Tutorial

The logging library follows a modular approach with four component classes: loggers, handlers, filters, and formatters.

Loggers expose the interface used by application code.

Handlers send logged records to appropriate destinations.

Filters provide fine‑grained control over which records are output.

Formatters define the final layout of log records.

Log records flow between these components via LogRecord instances.

Loggers

Calling methods on a Logger instance (e.g., debug() , info() , warning() , error() , critical() ) creates a log record. The logger’s name (e.g., foo.bar ) reflects its position in a hierarchical namespace.

The root logger is the ancestor of all loggers. If no handlers are configured, the root logger’s methods automatically configure a console handler that writes to sys.stderr with a default format.

Handlers

Handlers determine where a log record is sent (console, file, email, socket, etc.). The standard library provides StreamHandler and FileHandler among many others.

Handlers have their own level, which acts as an additional filter. The emit() method performs the actual output.

Formatters

A Formatter converts a LogRecord into a human‑readable string. It can be configured with a message format, a date format, and a style ( % , { , or $ ).

Example format string:

<code>'%(asctime)s - %(levelname)s - %(message)s'</code>

Configuring Logging

There are three ways to configure logging:

Explicitly create loggers, handlers, and formatters in code.

Use a configuration file with logging.config.fileConfig() .

Pass a dictionary to logging.config.dictConfig() (recommended for new applications).

Example of programmatic configuration:

<code>import logging
logger = logging.getLogger('simple_example')
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
ch.setFormatter(formatter)
logger.addHandler(ch)
logger.debug('debug message')
logger.info('info message')
logger.warn('warn message')
logger.error('error message')
logger.critical('critical message')</code>

Logging in Libraries

Library code should obtain a logger with logging.getLogger(__name__) and avoid configuring handlers directly. Adding a NullHandler to the top‑level logger prevents unwanted output when the application using the library has not configured logging.

Log Levels

Numeric values for the standard levels are:

Level

Value

CRITICAL

50

ERROR

40

WARNING

30

INFO

20

DEBUG

10

NOTSET

0

Loggers compare their effective level with the level of a logging call; if the logger’s level is higher, the call is ignored.

Custom Levels

Defining custom levels is possible but should be done with caution, especially in libraries, because different libraries may assign different meanings to the same numeric value.

Common Handlers

Beyond the core Handler , the logging.handlers module provides many specialised handlers such as RotatingFileHandler , TimedRotatingFileHandler , SMTPHandler , SysLogHandler , HTTPHandler , and QueueHandler . The NullHandler does nothing and is useful for libraries.

Exceptions During Logging

The logging package swallows most exceptions that occur while handling a log record, except for SystemExit and KeyboardInterrupt . If logging.raiseExceptions is true (default in development), tracebacks are printed to sys.stderr ; in production it should be set to false.

Logging Arbitrary Objects

Any object can be passed as the log message; its __str__() method is called when the logging system needs a string representation.

For the full source and additional details, see the original documentation.