Logging in Python

Coding with Python

🕑 This lesson will take about 20 minutes

When things go wrong in our programs, it can be helpful to refer to a log of events. Logging can be used to keep a record of events, warnings and information about errors that have occurred that we can use to debug our code. Python has a built-in logging system called the logging module. We can use it by importing the logging module into our program.

The code below shows how to create and configure a logger.

Configuring the logger

When setting up the logger, we can configure the filename for the log file (eg. “eventlog.log”), as well as specify the logging level (eg. DEBUG, INFO, WARNING, ERROR, CRITICAL) and other information such as a timestamp for logs. In the example code above, the logging level has been set to INFO. If you opened the log file and it is empty, read the important information below.

The logger has different levels of logging each represented by a numeric value. We can specify the type of log we are working with. The table below shows the logging levels and numeric values for each.

Logging level

  • NOTSET 0

  • DEBUG 10

  • INFO 20

  • WARNING 30

  • ERROR 40

  • CRITICAL 50

When using the logger, we can specify which type of log we’re making eg:

  • logger.debug("This is a log") created a DEBUG log

  • logger.info("This is a log") created an INFO log

  • logger.warning("This is a log") created a WARNING log

  • logger.error("This is a log") created a ERROR log

  • logger.critical("This is a log") created a CRITICAL log

Your Python environment will be configured to a certain logging level which means that only logs of that level or higher will be added to the log. This configuration can be changed at different times, so that for example, only critical messages are logged or in other situations.

To find out thecurrent logging level configured in your Python environment, you can use this line of code after setting up the logger:

print(logger.level)

If we use that function in our code and run the code, a number (eg. 30) will be displayed in the terminal - this number represents the current logging level. Loggers will only write messages with levels that are greater than or equal to the set logging level. If the current logging level is set to 30, then that means we cannot write an INFO message (as it has a level of 20), but we could write a WARNING, ERROR, or CRITICAL message.

We can set the logging level for the Python environment when we create the logger using the following line of code (in this example, the logging level is set to DEBUG so the logger will be able to log messages at DEBUG level 10 or higher):

logger.level = 10

If you open the log file you will see different parts to a log message: the message type (eg. INFO), the name of the logger (eg. ROOT), and the message itself, for example:

INFO:root:My first log message

Adding more information to a log

We can add more data to the log message such as a timestamp (when the log message was added). The following modified code has a string variable (LOG_FORMAT) which specifies the log format as level name, timestamp, and then the message (on line 3). Also, we have added format = LOG_FORMAT to the basicConfig parameters (on line 6).

Using the above code, log messages will now contain information about the the type of log, when the log was created, and the message. Here is an example of a log using this configuration:

INFO 2024-02-15 15:16:49,749 - This is a log message

Here is another example with different types of logs made:

When you open a log file in Visual Studio Code, the different level log messages will be colour-coded. For example, WARNING level messages will be displayed in red.

Screenshot of a log file with different types of logs colour-coded in Visual Studio Code.

If we changed the logging level to ERROR (level 40) and ran the code again we would only see ERROR (level 40) and CRITICAL (level 50) messages. You can change the level depending on what you want to see in your log. For example, after you finish writing and testing your code and launch your project, you might not want to see debug and info messages anymore and may only want to see error and critical messages.

By default, the filemode is set to append which means every time we add a log message it is appended (added to the end) to the log file. We can set the mode to write ('w') if we want to overwrite the log. Changing the basicConfig line to the code below will do this. This will mean the log is overwritten every time a message is logged (old messages are overwritten with the most recent one).

logging.basicConfig(filename = "eventlog.log", level=logging.DEBUG, format = LOG_FORMAT, filemode='w')

Logging is helpful when there are several steps in an algorithm. You can use a log to log debug messages at each step of an algorithm and find out the last step that was carried out before a problem occurred (or find out which log messages are missing to determine what steps in an algorithm could not be carried out).

Logging can also be very helpful when building a web-based application that runs on a server. If the user performs an action that results in an error or an exception is raised, we can use logs to gain insight into what might have gone wrong. We also don’t need to worry about the user having to contact us as the log can be automatically made without the user’s input.


Next lesson: Working with JSON in Python