Skip logs from imported modules

Question

Trying to create a logger class for my package and I am close to banging my head against the wall. No matter what I do, the logger will also log messages from external modules. Here's my class:

import logging
import os
import sys
from pathlib import Path
from datetime import date


class Logger:

    # Variables
    log_path = ''
    log_format = '%(asctime)s %(levelname)-8s %(message)s'
    date_format = '%Y-%m-%d'
    datetime_format = '%Y-%m-%d %H:%M:%S'
    logger = None

    # CTOR
    def __init__(self, base_path):
        self.log_path = os.path.join(base_path, "logs/")
        Path(self.log_path).mkdir(parents=True, exist_ok=True)
        logfile = os.path.join(self.log_path, date.today().strftime(self.date_format) + ".log")
        logging.basicConfig(
            filename=logfile,
            level=logging.WARNING,
            filemode='a',
            format=self.log_format,
            datefmt=self.datetime_format)
        self.logger = logging.getLogger('bleservice')
        self.logger.setLevel(logging.INFO)

    # Methods
    def log_information(self, message):
        self.logger.info(message)
        print('logged info')

    def log_error(self, message):
        self.logger.error(message)
        print('logged error into')

    def log_exception(self, exception):
        self.logger.exception(message)
        print('logged exception into')

Basically, I want to log warnings and errors from external modules and everything from info from my package into the same file. Yet it also logs info messages from external modules, making my log file grow to several GB in size. What am I doing wrong here?

EDIT: To make it more clear, what I need to do is:

All logs from my own module starting at level INFO should log into the file
All logs from external modules starting at level WARNING should also log into the file.

Currently, all logs starting from INFO from my module AND external modules are logged into the file.

*Your* logger isn't logging everyone else's messages, but you told *everyone's* loggers to log to your log file. — user2357112, Apr 12 '20 at 22:45
You write "I want to log warnings and errors from external modules". Then you write "it also logs messages from external modules". So, do you want to log everything from external modules? — Olvin Roght, Apr 12 '20 at 22:46
How do you use your `Logger`? And how do you imagine "external" code to use it? — Klaus D., Apr 12 '20 at 22:47
@user2357112supportsMonica but shouldn't `level=logging.WARNING` in the basic config cause only warnings to log to the file from the root logger? — LeonidasFett, Apr 12 '20 at 22:47
Does this answer your question? [How to set different levels for different python log handlers](https://stackoverflow.com/questions/11111064/how-to-set-different-levels-for-different-python-log-handlers) — Olvin Roght, Apr 12 '20 at 23:06
I am not trying to set different handles. I am trying to set different levels for different modules. — LeonidasFett, Apr 12 '20 at 23:17

score 1 · Accepted Answer · answered Apr 13 '20 at 08:41

First let me explain why you see the behaviour that you are getting. When you call basicConfig with a level and a filename argument it will do two things for you. It will set the level of the root logger to the provided level and it will create a logging.FileHandler with a level of NOTSET that will be attached to the root logger.

Now when some modules logger creates a log and has propagate set to true (which is the default) it will send it's logs directly to it's ancestors handlers. It's important to understand that this bypasses the ancestor loggers and their level. So the modules see the root as their ancestor and send their logs directly to the attached handler which doesn't have a level set and will therefore allow all logs.

What you need to do is either set the level of the handler that basicConfig creates to WARNING and add another handler with level INFO to your bleservice logger, or you should set the level of the module loggers to WARNING.

So what you are saying is that putting `level=logging.WARNING` in the `basicConfig` is not enough? I need to actually create a file handler, set it to `WARNING` and tell the root logger to use it? Sorry but I am a total Python noob and it's very confusing coming from a C# background. — LeonidasFett, Apr 13 '20 at 09:24
You don't need to create a file handler. basicConfig does that for you. You need to set the level of the handler. Add this line `logging.root.handlers[0].setLevel(logging.WARNING)` after the call to basicConfig to do that. Also have a look at this flowchart which explains the flow of logs in python: https://docs.python.org/3.7/howto/logging.html#logging-flow — blues, Apr 13 '20 at 09:45

Skip logs from imported modules

1 Answers1