Python logging module provides a general logging system for use by third-party modules or applications. The logging module provides different log levels and records logs by different methods including file, HTTP GET/POST, SMTP, and Socket. You can customize a log recording method as needed. The logging module is the same as Log4j except that they have different implementation details. The logging module provides the logger, handler, filter, and formatter features.
The formatter specifies the log record output format. The formatter construction mode are defined by two parameters: message format string and message date string. The parameters are optional.
Python sample log:
LOG_FILE = 'tst.log'
handler = logging.handlers.RotatingFileHandler(LOG_FILE, maxBytes = 1024*1024, backupCount = 5) # Instantiate the handler
fmt = '%(asctime)s - %(filename)s:%(lineno)s - %(name)s - %(message)s'
formatter = logging.Formatter(fmt) # Instantiate the formatter
handler.setFormatter(formatter) # Add the formatter to the handler
logger = logging.getLogger('tst') # Obtain the logger named tst
logger.addHandler(handler) # Add the handler to the logger
logger.info('first info message')
logger.debug('first debug message')
Output sample log:
2015-03-04 23:21:59,682 - log_test.py:16 - tst - first info message
2015-03-04 23:21:59,682 - log_test.py:17 - tst - first debug message
The formatter is configured in the
%(key)s format, that is, replacing the dictionary keywords. The following keywords are provided.
|%(name)s||Name of the logger (logging channel).|
|%(levelno)s||Numeric logging level for the message (DEBUG, INFO, WARNING, ERROR, CRITICAL).|
|%(levelname)s||Text logging level for the message (‘DEBUG’, ‘INFO’, ‘WARNING’, ‘ERROR’, ‘CRITICAL’).|
|%(pathname)s||Full path of the source file where the logging call was issued (if available).|
|%(filename)s||File name portion of the path name.|
|%(module)s||Module (name portion of the file name).|
|%(funcName)s||Name of function containing the logging call.|
|%(lineno)d||Source line number where the logging call was issued (if available).|
|%(created)f||Time when the LogRecord was created, in UNIX standard time format and seconds calculated from 1970-1-1 00:00:00 UTC.|
|%(relativeCreated)d||Time in milliseconds when the LogRecord was created, relative to the time the logging module was loaded.|
|%(asctime)s||Human-readable time when the LogRecord was created. By default this is of the form “2003-07-08 16:49:45,896” (the numbers after the comma are millisecond portion of the time).|
|%(msecs)d||Millisecond portion of the time when the LogRecord was created.|
|%(thread)d||Thread ID (if available).|
|%(threadName)s||Thread name (if available).|
|%(process)d||Process ID (if available).|
|%(message)s||The logged message.|
For details about the standard process, refer to Quick start.
The automatically generated regular expression is based on the sample log and does not cover every log type. Therefore, you need to tune the regular expression after it is generated.
A regular expression and other information for a common Python log is shown below.
2016-02-19 11:03:13,410 - test.py:19 - tst - first debug message
%(asctime)s - %(filename)s:%(lineno)s - %(levelno)s %(levelname)s %(pathname)s %(module)s %(funcName)s %(created)f %(thread)d %(threadName)s %(process)d %(name)s - %(message)s
2016-02-19 11:06:52,514 - test.py:19 - 10 DEBUG test.py test <module> 1455851212.514271 139865996687072 MainThread 20193 tst - first debug message