Aggregating Logs

All MATRIXX components produce log output used for debugging any issues that may arise in the system. In a Kubernetes environment, you can view and search logs without having to log in to individual Kubernetes pods. To persist logs in a cloud native environment, use a log aggregator, which collects log output and sends it to persistent storage where it can be queried. This document describes the reference architecture for handling MATRIXX logs in a Kubernetes environment.

Default Logging to a File

By default, all work pods write logs to a file in a dedicated shared storage volume mounted at the directory specified with the global.storage.localStorageDir property (/home/data by default). Each pod creates a dedicated sub-directory containing all log files for that pod, which can be downloaded for debugging. You can override this default behavior so that all log output is redirected to STDOUT.

Note: By default, Engine Manager writes logs to shared_storage_root/mgr-engine-sXeY/mtx-engine-mgr.log, where shared_storage_root is the directory specified with the global.storage.localStorageDir property (/home/data by default).

For more information about MATRIXX Engine management methods, see the discussion about configuring MATRIXX using Helm in MATRIXX Configuration and the discussion about Topology Operator in MATRIXX Installation and Upgrade.

Logging to STDOUT

You can override the default file logging and direct it to STDOUT.

Note: Log aggregation is not supported when directing logs to STDOUT.

Override default file logging by omitting PVs and PVCs with configuration similar to the following:

global:
 # Enabled file logging to a persistent volume
 logging:
   output:
     persistentVolume: false
...
 
  storage:
    logging:
      # Should the Shared Logging Storage PVC and PV be created. These are only needed once per Namespace
      createPersistentVolumeClaim: false
      createPersistentVolume: false

PersistentVolume Parameters lists the PersistentVolume (PV) parameters used when all log output is directed to the STDOUT of the Kubernetes pod. These parameters are true by default.

Table 1. PersistentVolume Parameters
Parameter	Description
global.logging.output.persistentVolume	When set to `true`, pods write log files to the directory specified with the `global.storage.localStorageDir` property (/home/data by default). Otherwise logging information is written to STDOUT where possible. The default value is `true`.
global.storage.logging.createPersistentVolumeClaim	When set to `false`, disables the creation of the logging PersistentVolumeClaim (PVC).
global.storage.logging.createPersistentVolume	When set to `false`, disables the creation of the logging PV.

File Logging Options describes the parameters you use to configure logging.

Table 2. File Logging Options
Parameter	Description
global.logging.fluentbit.configuration.configmap	ConfigMap contents containing all Fluent Bit configuration files.
global.logging.fluentbit.configuration.sources[x].refName	The name of an entry in the `global.configurationSources` map. For more information about configuration sources, see the discussion about configuration sources for other components in MATRIXX Installation and Upgrade.
global.logging.fluentbit.enabled	Setting to `true` creates the Fluent Bit aggregation deployment. The default value is `false`.
global.logging.output.persistentVolume	When set to `true`, enables file logging to a PV. If enabled, an associated PVC called shared-logging-storage must already exist.
global.storage.logging.createPersistentVolumeClaim	(Optional) Creates a PVC. If enabled and if `global.logging.output.persistentVolume` is also enabled, an associated volume is created.
global.storage.logging.createPersistentVolume	(Optional) Creates an associated PV. If enabled, a default PV is created that maps to a directory in the local storage directory on the host machine.
global.storage.logging.storage	(Optional) Specifies a size for the PV, if created.
global.storage.logging.storageClassName	(Optional) Specifies a storage class name.
global.storage.logging.persistentVolume	(Optional) Adds a PV configuration to the PVC.

For information about logging in cloud native MATRIXX deployments, including example Fluent Bit ConfigMap configuration, see the discussion about log aggregation configuration in MATRIXX Configuration.