Cloud Native GL Processor

You can send GL information to a PostgreSQL database in cloud native MATRIXX deployments when you do not want to use a MongoDB database for the Event Repository.

Important: You can send GL information to a PostgreSQL database when using the cn-gl-processor image only. If you configure MATRIXX Engine to add general ledger (GL) information to Event Detail Records (EDRs), you can process that GL information using a cn-gl-processor REST call.

The information in this section assumes you are already familiar with GL records in MATRIXX.

To stream to a PostgreSQL database, you must:
  • Have a provisioned PostgreSQL database. This can be a self-managed pod within the cluster, or it can be hosted by a provider such a Amazon RDS on AWS, or Cloud SQL on GCE.
  • Export data from MEF files or send it to a Kafka broker for consumption by the PostgreSQL database. When streaming events from the Event Repository, you must configure the environment for Kafka. See the discussion about enabling Kafka for event streaming.

MATRIXX also supports partitioning large datasets in PostgreSQL to support efficient processing of GL data. To align the partitions with financial reporting periods, the default retention period is defined in calendar months, and can be changed if required.

The tables that require partitioning in GL Processor are:
  • GlMtxEvent (event data)
  • DailySummary (GL Processor table - process from event table)

The tables are partitioned into ranges defined by a key column (date range), with no overlap between the ranges of values assigned to different partitions. Each range's bounds are inclusive at the lower end and exclusive at the upper end. For example, if one partition's range is from 1 to 10, and the next one's range is from 10 to 20, then the value 10 belongs to the second partition not the first.

Configure the partitioning properties in application.yaml before installing GL Processor. For example:
glPartitioning:
  cronExpression: 0 0 6 * * ?
  retentionPeriod: 6
  • cronExpression — The cron expression for the automated partitioning process scheduler to be run. The default expression triggers the scheduler to run at 6 am every day. This property is configurable.
  • retentionPeriod — Represents the time, in months, that the data is retained. Partitions before this period are purged automatically.

You can configure partitioning properties in your Helm values file as environment variables. For a sample cn-gl-processor Helm values file, see the discussion about the cloud native GL processor Helm values file example.

After deploying GL processor, it automatically runs the setup partition, which executes the initial setup script that declares the partition strategy for the schema involved (GlMtxEvent, DailySummary). The partition for the current month is also created in the setup process. For the GL Processor to process and store data, the partitions must be created up front to anticipate incoming requests. The subsequent partition is handled automatically.

It also detects any gaps in the stream of events to the GL Processor and generates an alert if necessary. This might be performed by analyzing statistics sent to the monitoring system.

A CronJob triggers the creation of partitions using the MATRIXX REST APIs or PostgreSQL partitioning. For more information about these REST APIs, see the discussion about MATRIXX REST APIs for PostgreSQL partitioning.

Figure 1 shows the GL processing and posting process in a cloud native MATRIXX deployment.
Figure 1. GL Processing with PostgreSQL
Processing starts with events that are saved to MEF and then processed and posted by MATRIXX GL.