Home
MATRIXX 5G Event Streaming
Configuring the Applications
The 5G event streaming applications are configured in the Helm values file, the nf.yaml file, and an application.yaml files for each application.
Configuring ASN.1 sFTP Sink
Several characteristics of the ASN.1 sFTP Sink application are configurable.
Upload Settings
5g_integration/c_5g_events_sink_upload.html#concept_nvm_jkx_4xb__table_lvp_g4x_kpb describes properties for configuring several characteristics of upload.

Welcome
MATRIXX Release Notes
MATRIXX Architecture
MATRIXX Installation and Upgrade
MATRIXX Configuration
MATRIXX Security
MATRIXX Integration
MATRIXX Diameter Integration
MATRIXX Call Control Framework Integration
MATRIXX TM Forum Integration
MATRIXX 5G Integration
MATRIXX 5G Event Streaming
- About MATRIXX 5G Event Streaming
  MATRIXX 5G Event Streaming describes how to aggregate and stream charging data records (CDRs) from the SBA Gateway to an sFTP server using a Kafka broker.
- 5G Event Streaming Overview
  SBA Gateway can stream HTTP requests and MATRIXX Engine HTTP responses to a Kafka broker. The 5G event streaming applications convert the received JSON objects to ASN.1 binary format for export to Kafka applications and can optionally aggregate and route based on content. The GTP Prime Adapter microservice is provided for sending records from Kafka to a GPRS Tunneling Protocol (GTP) server.
- Converting CDRs to ASN.1 Format
  Charging data records (CDRs) are processed by the CDR ASN.1 Streamer Kafka client application to convert them from JSON into ASN.1 objects. After the ASN.1 objects are serialized as a binary message using Basic Encoding Rules (BER), they are placed in another Kafka topic for further processing. The ASN.1 sFTP Sink application then writes the message to file before transferring it over Secure FTP (sFTP).
- sFTP Sink Microservice Deployment
- ASN.1 Record Naming and Counting
  Each time the ASN.1 sFTP Sink application receives a batch of records, if the number of records is greater than or equal to the configured closure condition, the aggregated ASN.1 record is generated. Otherwise, the batch is held for the next ASN.1 record.
- CDR Aggregation Function
  Every request received by SBA Gateway from the SMF results in the creation of a 5G event, also known as a network _glossary/cdr.html. To avoid sending excessive records to a downstream system, CDRs must be aggregated on a per session basis so that only summary level CDRs are sent to third-party CGF or the BSS domain. The CDR Aggregation Function (CAF) application is a Kafka Streams application that aggregates network interactions handled by SBA Gateway on the 5G N40 interface.
- Content-Based CDR Routing
  The CDR Kafka router republishes each _glossary/cdr.html based on its content, either as received or enriched with additional data on a conditional basis.
- Write To Disk
  When Kafka is unavailable, the producer continues attempting to send to the Kafka broker until a timeout is reached. On timeout, the records are written to a separate log file on disk.
- Record Formats and Schemas
  Request and response records are sent in JSON format.
- Kafka Partition Key
  The Kafka partition key used is the SessionId for the exchange. If none is available, for example if there is an error returned by the engine, then no partition key is used.
- Installing 5G Event Streaming Applications
  The 5G event streaming applications are delivered as images for container creation in a Kubernetes cluster using a Helm chart. The following sections describe the installation process. The procedures assume you have knowledge of and access to the kubectl tool.
- Configuring 5G Event Streaming
  Configure 5G event streaming by providing values for properties in one or more Helm values files.
- Configuring the Applications
  The 5G event streaming applications are configured in the Helm values file, the nf.yaml file, and an application.yaml files for each application.
  - Kafka Producer Configuration Properties
    You can configure SBA Gateway as a Kafka producer for sending request and response information using the nf.yaml file, the MATRIXX Helm chart, or environment variables.
  - Kafka Configuration Overrides
    The Kafka producer client uses default configuration values that can be overridden by setting the property in the gateway.yaml file or by using environment variables. When overriding these options, prepend the prefix gateway.external.kafka.config to the property name.
  - Kafka Producer Filters
    The configuration can use filters to decide whether a request and its corresponding MATRIXX Engine response are sent to the Kafka broker.
  - Kafka Producer Metrics
    Kafka producers have a default set of metrics that can be reported with Prometheus.
  - Configuring CDR Kafka Router
    The CDR Kafka Router application is configured with Kafka Streams configuration properties and application-specific properties. The following sections describe the available configuration properties.
  - Configuring CDR Aggregation Function
    The CDR Aggregation Function (CAF) application is configured by Kafka Streams configuration properties and application-specific properties.
  - Configuring CDR ASN.1 Streamer Application
    The CDR ASN.1 Streamer application loads configuration from application.yaml by default.
  - Configuring ASN.1 sFTP Sink
    Several characteristics of the ASN.1 sFTP Sink application are configurable.
    - Kafka Configuration
      5g_integration/c_5g_events_sink_kafka.html#c_5g_events_sink_kafka__table_xjm_lml_3tb describes the most relevant Kafka-related configuration properties available for the ASN.1 sFTP Sink application.
    - Record Recovery Configuration
      5g_integration/c_5g_events_rec_recovery.html#concept_dt3_drr_r5b__table_krm_hrr_r5b describes the properties available for ASN.1 sFTP Sink application record recovery.
    - Local Backup Configuration
      5g_integration/c_5g_events_local_backup.html#concept_itk_nt2_55b__table_j5n_st2_55b describes the properties available for configuring local backup of ASN.1 files.
    - Storage Configuration
      Each instance of the ASN.1 sFTP Sink application requires its own persistent volume (PV). The persistent volume claim (PVC) is managed by a Kubernetes StatefulSet to ensure that the same volume is reattached when the application restarts.
    - File Closure Control Properties and Conditions
      Files can be closed and uploaded based on their age, size, or remaining storage space available.
    - File Naming
      Files can be named based on node ID, file count, date, time, and any private information.
    - CDR Versions
      5g_integration/c_5g_events_config_cdr_ver.html#concept_xl2_s3x_4xb__table_mrh_hkx_kpb describes the properties used to specify CDR version information.
    - CDR Routing Information
      5g_integration/c_5g_events_config_cdr_routing.html#concept_edy_pjx_4xb__table_cd5_qkx_kpbdescribes the property used to specify the CDR routing filter, as defined in 6.1.1.12 of 3GPP TS 32.297 v15.4.0. The default value encodes the route from the CHF through to the sink.
    - Upload Settings
      5g_integration/c_5g_events_sink_upload.html#concept_nvm_jkx_4xb__table_lvp_g4x_kpb describes properties for configuring several characteristics of upload.
    - Record Buffer
      5g_integration/c_5g_events_sink_rec_buf.html#concept_slf_jlx_4xb__table_ej3_mpx_kpb describes properties for configuring the record buffer.
    - ASN.1 File Upload Settings
      By default, the ASN.1 sFTP Sink application uploads the ASN.1 file with a .writing extension to indicate the file upload is in progress. After the ASN.1 file is uploaded, the sink renames the file back to the original filename without the .writing extension. If the temporary file is locked by the sFTP server (for example, as part of a scan), the sink waits a configured amount of time before it attempts to rename the file again. The renaming process repeats until the file is successfully renamed or the maximum number of attempts is reached. The properties listed in 5g_integration/c_5g_events_sink_asn_file_up.html#concept_ovs_mmx_4xb__table_eqd_r24_15b control this process.
    - Shutdown Hook
      By default, the ASN.1 sFTP Sink application implements a PreStop hook, which executes an HTTP request for the endpoint. The hook processes pending records attached to the persistent volume claim (PVC) before the application shuts down.
    - ASN.1 sFTP Sink Disk Write Retry
      You can configure a property to try rewriting a batch of records from Kafka to the disk after a specified delay if a write operation by the ASN.1 sFTP Sink application is unsuccessful. This configuration helps to prevent a disk failure if there is an issue with writing records to the disk.
    - Routing and sFTP Server Configuration
      The ASN.1 sFTP Sink application connects to an sFTP server to upload files.
  - Configuring GTP Prime Adapter
    The GTP Prime Adapter can be configured for processing CDR messages from a Kafka topic and converting them from JSON into ASN.1 using a dynamic mapper before sending them to a GTP server.
- Applying Configuration
  You can apply configuration for 5G event streaming applications in several ways using the Helm values chart. These methods are not mutually exclusive and can be used together.
- Appendixes
MATRIXX Event Streaming
MATRIXX Administration
MATRIXX Web App Administration
MATRIXX Monitoring and Logging
MATRIXX Policy
MATRIXX Kafka CDR Consumer
MATRIXX Pricing and Rating
MATRIXX Pay Now
MATRIXX Subscriber Management
MATRIXX Subscriber Management API
MATRIXX Business API SDK
My MATRIXX Help
MATRIXX Backoffice Customer Tool Help
MATRIXX Third-Party Licenses
Glossary

Upload Settings

Upload Configuration Properties describes properties for configuring several characteristics of upload.

Note: In release 5222 and later, the inclusion of individual record headers and the file header is optional. For more information, see the configuration option sftp.internal.uploader.fileFormat.headers.

Table 1. Upload Configuration Properties
Property	Description
sftp.internal.uploader.path	The internal path where files are exported before being uploaded. The default value is /opt/mtx/data/internal_upload.
sftp.internal.uploader.heartbeat	Interval in seconds between scans for new files to be uploaded. The default value is 10.
sftp.internal.uploader.counter.min	Minimum (and reset) value for the file sequence number. The default value is 0.
sftp.internal.uploader.counter.max	Maximum value for the file sequence number, after which the value is reset. The default value is `Integer.MAX_VALUE`.
sftp.internal.uploader.counter.path	The path on disk of the serialized counter. The default value is /opt/mtx/data/counter.
sftp.internal.uploader.counter.filename	The filename for the serialized counter. The default value is `counter`.
sftp.internal.uploader.fileFormat.runningCountPadding	When not `null`, left-pad the running count in the exported filename by the specified number of places. The running count resets to 1 when the max value is reached. For example, a padding of 3 give the sequence 001, 002 ... 999, then continue from 001. The default value is `null`.
sftp.internal.uploader.fileFormat.headers	(Release 5222 and later) Indicates whether the individual record headers and file header should be written to the file. Options are: `ALL` — Both the individual record headers and the file header are written. `NONE` — No headers are written. `FILE_HEADER_ONLY` — Only the file header is written to the file. `RECORD_HEADERS_ONLY` — Only the individual record headers are written. The default value is `ALL`.

Note: Once a file is successfully uploaded, it is deleted. If the upload.heartbeat is set to zero (0), files are not uploaded and remain in the directory.