Home
MATRIXX Integration
Data Export
The MATRIXX Data Export image (data-export) has the data-export.jar utility. This utility uses checkpoint files to transform database contents and MATRIXX Event Files (MEFs) to comma-separated value (CSV) files. This image also creates SQL files for creating RDBMS tables and loading the data from the CSV files.
Export Data from the Subscriber Database
MATRIXX provides an example subscriber_export.yaml file to use with the data-export image to convert checkpoint data to CSV format. The image also generates script files to create SQL tables and load the exported subscriber data into the SQL tables.

Welcome
MATRIXX Release Notes
MATRIXX Architecture
MATRIXX Installation and Upgrade
MATRIXX Configuration
MATRIXX Security
MATRIXX Integration
- About MATRIXX Integration
  MATRIXX Integration describes the tools and tasks for integrating MATRIXX functionality in a MATRIXX environment.
- MATRIXX System MDC Definitions
  The /opt/mtx/data/mdc_config_system.xml file has all system MATRIXX Data Container (MDC) definitions, subtype definitions, and relationship definitions.
- Cloud Native Utilities
  MATRIXX includes images that contain utilities to aid in the deployment, operation, and configuration of cloud native MATRIXX installations. You can run these utilities directly or scheduled as cron jobs using the MATRIXX Helm chart.
- MATRIXX Event Detail Records
  MATRIXX Engine generates an _glossary/edr.html for all activities that can trigger rating, such as usage, catalog item purchases and cancelations, recurring cycle processing, or the first use of a balance. MATRIXX Engine also generates EDRs for non-rated events such as forfeitures. You configure event generation in My MATRIXX, including the events to generate and whether to add a custom container with additional mapped fields.
- MATRIXX Event Files
  MATRIXX Event Files (MEFs) contain a list of usage and non-usage event types generated during transaction processing. MEFs are meant to be published to, and consumed outside of, MATRIXX, and can be input to third-party applications.
- Subscriber Data
  This section provides an overview of subscriber data relationships.
- Event Repository
  A MATRIXX Engine can optionally load EDRs (Event Detail Records) into the MATRIXX Event Repository, a database separate from the MATRIXX Engine platform that serves as a long-term event storage repository. The Event Repository is an optional component separate from MATRIXX Engine that is implemented in MongoDB (you must purchase and install MongoDB).
- MATRIXX Notification Framework
  This section describes how the MATRIXX Engine publishes notifications to a JMS message broker and how you can customize the information included in notifications.
- Business API Gateway
  The Business API Gateway is a development framework that includes the Business API SDK, Rest Services Gateway (RS Gateway), Gateway Proxy, and application tier. It allows custom mobile applications, web clients, and other custom implementations to integrate with MATRIXX.
- Extending MATRIXX Engine Functionality
  You can extend MATRIXX Engine functionality to capture new subscriber-related data and then use the extended fields as a basis for pricing, rating, and policy decisions. You can also add new Diameter AVPs to capture information from the network and base rating and post-rating operations on this data.
- Data Export
  The MATRIXX Data Export image (data-export) has the data-export.jar utility. This utility uses checkpoint files to transform database contents and MATRIXX Event Files (MEFs) to comma-separated value (CSV) files. This image also creates SQL files for creating RDBMS tables and loading the data from the CSV files.
  - Create a Custom Data Export Configuration File
    You can customize the entries in the YAML file that determine which MTX objects and fields are exported to CSV files.
  - Export Data from the Subscriber Database
    MATRIXX provides an example subscriber_export.yaml file to use with the data-export image to convert checkpoint data to CSV format. The image also generates script files to create SQL tables and load the exported subscriber data into the SQL tables.
    - Exported Subscriber Database Data Model
      dbs_storage/r_cn_exported_subscriber_database_model.html#r_cn_exported_subscriber_database_model__fig_A04654A30B19448AB47E6C2DD23B2862 shows the relational data model produced using the subscriber_export.yaml file to export data from the Subscriber database.
  - Export Event Data from MATRIXX Event Files
    The event_export.yaml file is provided for you to use with the data-export image to export event data from MATRIXX Event Files (MEFs) to CSV files. The program also generates script files to create SQL tables and load the exported event data into the SQL tables. Data export supports MEF input in compact MDC format only.
  - Export Data from the Subscriber Database and MATRIXX Event Files
    The data_export.yaml file is an example configuration file for use with the data-export image to transform subscriber data in static checkpoint files and event data in MATRIXX Event Files (MEFs) to CSV format. The program also generates script files to create SQL tables and load the exported subscriber and event data into the SQL tables.
  - Adding Custom Fields to Export
    The add_custom_field_to_export.py script is used to add custom MDC fields to a configuration YAML file that is used by the data-export image.
- Selective Updates
  Selective updates are commands that enable the system to selectively update a MATRIXX Data Container (MDC) as it is processed. They perform basic transformations on the MDCs.
- MATRIXX Toolbox Utilities
  The MATRIXX Toolbox image has scripts and utilities to aid in the deployment and operation of cloud native MATRIXX installations.
- MATRIXX Miscellaneous Utilities
  The MATRIXX Misc image includes utilities and files to aid in the deployment and operation of cloud native MATRIXX.
- MATRIXX Third-Party Analysis Utilities
  You can use the third-party Linux tcpdump utility to capture network traffic for a MATRIXX application.
- General Ledger Records
  MATRIXX generates General Ledger (GL) information for accounts that you specify during rating. It then captures that information in MATRIXX Event Files (MEFs).
- Appendixes
MATRIXX Diameter Integration
MATRIXX Call Control Framework Integration
MATRIXX TM Forum Integration
MATRIXX 5G Integration
MATRIXX 5G Event Streaming
MATRIXX Event Streaming
MATRIXX Administration
MATRIXX Web App Administration
MATRIXX Monitoring and Logging
MATRIXX Policy
MATRIXX Kafka CDR Consumer
MATRIXX Pricing and Rating
MATRIXX Pay Now
MATRIXX Subscriber Management
MATRIXX Subscriber Management API
MATRIXX Business API SDK
My MATRIXX Help
MATRIXX Backoffice Customer Tool Help
MATRIXX Third-Party Licenses
Glossary

Export Data from the Subscriber Database

MATRIXX provides an example subscriber_export.yaml file to use with the data-export image to convert checkpoint data to CSV format. The image also generates script files to create SQL tables and load the exported subscriber data into the SQL tables.

Procedure

(Optional) Run the /opt/mtx/bin/create_checkpoint.py file to create a new checkpoint of the in-memory databases.
This step is necessary only if a recent checkpoint does not exist on the shared storage device. The publishing server creates these checkpoints at a configured interval. If you create a new checkpoint, specify the checkpoint directory with MTX_INPUT_DIRECTORY parameter.
Run the data-export image with the example subscriber_export.yaml file as the entry for MTX_EXPORT_CONFIG_FILE or the file that you customized before.
The following is an example using Docker:
```
docker run --rm \
    -v /temp/mtx/engine/config/output/mdc_config_custom.xml:/opt/mtx/conf/mdc_config_custom.xml \
    -v /opt/mtx/shared:/shared \
    -v /opt/mtx/data/export:/opt/mtx/data/export \
    -v /matrixx-shared-logging-storage/Pricing/jdoe/subscriber_export.yaml:/opt/mtx/data/subscriber_export.yaml \
    -e MTX_EXPORT_CONFIG_FILE="subscriber_export.yaml" \
    -e MTX_VERSION=version \
    -e MTX_OUTPUT_DIRECTORY_POSTFIX_EXP="echo "/$(date +"%Y%m%d-%H%M%S")"" \
    my_registry/data-export:version \
    
```
Where my_registry is the name of your registry and versionis the version of your data-export image.
Three sets of files are generated when the data-export image successfully completes:
- Extracted subscriber data that is written to one or more CSV files.
- A script file to create SQL tables for importing the extracted subscriber data.
- A script file to load the exported subscriber data into the SQL tables after they are created.
Run the create_tables.sql script to create new SQL tables.
Run the load_tables.sql script to import the CSV data into the new SQL tables.

Results

The exported data are available in SQL tables and are ready for analysis. See the discussion about the exported Subscriber database data model for the database schema diagrams.