Home
MATRIXX Integration
Subscriber Data
This section provides an overview of subscriber data relationships.
Exporting Subscription Data
MATRIXX provides a Java utility, data_export.jar, which uses checkpoint files to transform the database contents and MATRIXX Event Files (MEFs) to comma-separated value (CSV) files. The data_export.jar file also creates SQL files for creating RDBMS tables and loading the data from the CSV files.
Export Data from the Subscriber Database
The subscriber_export.yaml file is a sample configuration file that is provided for you to use with the data_export.jar utility for converting checkpoint data to CSV format. The utility also generates script files to create SQL tables and to load the exported subscriber data into the SQL tables.

Welcome
MATRIXX Release Notes
MATRIXX Architecture
MATRIXX Installation and Upgrade
MATRIXX Configuration
MATRIXX Security
MATRIXX Integration
- About MATRIXX Integration
  MATRIXX Integration describes the tools and tasks for integrating MATRIXX functionality in a MATRIXX environment.
- MATRIXX System MDC Definitions
  The /opt/mtx/data/mdc_config_system.xml file has all system MATRIXX Data Container (MDC) definitions, subtype definitions, and relationship definitions.
- Cloud Native Utilities
  MATRIXX includes images that contain utilities to aid in the deployment, operation, and configuration of cloud native MATRIXX installations. You can run these utilities directly or scheduled as cron jobs using the MATRIXX Helm chart.
- MATRIXX Event Detail Records
  MATRIXX Engine generates an _glossary/edr.html for all activities that can trigger rating, such as usage, catalog item purchases and cancelations, recurring cycle processing, or the first use of a balance. MATRIXX Engine also generates EDRs for non-rated events such as forfeitures. You configure event generation in My MATRIXX, including the events to generate and whether to add a custom container with additional mapped fields.
- MATRIXX Event Files
  MATRIXX Event Files (MEFs) contain a list of usage and non-usage event types generated during transaction processing. MEFs are meant to be published to, and consumed outside of, MATRIXX, and can be input to third-party applications.
- Subscriber Data
  This section provides an overview of subscriber data relationships.
  - MATRIXX In-Memory Databases
    MATRIXX Engine uses an in-memory database set to hold all transactional data. This object-oriented database works seamlessly with the non-locking transaction model. The database allows multiple read and update operations to the same objects to occur simultaneously while maintaining full transactional integrity.
  - Database Checkpoints
    Database checkpoints contain an exact snapshot of the in-memory databases (IMDBs) and configuration files at a specific time. Periodic database checkpoints are used to recover data from whole cluster failures and to reinstate an active processing cluster from a standby processing cluster after a disaster recovery operation. Use on-demand checkpoints for data analysis, setting up test environments, and troubleshooting. You confirm checkpoints using the validateCheckpoint.jar utility.
  - Implementation Overview and Constraints
    Object Relational Mapping (O/R Mapping) creates a map from MATRIXX Data Container (MDC)-based objects to a relational database model for various use cases. A database version of an MDC can be created from a checkpoint and used for queries, data reporting, and analysis.
  - Database Schema Design
    The MATRIXX Subscriber database consists of four principal objects, all derived from MtxObject: MtxSubscriberObject, MtxGroupObject, MtxDeviceObject, and MtxWalletObject. Each one is mapped onto a corresponding physical table following the one-table-per-concrete-type model.
  - Exporting Subscription Data
    MATRIXX provides a Java utility, data_export.jar, which uses checkpoint files to transform the database contents and MATRIXX Event Files (MEFs) to comma-separated value (CSV) files. The data_export.jar file also creates SQL files for creating RDBMS tables and loading the data from the CSV files.
    - Create a Custom Data Export Configuration File
      You can customize the entries in the YAML file that determine which MTX objects and fields are exported to CSV files. If you have custom MATRIXX Data Containers (MDCs) that extend MDCs, you can use the add_custom_field_to_export.py script to create a new YAML configuration file that includes your custom fields.
    - Export Data from the Subscriber Database
      The subscriber_export.yaml file is a sample configuration file that is provided for you to use with the data_export.jar utility for converting checkpoint data to CSV format. The utility also generates script files to create SQL tables and to load the exported subscriber data into the SQL tables.
      - Exported Subscriber Database Data Model
        The following figure shows the relational data model produced using the subscriber_export.yaml file to export data from the Subscriber database.
    - Export Event Data from MATRIXX Event Files
      The event_export.yaml file is an example configuration file that is provided for you to use with the data_export.jar utility for exporting event data from MATRIXX Event Files (MEFs) to CSV files. The utility also generates script files to create SQL tables and to load the exported Event data into the SQL tables. data_export.jar supports MEF input in compact MDC format only.
    - Export Data from the Subscriber Database and MATRIXX Event Files
      The data_export.yaml file is a sample configuration file for use with the data_export.jar utility for transforming both subscriber data in static checkpoint files and event data in MATRIXX Event Files (MEFs) to CSV format. The utility also generates script files to create SQL tables and to load the exported subscriber and event data into the SQL tables.
- Event Repository
  A MATRIXX Engine can optionally load EDRs (Event Detail Records) into the MATRIXX Event Repository, a database separate from the MATRIXX Engine platform that serves as a long-term event storage repository. The Event Repository is an optional component separate from MATRIXX Engine that is implemented in MongoDB (you must purchase and install MongoDB).
- MATRIXX Notification Framework
  This section describes how the MATRIXX Engine publishes notifications to a JMS message broker and how you can customize the information included in notifications.
- Business API Gateway
  The Business API Gateway is a development framework that includes the Business API SDK, Rest Services Gateway (RS Gateway), Gateway Proxy, and application tier. It allows custom mobile applications, web clients, and other custom implementations to integrate with MATRIXX.
- Extending MATRIXX Engine Functionality
  You can extend MATRIXX Engine functionality to capture new subscriber-related data and then use the extended fields as a basis for pricing, rating, and policy decisions. You can also add new Diameter AVPs to capture information from the network and base rating and post-rating operations on this data.
- Data Export
  The MATRIXX Data Export image (data-export) has the data-export.jar utility. This utility uses checkpoint files to transform database contents and MATRIXX Event Files (MEFs) to comma-separated value (CSV) files. This image also creates SQL files for creating RDBMS tables and loading the data from the CSV files.
- Selective Updates
  Selective updates are commands that enable the system to selectively update a MATRIXX Data Container (MDC) as it is processed. They perform basic transformations on the MDCs.
- MATRIXX Toolbox Utilities
  The MATRIXX Toolbox image has scripts and utilities to aid in the deployment and operation of cloud native MATRIXX installations.
- MATRIXX Miscellaneous Utilities
  The MATRIXX Misc image includes utilities and files to aid in the deployment and operation of cloud native MATRIXX.
- MATRIXX Third-Party Analysis Utilities
  You can use the third-party Linux tcpdump utility to capture network traffic for a MATRIXX application.
- General Ledger Records
  MATRIXX generates General Ledger (GL) information for accounts that you specify during rating. It then captures that information in MATRIXX Event Files (MEFs).
- Appendixes
MATRIXX Diameter Integration
MATRIXX Call Control Framework Integration
MATRIXX TM Forum Integration
MATRIXX 5G Integration
MATRIXX 5G Event Streaming
MATRIXX Event Streaming
MATRIXX Administration
MATRIXX Web App Administration
MATRIXX Monitoring and Logging
MATRIXX Policy
MATRIXX Kafka CDR Consumer
MATRIXX Pricing and Rating
MATRIXX Pay Now
MATRIXX Subscriber Management
MATRIXX Subscriber Management API
MATRIXX Business API SDK
My MATRIXX Help
MATRIXX Backoffice Customer Tool Help
MATRIXX Third-Party Licenses
Glossary

Export Data from the Subscriber Database

The subscriber_export.yaml file is a sample configuration file that is provided for you to use with the data_export.jar utility for converting checkpoint data to CSV format. The utility also generates script files to create SQL tables and to load the exported subscriber data into the SQL tables.

Procedure

(Optional) Run the /opt/mtx/bin/create_checkpoint.py file to create a new checkpoint of the in-memory databases.
This step is necessary only if a recent checkpoint does not exist on the shared storage device. The publishing blade creates these at a configured interval. The MTX_INPUT_DIRECTORY parameter described in step 2 must point to the directory where the checkpoint files are located.
Run the /opt/mtx/bin/data_export.jar utility with the sample subscriber_export.yaml file as the entry for exportConfigFile or the file that you customized previously.
```
java -jar data_export.jar <inputFolder> <outputFolder> <customMdcConfigFile> <MtxVersion> <exportConfigFile> [sqlSchemaName]
```
For example:
```
java -jar /opt/mtx/bin/data_export.jar /opt/mtx/checkpoints/mtx_ckpt_v5100.2.1421221572 /tmp/subscriber_export/csv /opt/mtx/conf/mdc_config_custom.xml 5100 /opt/mtx/data/subscriber_export.yaml
```
See the discussion about data_export.jar for a complete description of each of the data_export.jar parameters.
Three sets of files are generated when the data_export.jar utility successfully completes:
- Extracted subscriber data that is written to one or more CSV files.
- A script file to create SQL tables for importing the extracted subscriber data.
- A script file to load the exported subscriber data into the SQL tables after they are created.
Run the create_tables.sql script to create new SQL tables.
Run the load_tables.sql script to import the CSV data into the new SQL tables.

Results

The exported data are available in SQL tables and are ready for analysis. See the discussion about the exported Subscriber database data model for the database schema diagrams.