Data Export
The MATRIXX Data Export Docker image (data-export) contains the data-export.jar utility which uses checkpoint files to transform database contents and MATRIXX Event File (MEF) files to comma-separated value (CSV) files. This image also creates SQL files for creating RDBMS tables and loading the data from the CSV files.
For information about Docker images, see the discussion about MATRIXX Digital Commerce Docker images and supporting files in MATRIXX Cloud Native Installation and Upgrade.
- If necessary, running the create_checkpoint.py script on a publishing blade to create a new checkpoint of the in-memory database.
- Running the data-export image to create CSV files used to load data into a relational database.
- Running SQL scripts to create RDBMS tables and load the data.
See the discussion about third-party software requirements in Installation and Configuration for the supported SQL database release level.
Data Export
The data-export image takes MEFs and static checkpoints of the following databases as input:
- Pricing
- Balance set
- Subscriber
Enter export parameters at the command line or use a
configuration file to determine which MTX objects and fields to export. You can
specify if the objects in the checkpoint and MEF files are transformed to SQL tables
in the exported CSV files. The configuration file is a Java properties file with a
.yaml file extension and its entries are stated in
name:value
format. Sample .yaml files are
provided under the /opt/mtx/data directory, or you can create a
custom configuration file.
If you have custom MDCs that extend MATRIXX MDCs, use the add_custom_field_to_export.py script to create a custom .yaml configuration file that includes your custom fields. For details, see the discussion about add_custom_field_to_export.py in MATRIXX Engine Integration.
Output Files
Based on the configurations in the .yaml file, the data-export image will generate the following files:
- The extracted database content in one or more CSV files.
- The create_tables.sql script,
which creates new SQL tables for the exported data.Note: The properties files and the CREATE TABLE statements in the create_tables.sql script provide descriptions of the fields in the output CSV files.
- The load_tables.sql script, which loads the exported CSV data into the SQL tables.
Database Export and SQL Loading Process
- Select or create a configuration
.yaml file to use with the
data-export image based on the type of MTX objects
you want to export (subscriber, event, or both). Tip: If you have custom MDCs that extend MATRIXX MDCs, use the add_custom_field_to_export.py script to create a new .yaml configuration file that includes your custom fields to use as input for the data-export image.
- Use the configuration .yaml file to run the data-export image and export your data.
- Execute the create_tables.sql script generated by the data-export image to create new SQL tables.
- Execute the load_tables.sql script generated by the data-export image to load the exported CSV data into the new SQL tables.