MEF Automatic Publishing

After you generate MATRIXX Event Files (MEFs), they are intended to be published to an off-engine location for archiving and failover. You use the MEF publishing parameters explained here to configure automatic MEF publication. The automatic MEF publishing process ensures that no data is lost during MEF publication if the active engine fails.

Note: If you implement MATRIXX on the Google Cloud Platform you can also publish MEFs to Google Cloud Buckets (GCBs).

Configure MEF Publishing

You configure the properties listed in MEF Publication Configuration Parameters for MEF publication when configuring MATRIXX Engine. For more information, see the discussion about MEF publication configuration in MATRIXX Configuration.
Table 1. MEF Publication Configuration Parameters
Parameter Description Default Value
Enable MEF Publication MATRIXX event file (MEF) publication is turned off by default.
Note: When publication is disabled, MEF is used for backward compatibility of configuration files.
To enable the publishing server to generate MEFs, answer y to the question: Do you want to publish MATRIXX Event Files (y/n)? 		n
Size Limit for MEFs The MEF is closed and the data is flushed when the configured size is exceeded.

create_config.info question: What is the size in MBytes that you want to use to trigger a file close for a MATRIXX Event File?

 100
Time Limit for MEFs The MEF is closed and the data is flushed when the configured time is exceeded. This question works in conjunction with the Idle Time Limit for MEFs question.

create_config.info question: What is the time in seconds that you want to use to trigger a file close for a MATRIXX Event File?

 30
Event Count Limit for MEFs The MEF is closed when the configured event count is exceeded.

create_config.info question: What is the event count that you want to use to trigger a file close for a MATRIXX Event File?

 500000
Idle Time Limit for MEFs The MEF is closed when idle for longer than the configured time. Enter 0 if you do not want to close the file when idle. This question works in conjunction with the Time Limit for MEFs question.

create_config.info question: What is the idle time in milliseconds that you want to use to trigger a file close for a MATRIXX Event File?

 0
MEF File Publishing Rate Specify the time interval in seconds to publish the MEFs to the publishing host after it is closed. Enter 0 to publish the MEF as soon as it is closed.

create_config.info question: How often (in seconds) should the MATRIXX Event Files be published?

 30
MEF Publication Username The username for MEF publishing. MEF publication must be enabled by answering y to the preceding question.
Important: This user on the target host must be accessed from the publishing server without having to enter a password. MATRIXX Support recommends using a network authentication protocol so that no password is needed.

create_config.info question: What is the user name to use when publishing MATRIXX Event Files?

 mtx
MEF Publication Host name/IP address The host name or IP address where MEFs are published. The mtx user must have password-less SSH access to this host.

create_config.info question: What is the host name/IP address to which MATRIXX Event Files will be published?

Note: If you are implementing MATRIXX on Google Cloud Platform (GCP), the answer to this question is ignored.
 n/a
MEF Publication Directory The full path where the MEFs are published, for example, /var/log/mtx/published_mefs. This directory must be manually created on the publishing host.

create_config.info question: What is full path to the directory on hostname where MATRIXX Event Files will be published?

Note: hostname is the host name or IP address entered in the earlier question.
If you implement customer sub-domains and your sub-domains are processed in separate MATRIXX engine chains, publish the MEFs for each sub-domain to a unique publishing directory. For example, on the off-engine host to which MEFs are published:
  • Specify /var/log/mtx/published_mefs1 for the active and standby engines processing customer sub-domain 1
  • Specify /var/log/mtx/published_mefs2 for the active and standby engines processing customer sub-domain 2
n/a
MEF Output Format
The format in which MEF data is published:
  • compact_mdc
  • element_xml
create_config.info question: What is the data format to use for the MATRIXX Event Files?
Note: To read the contents of a MEF file in compact MDC format, use the print_data_container_file.py script.

For an example of the output format, see the discussion about MEF output format.

 compact_mdc
Maximum Number of MEFs in Backlog The number of MEF input files generated by the dispatcher task for worker tasks to load. If the worker tasks fall behind (because MongoDB is down), the dispatcher delays making more MEFs until the worker tasks catch up.

create_config.info question:

What is the maximum number of MEFs that can be in backlog before delaying MEF creation?		 100

After an engine failover, if the MEF publishing target is configured, the newly active engine initiates the MEF recovery process. If automatic MEF recovery fails, an error is logged in the mtx_debug.log file on the newly active engine.