Event Repository Quick Start (Sharded)

The topics in this section serve as a quick start guide for setting up the Event Repository on a MongoDB sharded deployment. If you plan to have a non-sharded cluster, refer to the quick start guide for setting up the Event Repository on a non-sharded MongoDB deployment. This information describes configuring the integration points between MATRIXX Engine and your MongoDB system only. For all instructions about setting up a sharded cluster in your MongoDB system, refer to your MongoDB product documentation.

The following is a summary of tasks for setting up the Event Repository on a sharded MongoDB deployment. The summary can be used as a task checklist during setup. Refer to a task-related topic for details about performing each task.

Important: While these instructions have quick start guidelines for installing and configuring MongoDB for your Event Repository, refer to your MongoDB product documentation as the primary source of information about installing, provisioning, and managing your MongoDB system. To plan your MongoDB system, be familiar with MongoDB operations, networking, and security best practices.
  1. Install MongoDB on every host server where database processes/tools run.

    You must purchase and install MongoDB individually.

  2. Install the PyMongo driver on every host server where database processes/tools run.
  3. Install the MongoDB mongos software on every host server where client applications are to run. See the discussion about Event Repository system requirements for information about software versions.
  4. (Recommended) Perform pre-requisite tasks on mongod host servers so that you can put indexes on SSDs (create a soft link "index" and create target index directory so that the index files MongoDB creates are stored there).
  5. Create a key file and distribute a copy to all host servers where any MongoDB process runs, including where mongos routing processes run.
  6. Edit the configuration options for each MongoDB instance.
    Note: Set values for base settings and instance-specific settings in the instance .conf file.
    1. Set the MongoDB configuration file options for each shard, for each member of each shard replica set.
    2. Set the MongoDB configuration file options for each config server for all three members of one config-server replica set.
    3. Set the MongoDB configuration file options for each query router (mongos) in your deployment.
  7. Start all mongod instances on each of the host servers in your deployment (do not start mongos instances yet).
  8. Initiate each shard replica set of your deployment.
  9. Initiate the config server replica set of your deployment.
  10. Start all mongos instances in your deployment.
  11. Create users for each shard replica set.

    You must be connected to a primary to create the users. The minimum user needed is the MtxAdmin user.

  12. Initiate the sharded cluster.
  13. Create sharded cluster users.

    You must be connected to a primary to create the users. The minimum users needed are the MtxAdmin and MtxEventLoader users.

    If you plan to use MATRIXX GL utilities to process GL information from your Event Repository, you must create MongoDB users to be used by the utilities.

  14. Download MATRIXX Engine to obtain the Event Repository administration scripts (which are included in the MATRIXX Engine RPM).
  15. Perform the required setup for running the Event Repository administration scripts such as creating script user credentials.
  16. Run the Event Repository administration script that saves a partition-method configuration and index definitions to your database (MtxEventDatabase).

    (GL Processing Only) Manually add GL index definitions before running this script. If you plan to use MATRIXX GL utilities to process GL information from your Event Repository, you must add additional indexes.

  17. Run the Event Repository administration script that checks the partition-method configuration and index definitions to ensure they are correct.
  18. Manually add any custom indexes (TTL). For example, if you plan to add TTL to the statuses and trace collections, you must add additional indexes.
  19. Run the Event Repository administration script that adds the indexes and collections, based on the saved partition and index configurations, to MongoDB (MtxEventDatabase).
  20. If you have not already done so, install MATRIXX Engine software.
  21. Configure MATRIXX Engine.
  22. As part of configuring MATRIXX Engine, answer the create_config.info configuration questions that pertain to the Event Repository. For details about each configuration question, see the discussion about MATRIXX Event Repository configuration in MATRIXX Configuration. You must configure the Event Loader connection string for connecting to MongoDB (connecting to a mongos).
  23. (GL utilities Only) Verify that the GL utilities are configured to connect to your MongoDB system. Set the GL utilities connection string for connecting to MongoDB (connecting to a mongos). See the discussion about setting the MongoDB connection string for GL utilities.
  24. Start MATRIXX Engine.

    When you start the engine, the publishing server starts and the Event Loader service that is part of the software stack on the publishing server starts.

  25. Run a workload.
  26. Verify that events are being loaded into MongoDB by the Event Loader.

The Event Repository setup is complete.

When MEFs are generated, the Event Loader service loads event objects into the Event Repository. You can verify event objects are loaded by running a workload and querying the document count in the EventCollection. Three scripts can be used to monitor the Event Repository and verify events are being loaded into it: print_event_repository_loader_stats.py, print_event_repository_loader_status.py, and print_event_repository_loader_trace.py. You can view configuration information and statistics about the event repository by running Event Repository administration scripts.