normalize_mtx_debug_log.py

The normalize_mtx_debug_log.py script removes fields and lines from log files that have variable information (such as memory addresses, time stamps, and PIDs) and writes the modified lines to an output file.

Syntax

normalize_mtx_debug_log.py [-d] [-i] [-f] [-t] [-r] [-o]

Options

-h, --help
Help about the script options.
-d
When this option is specified, extra messages are printed to help with debugging this script.
-i infile, --infile=infile
Specifies the input log file. This is the same as using the -f option. If neither argument is specified, stdin is used.
-f file, --file=file
Specifies the input log file. This is the same as using the -i option. If neither option is specified, stdin is used.
-o outfile, --outfile=outfile
Specifies the name of an output file. If you do not specify an output file, stdout is used.
-r, --reset
Resets the standard filters so that no default filters are used.
-t file name list, --filter=file name list
An extra filter. Use a filter name of "help" to see all possible values. Use a filter name of "reset" to reset the standard filters so that no default filters are used. If you precede the filter name with a "-," it removes it from the default list.

Filters

Filters describes the normalization filters that you can specify in the filter list.
Table 1. Filters
Filter Description
bufferIdLines Remove the <id> from buffer ID: "lines.
configInitLines Remove the Config::init lines.
configMapLines Remove the Config NameValueMap lines.
contactSubscriberIdLines Remove the ContactSubscriberId lines.
dataContainerDescIndexCstorLines Remove the DataContainerDescriptorIndex constructor lines.
dataContainerFieldOffsetAndSize Remove the MATRIXX Data Container (MDC) field's offset and MaxSz values.
dataContainerHeader Remove the MDC header sizes.
dateTime Remove date-time fields.
errFileNameAndLineNumbers Remove the filename and line number from a printed Err object.
eventIdField Remove EventID values.
eventIdLines Remove lines with EventId.
eventTimeLines Remove the EventTime lines.
freeEntryKeysAndValues Remove the free entry keys and values.
lmDebugOrConfigPrintMapOrBlank Remove lines that have LM_DEBUG, are from a printout of the configuration map, or are blank.
lmLines Remove lines with LM_.
lmReleaseVersion Remove the release version from the LM_ lines.
lmTimestamps Remove the LM_ timestamp field.
maxBufferSize Remove the <size> from bufMaxSize=<size> lines.
memoryAddresses Remove the memory addresses (0x12345678).
minDescriptorKey Remove the DataContainerDescriptorIndex: MinDescriptorKey lines.
msgRdmIdLines Remove lines with MsgRdmId.
notificationIdLines Remove the NotificationId lines.
oid Remove OID values.
perMsgStatsLines Remove per message statistics lines.
pidValues Remove PID values.
queueStatsLines Remove the queue statistics lines.
rdmIdLines Remove the RDM id lines.
rdmIds Remove the RDM id values.
rdmIdsLower Remove the rdm ID values.
receiveFromFd Remove the Received from socket fd text.
replaceIdLower Replace RDM ID values preceded by ID= or ID with ?.
replaceIdUpper Replace RDM ID values preceded by ID= or ID with ?.
replaceLmInfoPid Replace PID values in LM_INFO lines with <pid>.
replaceMaxDescriptorKeyValue Replace the MaxDescriptorKey four-digit value with max key.
replaceMicroSecTimeStamp Replace time stamp fields matching YYYY-MM-DD HH:mm:SS.nnnnnn with TimeStamp.
replaceMicroSecTimeStampZ Replace time stamp fields matching YYYY-MM-DD HH:mm:SS.nnnnnnZ with TimeStamp.
replaceMtxSubscriberObjectValues Replace MtxSubscriberObject field values with ?.
replaceSubscriberFieldValues Replace Subscriber field values with ?.
replaceWalletFieldValues Replace Wallet field values with ?.
replaceZuluTimeStamp Replace time stamp fields matching YYYY-MM-DD HH:mm:SSZ with TimeStamp.
serviceSchemaVersionsXml Remove the service_provider schema versions (in XML format).
settlementTimeField Remove the SettlementTime lines.
socketFd Remove the socket fd=.., text.
socketIdLines Remove the SocketId lines.
stackTraceLines Remove the stack trace lines.
systemSchemaVersionsCompact Remove the system schema versions (compact format).
systemSchemaVersionsContainerId Remove the system schema versions (descriptive format) from containerId= fields.
systemSchemaVersionsDescName Remove the system schema versions (descriptive format) from descName= fields.
systemSchemaVersionsXml Remove the system schema versions (in XML format).
timeFieldsLines Remove the CreateTime, UpdateTime, ReceiveTime, and TimeArray lines.
totalFreeValues Remove the <size> from Total Free: <size>.
totalInUseValues Remove the <size> from Total In Use: <size>.
transactionIds Remove the transaction IDs.
variableRdmStats Remove variable RDM statistics.
xercesLines Remove Xerces is being called for lines.

Using Default Filters

The following is an example of using default filters:
normalize_mtx_debug_log.py -f mtx_debug.log > mtx_debug_normalized.log

Using a Specific Filter

The following is an example of using a specific filter:
normalize_mtx_debug_log.py -f mtx_debug.log -t replaceWalletFieldValues > mtx_debug_normalized.log

Excluding a Specific Filter

The following is an example of excluding a specific filter:
normalize_mtx_debug_log.py -f mtx_debug.log -t "-totalFreeValues" > mtx_debug_normalized.log

Excluding Default Filters

The following is an example of excluding default filters:
normalize_mtx_debug_log.py -f mtx_debug.log --reset -t lmTimestamps > mtx_debug_normalized.log