Polling the Prometheus REST API

You can monitor system load with a Prometheus REST API, using PromQL.

The variables section in the common configuration (gateway.yaml) file is as follows:

# Global variables. These can be used in NRF registration/hearbeating.
# Each variable must have a unique name and can be referenced
# in the NRF configuration section.
variables:
  # Define static variables here with a value.
  static:
    - name: example
      value: 1.0
  # Define variables that will be populated by querying a REST API here.
  # The REST API is assumed to return a JSON payload.
  restApi:
    -
      enabled: true
      # Polling interval in seconds (minimum 1)
      pollingInterval: 10
      # URL to query
      # (Example PromQL query to get the CPU usage as a percentage)
      url: http://localhost:9090/api/v1/query?query=ceil(100 - (avg (irate(node_cpu_seconds_total{job="node",mode="idle"}[5m])) * 100))
      # HTTP method to use
      method: GET
      # Optional path to a file containing a body (e.g. for POST requests).
      # body: /opt/mtx/conf/loadQuery.json
      # Optional additional headers to send in the query (e.g. for HTTP BASIC authentication)
      # headers:
      #   Authorization: Basic <credentials>
      # Values are read from the JSON response of a REST API query.
      # Paths to the values are defined using jsonPath syntax.
      values:
        - name: load
          # default value to use if value not present or error occurs
          value: 1
          # Path to the value in the JSON response
          jsonPath: $.data.result[0].value[1]
          # Convert the value received in the JSON to a different data type.
          # Allowed values: STRING, NUMBER, BOOLEAN, DECIMAL
          convertToType: NUMBER
   -
      enabled: false
      # Polling interval in seconds (minimum 1)
      pollingInterval: 10
      # URL to query
      # (Example PromQL query to get the CPU usage as a percentage)
      url: http://localhost:9090/api/v1/query?query=sum(increase(vertx_http_server_requests_total{application="sba-5gc-networkfunctions-chf", code=~"2.*"}[1m]))/sum(increase(vertx_http_server_requests_total{application="sba-5gc-networkfunctions-chf"}[1m])) <= bool 1
      # HTTP method to use
      method: GET
      # Optional path to a file containing a body (e.g. for POST requests).
      # body: /opt/mtx/conf/loadQuery.json
      # Optional additional headers to send in the query (e.g. for HTTP BASIC authentication)
      # headers:
      #   Authorization: Basic <credentials>
      # Values are read from the JSON response of a REST API query.
      # Paths to the values are defined using jsonPath syntax.
      values:
        - name: nfStatus
          # default value to use if value not present or error occurs
          value: REGISTERED
          # default value to use if value not present or error occurs
          integerToStringMap:
            0: REGISTERED
            1: SUSPENDED
          # Path to the value in the JSON response
          jsonPath: $.data.result[0].value[1]
          # Convert the value received in the JSON to a different data type.
          # Allowed values: STRING, NUMBER, BOOLEAN, DECIMAL
          convertToType: STRING

Variables must have a unique name, and can either be statically defined or populated from a REST API. The example shows an HTTP GET request sent to the configured URL every 10 seconds. The result is assumed to be in JSON format. When the result is received, the load variable is populated from the JSON object using jsonPath to define the path to the value in the JSON document.

For more information about jsonPath syntax, see the jsonPath documentation on the GitHub website.

For more information about PromQL and the Prometheus REST API, see the Prometheus documentation on the Prometheus website.

The following example shows a Prometheus query and response:

GET http://localhost:9090/api/v1/query?query=ceil(100 - (avg (irate(node_cpu_seconds_total{job="node",mode="idle"}[5m])) * 100))


200 OK
{
    "status": "success",
    "data": {
        "resultType": "vector",
        "result": [
            {
                "metric": {},
                "value": [
                    1580823078.046,
                    "8"
                ]
            }
        ]
    }
}

http://localhost:9090/api/v1/query?query=sum(increase(vertx_http_server_requests_total{application="sba-5gc-networkfunctions-chf", code=~"2.*"}[1m]))/sum(increase(vertx_http_server_requests_total{application="sba-5gc-networkfunctions-chf"}[1m])) <= bool 1
200 OK
{
    "status": "success",
    "data": {
        "resultType": "vector",
        "result": [
            {
                "metric": {},
                "value": [
                    1580823078.046,
                    "1"
                ]
            }
        ]
    }
}

The query has the following parts:

(irate(node_cpu_seconds_total{job="node",mode="idle"}[5m])): This returns the idle CPU counter and applies the irate function to turn it into a per-second value.
(100 - (avg (irate(node_cpu_seconds_total{job="node",mode="idle"}[5m])) * 100): The rate is then averaged over all CPUs and turned into a percentage.
ceil(100 - (avg (irate(node_cpu_seconds_total{job="node",mode="idle"}[5m])) * 100)): The final value is rounded up to the nearest whole number with the ciel function.

This returns the JSON shown in the example. The following configuration references the value (8) in the example JSON:

jsonPath: $.data.result[0].value[1]

This references item 1 in the value array, within item 0 in the result array, within the data object.

Once the load metric is stored in a variable, the following configuration adds this value to the registration and heartbeat messages:

nrf:
 ## nrf config here
   registrationVariables:
      -
       # Set the "load" field to the value of the variable "load"       
       jsonPath: $       
       field: load       
       variable: load
 
  heartbeatVariables:
    -
  #    # Set the "value" field in the object with a path equal to '/load' to the value of the variable "load"
      jsonPath: $[?(@.path == '/load')]
      field: value
      variable: load
    -
      jsonPath: $[?(@.path == '/nfServices/0/nfServiceStatus')]
      field: value
      variable: nfStatus
    -
      jsonPath: $[?(@.path == '/nfServices/1/nfServiceStatus')]
      field: value
      variable: nfStatus

The one-element mapping sequence registrationVariables sets fields in the nfProfile used during registration to the value of the specified variable. The heartbeatVariables sequence sets fields in the heartbeat message to the value of the specified variable.