Chapter 29. Monitoring Activity

DOC has several types of system monitoring:

Monitoring Docker services. For more details, refer to Sections Launching the Infrastructure and Optimization Server Services and Launching the Application Services Using Docker;
Monitoring DOC activity; and
Monitoring the number of active sessions, either using the logs or a dedicated REST endpoint.

1. Monitoring the Activity

DOC exposes several activity metrics:

for the execution-service:
- EXECUTION_JOBS_ALIVE exposes the number of unfinished jobs.
- EXECUTION_JOBS_IDLE_TIME exposes the number of seconds since the latest job was finished.
for the scenario-service:
- SCENARIO_SESSIONS_ACTIVES exposes the number of active Keycloak sessions.
- GENE_IDLE_TIME (aggregated metric) exposes the number of seconds since the system was used for the last time.

These metrics are available though dedicated individual metric endpoints or via Prometheus.

For example, the curl command below returns the idle time of the system. Here, the value 0.0 indicates that the system is not idle, i.e. it is in use.

❯ curl -s http://scenario-service:8079/actuator/prometheus | grep gene_idle_time
# HELP gene_idle_time_seconds
# TYPE gene_idle_time_seconds gauge
gene_idle_time_seconds 0.0

2. Monitoring the Active Sessions

DOC includes a Keycloak SPI (Service Provider Interface) that emits an AMQP message when the number of sessions is updated. This can, for instance, happen when a user logs in or out. Note that no messages are emitted when a session expires. For more details, refer to Chapter Managing Users.

The Scenario Service listens to emitted messages and stores them in a MongoDB collection.

By default, there are two ways to view the stored messages:

by configuring the log level
by using a dedicated REST endpoint

The stored data are automatically deleted after two years.

2.1. Monitoring Active Sessions Using Logs

To update the logs with the number of active sessions, set the log level to debug for the class com.decisionbrain.keycloak.spi.event.consumer.impl.LoggerConsumer.

To do so, in the Scenario Service extension, add the following snippet to the application.yml file:

logging:
    level:
        com.decisionbrain.keycloak.spi.event.consumer.impl.LoggerConsumer: DEBUG

2.2. Monitoring Active Sessions Using REST

If needed, a REST endpoint allows accessing metrics about any user session. To access these metrics, you need to be logged in as a user that has role SESSION_TRACKING.

Then, you can call the endpoint /api/scenario/session-tracking/aggregation. This endpoint will respond a JSON having the following format:

{
    "min": 4,
    "max": 8
}

You can filter out the results by giving a start and an end parameter to the endpoint. These parameters represent timestamps.

For example: /api/scenario/session-tracking/aggregation?start=1651356000000&end=1653948000000