Predefined JMX importers

Lightbend Telemetry comes with some predefined importers to provide for easy import from JMX.

HikariCP importer

HikariCP is a recommended connection pool framework that is used in Play.

HikariCP dependency

Add the HikariCP JMX Importer dependency to your build:

sbt

libraryDependencies += Cinnamon.library.cinnamonHikariCPJmxImporter

Maven

<dependency>
    <groupId>com.lightbend.cinnamon</groupId>
    <artifactId>cinnamon-jmx-importer-hikaricp</artifactId>
    <version>2.17.5</version>
</dependency>

Gradle

dependencies {
    implementation group: 'com.lightbend.cinnamon', name: 'cinnamon-jmx-importer-hikaricp', version: '2.17.5'
}

Note

Make sure that HikariCP is configured to register MBeans by setting the registerMbeans setting to true for HikariCP.

HikariCP configuration

Required

No additional configuration is required for the HikariCP JMX importer to import the MBeans.

Reference

query = "com.zaxxer.hikari:type=Pool *"
attributes = [
  {
    attribute = "ActiveConnections",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "IdleConnections",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "ThreadsAwaitingConnection",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "TotalConnections",
    metric-type = "GAUGE_LONG"
  }
]
query = "com.zaxxer.hikari:type=PoolConfig *"
attributes = [
  {
    attribute = "ConnectionTimeout",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "IdleTimeout",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "LeakDetectionThreshold",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "MaxLifetime",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "MaximumPoolSize",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "MinimumIdle",
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = "ValidationTimeout",
    metric-type = "GAUGE_LONG"
  }
]

Note

These settings are defined in the reference.conf. You only need to specify any of these settings when you want to override the defaults.

Apache Kafka importers

Lightbend Telemetry comes with two importers for Apache Kafka to generate metrics for Kafka consumers and producers.

Apache Kafka importer dependency

Add the desired Kafka JMX Importer dependencies to your build:

sbt

// Consumer metric generation
libraryDependencies += Cinnamon.library.cinnamonKafkaConsumerJmxImporter

// Producer metric generation
libraryDependencies += Cinnamon.library.cinnamonKafkaProducerJmxImporter

Maven

<!-- Generate consumer metrics -->
<dependency>
    <groupId>com.lightbend.cinnamon</groupId>
    <artifactId>cinnamon-jmx-importer-kafka-consumer</artifactId>
    <version>2.17.5</version>
</dependency>

<!-- Generate producer metrics -->
<dependency>
    <groupId>com.lightbend.cinnamon</groupId>
    <artifactId>cinnamon-jmx-importer-kafka-producer</artifactId>
    <version>2.17.5</version>
</dependency>

Gradle

dependencies {
    // generate consumer metrics
    implementation group: 'com.lightbend.cinnamon', name: 'cinnamon-jmx-importer-kafka-consumer', version: '2.17.5'

    // generate producer metrics
    implementation group: 'com.lightbend.cinnamon', name: 'cinnamon-jmx-importer-kafka-producer', version: '2.17.5'
}

Apache Kafka importer configuration

Required

No additional configuration is required for the Kafka JMX importers to import the MBeans.

Reference (Consumer)

# Fetch Metrics
query = "kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*"
attributes = [
  {
    attribute = records-lag-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-size-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-size-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = bytes-consumed-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = records-per-request-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = records-consumed-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-latency-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-latency-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-throttle-time-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-throttle-time-max
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  }
]
# Topic-level Fetch Metrics
query = "kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*,topic=*"
attributes = [
  {
    attribute = fetch-size-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = fetch-size-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = bytes-consumed-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = records-per-request-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = records-consumed-rate
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  },
  {
    category = "topics"
    key = "topic"
  }
]
# Consumer Group Metrics
query = "kafka.consumer:type=consumer-coordinator-metrics,client-id=*"
attributes = [
  {
    attribute = assigned-partitions
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = commit-latency-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = commit-latency-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = commit-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = join-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = join-time-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = join-time-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = sync-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = sync-time-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = sync-time-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = heartbeat-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = heartbeat-response-time-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = last-heartbeat-seconds-ago
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  }
]
# Global Request Metrics
query = "kafka.consumer:type=consumer-metrics,client-id=*"
attributes = [
  {
    attribute = request-latency-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-latency-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = response-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = incoming-byte-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = outgoing-byte-rate
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  }
]
# Global Connection Metrics
query = "kafka.consumer:type=consumer-metrics,client-id=*"
attributes = [
  {
    attribute = connection-count
    metric-type = "GAUGE_LONG"
  },
  {
    attribute = connection-creation-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = connection-close-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-ratio
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-time-ns-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-wait-ratio
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = select-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-wait-time-ns-avg
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  }
]
# Per-Broker Metrics
query = "kafka.consumer:type=consumer-node-metrics,client-id=*,node-id=*"
attributes = [
  {
    attribute = request-size-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-size-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = response-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = incoming-byte-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = outgoing-byte-rate
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  },
  {
    category = "node-ids"
    key = "node-id"
  }
]

Reference (Producer)

# Global Request Metrics
query = "kafka.producer:type=producer-metrics,client-id=*"
attributes = [
  {
    attribute = request-latency-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-latency-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = response-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = incoming-byte-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = outgoing-byte-rate
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  }
]
# Global Connection Metrics
query = "kafka.producer:type=producer-metrics,client-id=*"
attributes = [
  {
    attribute = connection-count
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = connection-creation-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = connection-close-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-ratio
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-time-ns-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-wait-ratio
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = select-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = io-wait-time-ns-avg
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  }
]
# Per-Broker Metrics
query = "kafka.producer:type=producer-node-metrics,client-id=*,node-id=*"
attributes = [
  {
    attribute = request-size-max
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-size-avg
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = request-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = response-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = incoming-byte-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = outgoing-byte-rate
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  },
  {
    category = "node-ids"
    key = "node-id"
  }
]
# Per-Topic Metrics
query = "kafka.producer:type=producer-topic-metrics,client-id=*,topic=*"
attributes = [
  {
    attribute = byte-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = record-send-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = compression-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = record-retry-rate
    metric-type = "GAUGE_DOUBLE"
  },
  {
    attribute = record-error-rate
    metric-type = "GAUGE_DOUBLE"
  }
]
additional-identity-keys = [
  {
    category = "client-ids"
    key = "client-id"
  },
  {
    category = "topics"
    key = "topic"
  }
]

Extending the settings

It is possible to override and extend the above settings, as well as add more MBeans by extending the settings.