Akka Insights

Version 2.20.3

Metrics and Events

This section contains a list of the different metrics and events for the different instrumentations. Please note that the units described here in the meta data is just a textual representation of the units that the metrics will be collected and reported in regardless of any changes to this configuration. That means that for example changing a nanoseconds to milliseconds will not change the unit that the metric is collected and reported in. Changing the unit that for example times are reported in is something that has to be done in the plugin or reporter configuration.

Note

Any metric or event can be disabled by setting the enabled configuration to false as described here. It is also possible to rate limit events to manage the amount of data being reported.

Akka Actor

Metric Type Description
ActorFailure Event when an actor fails, i.e. throws an exception. Event information: actor-ref — the actor failing cause — the exception being thrown
OpenTelemetry
akka.actor.actor-failure
Prometheus
akka_actor_actor_failure
Elastic
actors.actor-failure
Config
cinnamon.akka.meta.descriptor {
  actor {
    actor-failure {
      key = "actor-failure"
    }
  }
}
DeadLetter Event when a message is sent to an actor that no longer exists. Event information: recipient — the intended recipient of the message message — the message being sent actor-ref — the actor sending the message
OpenTelemetry
akka.actor.dead-letter
Prometheus
akka_actor_dead_letter
Elastic
actors.dead-letter
Config
cinnamon.akka.meta.descriptor {
  actor {
    dead-letter {
      key = "dead-letter"
    }
  }
}
Dropped messages Rate statistics for the number of messages dropped from bounded mailboxes per actor
OpenTelemetry
akka.actor.dropped-messages
Prometheus
akka_actor_dropped_messages
Elastic
actors.dropped-messages
Config
cinnamon.akka.meta.descriptor {
  actor {
    dropped-messages {
      key = "dropped-messages"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
LogError Event when an actor logs an error. Event information: actor-ref — the actor logging the error error — the error being logged
OpenTelemetry
akka.actor.log-error
Prometheus
akka_actor_log_error
Elastic
actors.log-error
Config
cinnamon.akka.meta.descriptor {
  actor {
    log-error {
      key = "log-error"
    }
  }
}
LogWarning Event when an actor logs a warning. Event information: actor-ref — the actor logging the warning warning — the warning being logged
OpenTelemetry
akka.actor.log-warning
Prometheus
akka_actor_log_warning
Elastic
actors.log-warning
Config
cinnamon.akka.meta.descriptor {
  actor {
    log-warning {
      key = "log-warning"
    }
  }
}
Mailbox size Recorder statistics for actor mailbox sizes
OpenTelemetry
akka.actor.mailbox-size
Prometheus
akka_actor_mailbox_size
Elastic
actors.mailbox-size
Config
cinnamon.akka.meta.descriptor {
  actor {
    mailbox-size {
      key = "mailbox-size"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
MailboxSizeLimit Event mailbox queue grows too large. Event information: actor-ref — the actor whose mailbox size has exceeded the limit message — the message being enqueued in the mailbox size — the mailbox size limit — the mailbox size limit
OpenTelemetry
akka.actor.mailbox-size-limit
Prometheus
akka_actor_mailbox_size_limit
Elastic
actors.mailbox-size-limit
Config
cinnamon.akka.meta.descriptor {
  actor {
    mailbox-size-limit {
      key = "mailbox-size-limit"
    }
  }
}
Mailbox time Recorder statistics for the time that messages are in the mailbox
OpenTelemetry
akka.actor.mailbox-time
Prometheus
akka_actor_mailbox_time_ns
Elastic
actors.mailbox-time
Config
cinnamon.akka.meta.descriptor {
  actor {
    mailbox-time {
      key = "mailbox-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
MailboxTimeLimit Event message has been in the mailbox for too long. Event information: actor-ref — the actor whose mailbox time has exceeded the limit message — the message being dequeued from the mailbox nanos — the mailbox time threshold-nanos — the mailbox time limit
OpenTelemetry
akka.actor.mailbox-time-limit
Prometheus
akka_actor_mailbox_time_limit
Elastic
actors.mailbox-time-limit
Config
cinnamon.akka.meta.descriptor {
  actor {
    mailbox-time-limit {
      key = "mailbox-time-limit"
    }
  }
}
Processed messages Rate the number of messages that actors have processed in the selected time frame
OpenTelemetry
akka.message.processed-messages
Prometheus
akka_message_processed_messages
Elastic
messages.processed-messages
Config
cinnamon.akka.meta.descriptor {
  actor {
    processed-messages {
      key = "processed-messages"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Processing time Recorder statistics for the processing time of actors
OpenTelemetry
akka.message.processing-time
Prometheus
akka_message_processing_time_ns
Elastic
messages.processing-time
Config
cinnamon.akka.meta.descriptor {
  actor {
    processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
ProcessingTimeLimit Event message processing takes too long. Event information: actor-ref — the actor whose processing time has exceeded the limit message — the message that was just processed nanos — the processing time threshold-nanos — the processing time limit
OpenTelemetry
akka.actor.processing-time-limit
Prometheus
akka_actor_processing_time_limit
Elastic
actors.processing-time-limit
Config
cinnamon.akka.meta.descriptor {
  actor {
    processing-time-limit {
      key = "processing-time-limit"
    }
  }
}
Remote deserialization time Recorder statistics for the time that deserialization takes
OpenTelemetry
akka.message.remote-deserialization-time
Prometheus
akka_message_remote_deserialization_time_ns
Elastic
messages.remote-deserialization-time
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-deserialization-time {
      key = "remote-deserialization-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Remote large message received Event a message larger than the threshold has been received Event information: actor-ref — the actor who is receiving the large message message-class — the message class of the large message size — the size in bytes of the large message sender — the sender of the large message
OpenTelemetry
akka.actor.remote-large-message-received
Prometheus
akka_actor_remote_large_message_received
Elastic
actors.remote-large-message-received
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-large-message-received {
      key = "remote-large-message-received"
    }
  }
}
Remote large message sent Event a message larger than the threshold has been sent Event information: actor-ref — the actor who is sending the large message message-class — the message class of the large message size — the size in bytes of the large message recipient — the recipient of the large message
OpenTelemetry
akka.actor.remote-large-message-sent
Prometheus
akka_actor_remote_large_message_sent
Elastic
actors.remote-large-message-sent
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-large-message-sent {
      key = "remote-large-message-sent"
    }
  }
}
Remote received message size Recorder statistics for remote received message sizes
OpenTelemetry
akka.message.remote-received-message-size
Prometheus
akka_message_remote_received_message_size
Elastic
messages.remote-received-message-size
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-received-message-size {
      key = "remote-received-message-size"
      unit-type = "custom"
      unit-suffix = "byte"
    }
  }
}
Remote received messages Rate statistics for the number received remote messages
OpenTelemetry
akka.message.remote-received-messages
Prometheus
akka_message_remote_received_messages
Elastic
messages.remote-received-messages
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-received-messages {
      key = "remote-received-messages"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Remote sent message size Recorder statistics for remote sent message sizes
OpenTelemetry
akka.message.remote-sent-message-size
Prometheus
akka_message_remote_sent_message_size
Elastic
messages.remote-sent-message-size
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-sent-message-size {
      key = "remote-sent-message-size"
      unit-type = "custom"
      unit-suffix = "byte"
    }
  }
}
Remote sent messages Rate statistics for the number sent remote messages
OpenTelemetry
akka.message.remote-sent-messages
Prometheus
akka_message_remote_sent_messages
Elastic
messages.remote-sent-messages
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-sent-messages {
      key = "remote-sent-messages"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Remote serialization time Recorder statistics for the time that serialization takes
OpenTelemetry
akka.message.remote-serialization-time
Prometheus
akka_message_remote_serialization_time_ns
Elastic
messages.remote-serialization-time
Config
cinnamon.akka.meta.descriptor {
  actor {
    remote-serialization-time {
      key = "remote-serialization-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Actors (running) Counter the number of running actors (of an actor class or group)
OpenTelemetry
akka.actor.running-actors
Prometheus
akka_actor_running_actors
Elastic
actors.running-actors
Config
cinnamon.akka.meta.descriptor {
  actor {
    running-actors {
      key = "running-actors"
      unit-type = "custom"
      unit-suffix = "actor"
    }
  }
}
Sent messages Rate statistics for the number of sent messages per actor
OpenTelemetry
akka.message.sent-messages
Prometheus
akka_message_sent_messages
Elastic
messages.sent-messages
Config
cinnamon.akka.meta.descriptor {
  actor {
    sent-messages {
      key = "sent-messages"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Stash size Recorder statistics for actor stash sizes
OpenTelemetry
akka.actor.stash-size
Prometheus
akka_actor_stash_size
Elastic
actors.stash-size
Config
cinnamon.akka.meta.descriptor {
  actor {
    stash-size {
      key = "stash-size"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
StashSizeLimit Event stash queue grows too large. Event information: actor-ref — the actor whose stash size has exceeded the limit message — the message being stashed size — the stash size limit — the stash size limit
OpenTelemetry
akka.actor.stash-size-limit
Prometheus
akka_actor_stash_size_limit
Elastic
actors.stash-size-limit
Config
cinnamon.akka.meta.descriptor {
  actor {
    stash-size-limit {
      key = "stash-size-limit"
    }
  }
}
UnhandledMessage Event when an actor does not handle a message sent to it. Event information: actor-ref — the actor not handling the message message — the message not being handled sender — the sender of the message
OpenTelemetry
akka.actor.unhandled-message
Prometheus
akka_actor_unhandled_message
Elastic
actors.unhandled-message
Config
cinnamon.akka.meta.descriptor {
  actor {
    unhandled-message {
      key = "unhandled-message"
    }
  }
}

Akka Circuit-Breaker

Metric Type Description
CircuitBreakerStateChange Event state change event: 0 - none, 1 - open, 2 - half-open, 3 - close
OpenTelemetry
akka.circuit-breaker.circuit-breaker-state-change
Prometheus
akka_circuit_breaker_circuit_breaker_state_change
Elastic
circuit-breakers.circuit-breaker-state-change
Config
cinnamon.akka.meta.descriptor {
  circuit-breaker {
    state-change {
      key = "circuit-breaker-state-change"
    }
  }
}
Failure Rate the rate of failed calls
OpenTelemetry
akka.circuit-breaker.failure
Prometheus
akka_circuit_breaker_failure
Elastic
circuit-breakers.failure
Config
cinnamon.akka.meta.descriptor {
  circuit-breaker {
    failure {
      key = "failure"
      unit-type = "custom"
      unit-suffix = "request"
    }
  }
}
Latency Recorder the elapsed duration of the call in nanoseconds
OpenTelemetry
akka.circuit-breaker.latency
Prometheus
akka_circuit_breaker_latency_ns
Elastic
circuit-breakers.latency
Config
cinnamon.akka.meta.descriptor {
  circuit-breaker {
    latency {
      key = "latency"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
State GaugeLong current circuit-breaker state: 0 - none, 1 - open, 2 - half-open, 3 - close
OpenTelemetry
akka.circuit-breaker.state
Prometheus
akka_circuit_breaker_state
Elastic
circuit-breakers.state
Config
cinnamon.akka.meta.descriptor {
  circuit-breaker {
    state {
      key = "state"
    }
  }
}
Success Rate the rate of successful calls
OpenTelemetry
akka.circuit-breaker.success
Prometheus
akka_circuit_breaker_success
Elastic
circuit-breakers.success
Config
cinnamon.akka.meta.descriptor {
  circuit-breaker {
    success {
      key = "success"
      unit-type = "custom"
      unit-suffix = "request"
    }
  }
}
Throuphput Rate the total rate of calls
OpenTelemetry
akka.circuit-breaker.throughput
Prometheus
akka_circuit_breaker_throughput
Elastic
circuit-breakers.throughput
Config
cinnamon.akka.meta.descriptor {
  circuit-breaker {
    throughput {
      key = "throughput"
      unit-type = "custom"
      unit-suffix = "request"
    }
  }
}

Akka Cluster

Metric Type Description
Cluster domain event Event cluster domain events like leader changed, role leader changed or cluster shutting down. Event information: event — type of domain event, e.g. LeaderChanged role — the role for the domain event if any
OpenTelemetry
remote-node.cluster-domain-event
Prometheus
remote_node_cluster_domain_event
Elastic
remote-nodes.cluster-domain-event
Config
cinnamon.akka.meta.descriptor {
  node {
    cluster-domain-event {
      key = "cluster-domain-event"
    }
  }
}
Cluster member event Event cluster member events like member up, unreachable, reachable, exited or removed. Event information: event — type of domain event, e.g. MemberUp version — node version member-status — member status, e.g. Joining previous-member-status — previous member status, e.g. Up
OpenTelemetry
remote-node.cluster-member-event
Prometheus
remote_node_cluster_member_event
Elastic
remote-nodes.cluster-member-event
Config
cinnamon.akka.meta.descriptor {
  node {
    cluster-member-event {
      key = "cluster-member-event"
    }
  }
}
Node quarantined event Event node quarantine event information
OpenTelemetry
remote-node.node-quarantined-event
Prometheus
remote_node_node_quarantined_event
Elastic
remote-nodes.node-quarantined-event
Config
cinnamon.akka.meta.descriptor {
  node {
    node-quarantined-event {
      key = "node-quarantined-event"
    }
  }
}
Phi accrual value GaugeDouble statistics for the Phi accrual failure detector. A Phi value represents the connection between two nodes; self and remote. A self node can have a connection to any number of remote nodes and each connection will have its own Phi value. Note that internally in Akka the Phi accrual value can become Double.Infinity. If this happens Cinnamon will convert this value to 1024*1024. The reason for this is that most visualizers cannot handle infinity. If you therefore see the value 1048576 (1024*1024) this means that the Phi value has reached infinity.
OpenTelemetry
remote-node.phi-accrual-value
Prometheus
remote_node_phi_accrual_value
Elastic
remote-nodes.phi-accrual-value
Config
cinnamon.akka.meta.descriptor {
  node {
    phi-accrual-value {
      key = "phi-accrual-value"
    }
  }
}
Cluster current event Event a one time event, per cluster node, containing information about the state of the cluster
OpenTelemetry
self-node.cluster-current-event
Prometheus
self_node_cluster_current_event
Elastic
self-nodes.cluster-current-event
Config
cinnamon.akka.meta.descriptor {
  node {
    cluster-current-event {
      key = "cluster-current-event"
    }
  }
}
Cluster node status Event if cinnamon.akka.cluster.node-status is enabled one event per cluster member with the node status as seen from the emitting node. The events are emitted once on joining the cluster and then periodically. Interval for emitting is configured using cinnamon.akka.cluster.node-status-interval, default value is an interval of 30 seconds.
OpenTelemetry
self-node.cluster-node-status
Prometheus
self_node_cluster_node_status
Elastic
self-nodes.cluster-node-status
Config
cinnamon.akka.meta.descriptor {
  node {
    cluster-node-status {
      key = "cluster-node-status"
    }
  }
}
Cluster singleton event Event cluster singleton events with information about node, actor singleton class and name. Event information: event — type of singleton event, e.g. started class — singleton actor class actor — singleton actor name
OpenTelemetry
self-node.cluster-singleton-event
Prometheus
self_node_cluster_singleton_event
Elastic
self-nodes.cluster-singleton-event
Config
cinnamon.akka.meta.descriptor {
  node {
    cluster-singleton-event {
      key = "cluster-singleton-event"
    }
  }
}
Node unable to join event Event cluster node unable to join event with information about seed nodes and number of attempts to join
OpenTelemetry
self-node.node-unable-to-join-event
Prometheus
self_node_node_unable_to_join_event
Elastic
self-nodes.node-unable-to-join-event
Config
cinnamon.akka.meta.descriptor {
  node {
    node-unable-to-join-event {
      key = "node-unable-to-join-event"
    }
  }
}
Phi accrual thresdhold value GaugeDouble the configured Phi accrual threshold value
OpenTelemetry
self-node.phi-accrual-threshold-value
Prometheus
self_node_phi_accrual_threshold_value
Elastic
self-nodes.phi-accrual-threshold-value
Config
cinnamon.akka.meta.descriptor {
  node {
    phi-accrual-threshold-value {
      key = "phi-accrual-threshold-value"
    }
  }
}
Reachable nodes Counter number of reachable nodes in the cluster at any point in time
OpenTelemetry
self-node.reachable-nodes
Prometheus
self_node_reachable_nodes
Elastic
self-nodes.reachable-nodes
Config
cinnamon.akka.meta.descriptor {
  node {
    reachable-nodes {
      key = "reachable-nodes"
      unit-type = "custom"
      unit-suffix = "node"
    }
  }
}
Shard region count value GaugeLong number of shards regions per node
OpenTelemetry
self-node.shard-region-count-value
Prometheus
self_node_shard_region_count_value
Elastic
self-nodes.shard-region-count-value
Config
cinnamon.akka.meta.descriptor {
  shard {
    shard-region-count-value {
      key = "shard-region-count-value"
      unit-type = "custom"
    }
  }
}
Shard region event Event cluster shard region started/stopped event with information about actor, node, type name and type of shard region. Event information: event — type of shard region event, e.g. STARTED shard-region-actor — the actor controlling the shard region type-name — the entity type name of the shard region
OpenTelemetry
self-node.shard-region-event
Prometheus
self_node_shard_region_event
Elastic
self-nodes.shard-region-event
Config
cinnamon.akka.meta.descriptor {
  shard {
    shard-region-event {
      key = "shard-region-event"
    }
  }
}
Shard region proxy event Event cluster shard region started/stopped event with information about actor, node, type name and type of shard region proxy. Event information: event — type of shard region event, e.g. STARTED shard-region-actor — the actor controlling the shard region type-name — the entity type name of the shard region
OpenTelemetry
self-node.shard-region-proxy-event
Prometheus
self_node_shard_region_proxy_event
Elastic
self-nodes.shard-region-proxy-event
Config
cinnamon.akka.meta.descriptor {
  shard {
    shard-region-proxy-event {
      key = "shard-region-proxy-event"
    }
  }
}
Split brain resolver event Event Split brain resolver decision taken event with information about the decision, reachable and unreachable nodes. Event information: decision — type of SBR decision, e.g. DownUnreachable nodes — the nodes that this node finds reachable unreachable-nodes — the nodes that this node finds unreachable
OpenTelemetry
self-node.split-brain-resolver-event
Prometheus
self_node_split_brain_resolver_event
Elastic
self-nodes.split-brain-resolver-event
Config
cinnamon.akka.meta.descriptor {
  node {
    split-brain-resolver-event {
      key = "split-brain-resolver-event"
    }
  }
}
Unreachable nodes Counter number of unreachable nodes in the cluster at any point in time
OpenTelemetry
self-node.unreachable-nodes
Prometheus
self_node_unreachable_nodes
Elastic
self-nodes.unreachable-nodes
Config
cinnamon.akka.meta.descriptor {
  node {
    unreachable-nodes {
      key = "unreachable-nodes"
      unit-type = "custom"
      unit-suffix = "node"
    }
  }
}
Shard count value GaugeLong number of shards per shard region
OpenTelemetry
shard-region.shard-count-value
Prometheus
shard_region_shard_count_value
Elastic
shard-regions.shard-count-value
Config
cinnamon.akka.meta.descriptor {
  shard {
    shard-count-value {
      key = "shard-count-value"
      unit-type = "custom"
    }
  }
}
Shard region messages delivered Rate statistics for the number of messages that have been delivered by the shard region actor (regardless of where the shard resides)
OpenTelemetry
shard-region.shard-region-messages-delivered
Prometheus
shard_region_shard_region_messages_delivered
Elastic
shard-regions.shard-region-messages-delivered
Config
cinnamon.akka.meta.descriptor {
  shard {
    shard-region-messages-delivered {
      key = "shard-region-messages-delivered"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Shard entity count value GaugeLong number of shard entities per shard
OpenTelemetry
shard.shard-entity-count-value
Prometheus
shard_shard_entity_count_value
Elastic
shards.shard-entity-count-value
Config
cinnamon.akka.meta.descriptor {
  shard {
    shard-entity-count-value {
      key = "shard-entity-count-value"
      unit-type = "custom"
    }
  }
}

Akka Dispatcher (All time metrics)

Metric Type Description
Processing Counter how many tasks are being processed right now
OpenTelemetry
akka.dispatcher.processing
Prometheus
akka_dispatcher_processing
Elastic
dispatchers.processing
Config
cinnamon.akka.meta.descriptor {
  dispatcher {
    processing {
      key = "processing"
      unit-type = "custom"
      unit-suffix = "task"
    }
  }
}
Processing time Recorder statistics for how long the processing takes
OpenTelemetry
akka.dispatcher.processing-time
Prometheus
akka_dispatcher_processing_time_ns
Elastic
dispatchers.processing-time
Config
cinnamon.akka.meta.descriptor {
  dispatcher {
    processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Queue size Counter the number of tasks waiting to be processed
OpenTelemetry
akka.dispatcher.queue-size
Prometheus
akka_dispatcher_queue_size
Elastic
dispatchers.queue-size
Config
cinnamon.akka.meta.descriptor {
  dispatcher {
    queue-size {
      key = "queue-size"
      unit-type = "custom"
      unit-suffix = "task"
    }
  }
}
Queue time Recorder statistics for how long tasks are in the queue
OpenTelemetry
akka.dispatcher.queue-time
Prometheus
akka_dispatcher_queue_time_ns
Elastic
dispatchers.queue-time
Config
cinnamon.akka.meta.descriptor {
  dispatcher {
    queue-time {
      key = "queue-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.

Akka Dispatcher (Fork-Join)

Metric Type Description
Active thread count ProvidingGaugeLong an estimate of the number of threads running or stealing tasks
OpenTelemetry
akka.dispatcher.active-threads
Prometheus
akka_dispatcher_active_threads
Elastic
dispatchers.active-threads
Config
cinnamon.core.meta.descriptor {
  executor {
    active-thread-count {
      key = "active-threads"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Parallelism ProvidingGaugeLong the parallelism setting
OpenTelemetry
akka.dispatcher.parallelism
Prometheus
akka_dispatcher_parallelism
Elastic
dispatchers.parallelism
Config
cinnamon.core.meta.descriptor {
  executor {
    parallelism {
      key = "parallelism"
    }
  }
}
Pool size ProvidingGaugeLong the current size of the thread pool
OpenTelemetry
akka.dispatcher.pool-size
Prometheus
akka_dispatcher_pool_size
Elastic
dispatchers.pool-size
Config
cinnamon.core.meta.descriptor {
  executor {
    pool-size {
      key = "pool-size"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Queued task count ProvidingGaugeLong an estimate of the total number of tasks currently in queues
OpenTelemetry
akka.dispatcher.queued-tasks
Prometheus
akka_dispatcher_queued_tasks
Elastic
dispatchers.queued-tasks
Config
cinnamon.core.meta.descriptor {
  executor {
    queued-task-count {
      key = "queued-tasks"
      unit-type = "custom"
      unit-suffix = "task"
    }
  }
}
Runnning thread count ProvidingGaugeLong an estimate of the number of threads not blocked in managed synchronization
OpenTelemetry
akka.dispatcher.running-threads
Prometheus
akka_dispatcher_running_threads
Elastic
dispatchers.running-threads
Config
cinnamon.core.meta.descriptor {
  executor {
    running-thread-count {
      key = "running-threads"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}

Akka Dispatcher (Pinned)

Metric Type Description
Active thread count GaugeLong an estimate of the number of threads running tasks
OpenTelemetry
akka.dispatcher.active-threads
Prometheus
akka_dispatcher_active_threads
Elastic
dispatchers.active-threads
Config
cinnamon.core.meta.descriptor {
  executor {
    active-thread-count {
      key = "active-threads"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Pool size GaugeLong the current size of the pinned thread pool
OpenTelemetry
akka.dispatcher.pool-size
Prometheus
akka_dispatcher_pool_size
Elastic
dispatchers.pool-size
Config
cinnamon.core.meta.descriptor {
  executor {
    pool-size {
      key = "pool-size"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Runnning thread count GaugeLong an estimate of the total number of currently runnning threads
OpenTelemetry
akka.dispatcher.running-threads
Prometheus
akka_dispatcher_running_threads
Elastic
dispatchers.running-threads
Config
cinnamon.core.meta.descriptor {
  executor {
    running-thread-count {
      key = "running-threads"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}

Akka Dispatcher (Thread-Pool)

Metric Type Description
Active thread count ProvidingGaugeLong an estimate of the number of threads running tasks
OpenTelemetry
akka.dispatcher.active-threads
Prometheus
akka_dispatcher_active_threads
Elastic
dispatchers.active-threads
Config
cinnamon.core.meta.descriptor {
  executor {
    active-thread-count {
      key = "active-threads"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Core pool size ProvidingGaugeLong the minimum size of the thread pool
OpenTelemetry
akka.dispatcher.core-pool-size
Prometheus
akka_dispatcher_core_pool_size
Elastic
dispatchers.core-pool-size
Config
cinnamon.core.meta.descriptor {
  executor {
    core-pool-size {
      key = "core-pool-size"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Max pool size ProvidingGaugeLong the maximum size of the thread pool
OpenTelemetry
akka.dispatcher.max-pool-size
Prometheus
akka_dispatcher_max_pool_size
Elastic
dispatchers.max-pool-size
Config
cinnamon.core.meta.descriptor {
  executor {
    max-pool-size {
      key = "max-pool-size"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Pool size ProvidingGaugeLong the current size of the thread pool
OpenTelemetry
akka.dispatcher.pool-size
Prometheus
akka_dispatcher_pool_size
Elastic
dispatchers.pool-size
Config
cinnamon.core.meta.descriptor {
  executor {
    pool-size {
      key = "pool-size"
      unit-type = "custom"
      unit-suffix = "thread"
    }
  }
}
Processed task count ProvidingGaugeLong an estimate of the number of processed tasks
OpenTelemetry
akka.dispatcher.processed-tasks
Prometheus
akka_dispatcher_processed_tasks
Elastic
dispatchers.processed-tasks
Config
cinnamon.core.meta.descriptor {
  executor {
    processed-task-count {
      key = "processed-tasks"
      unit-type = "custom"
      unit-suffix = "task"
    }
  }
}

Akka Http Client

Metric Type Description
Akka HTTP client connections Counter the number of connections used in a pool
OpenTelemetry
akka-http.client-pool.connections
Prometheus
akka_http_client_pool_connections
Elastic
client-pools.connections
Config
cinnamon.akka.http.meta.descriptor {
  client-pool {
    connections {
      key = "connections"
      unit-type = "custom"
      unit-suffix = "connection"
    }
  }
}
Akka HTTP client requests Rate the rate of requests from a client to a service
OpenTelemetry
akka-http.http-client.http-client-requests
Prometheus
akka_http_http_client_http_client_requests
Elastic
http-clients.http-client-requests
Config
cinnamon.akka.http.meta.descriptor {
  client {
    requests {
      key = "http-client-requests"
      unit-type = "custom"
    }
  }
}
Akka HTTP client service response time Recorder the time for responses from a service
OpenTelemetry
akka-http.http-client.http-client-service-response-time
Prometheus
akka_http_http_client_http_client_service_response_time_ns
Elastic
http-clients.http-client-service-response-time
Config
cinnamon.akka.http.meta.descriptor {
  client {
    service-response-time {
      key = "http-client-service-response-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.

Akka Http Server

Metric Type Description
Outgoing endpoint response time Recorder the time for responses per endpoint
OpenTelemetry
akka-http.request-path.endpoint-response-time
Prometheus
akka_http_request_path_endpoint_response_time_ns
Elastic
request-paths.endpoint-response-time
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    response-time {
      key = "endpoint-response-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 2xx Recorder the time for endpoint responses with 200-299 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-response-time-2xx
Prometheus
akka_http_request_path_endpoint_response_time_2xx_ns
Elastic
request-paths.endpoint-response-time-2xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    response-time-2xx {
      key = "endpoint-response-time-2xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 3xx Recorder the time for endpoint responses with 300-399 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-response-time-3xx
Prometheus
akka_http_request_path_endpoint_response_time_3xx_ns
Elastic
request-paths.endpoint-response-time-3xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    response-time-3xx {
      key = "endpoint-response-time-3xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 4xx Recorder the time for endpoint responses with 400-499 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-response-time-4xx
Prometheus
akka_http_request_path_endpoint_response_time_4xx_ns
Elastic
request-paths.endpoint-response-time-4xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    response-time-4xx {
      key = "endpoint-response-time-4xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint response time 5xx Recorder the time for endpoint responses with 500-599 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-response-time-5xx
Prometheus
akka_http_request_path_endpoint_response_time_5xx_ns
Elastic
request-paths.endpoint-response-time-5xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    response-time-5xx {
      key = "endpoint-response-time-5xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing endpoint responses Rate the rate of responses per endpoint
OpenTelemetry
akka-http.request-path.endpoint-responses
Prometheus
akka_http_request_path_endpoint_responses
Elastic
request-paths.endpoint-responses
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    responses {
      key = "endpoint-responses"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing endpoint responses 2xx Rate the rate of endpoint responses with 200-299 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-responses-2xx
Prometheus
akka_http_request_path_endpoint_responses_2xx
Elastic
request-paths.endpoint-responses-2xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    responses-2xx {
      key = "endpoint-responses-2xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing endpoint responses 3xx Rate the rate of endpoint responses with 300-399 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-responses-3xx
Prometheus
akka_http_request_path_endpoint_responses_3xx
Elastic
request-paths.endpoint-responses-3xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    responses-3xx {
      key = "endpoint-responses-3xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing endpoint responses 4xx Rate the rate of endpoint responses with 400-499 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-responses-4xx
Prometheus
akka_http_request_path_endpoint_responses_4xx
Elastic
request-paths.endpoint-responses-4xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    responses-4xx {
      key = "endpoint-responses-4xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing endpoint responses 5xx Rate the rate of endpoint responses with 500-599 status codes (optional)
OpenTelemetry
akka-http.request-path.endpoint-responses-5xx
Prometheus
akka_http_request_path_endpoint_responses_5xx
Elastic
request-paths.endpoint-responses-5xx
Config
cinnamon.akka.http.meta.descriptor {
  endpoint {
    responses-5xx {
      key = "endpoint-responses-5xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Incoming connections Rate the rate of connections per server
OpenTelemetry
akka-http.http-server.connections
Prometheus
akka_http_http_server_connections
Elastic
http-servers.connections
Config
cinnamon.akka.http.meta.descriptor {
  server {
    connections {
      key = "connections"
      unit-type = "custom"
      unit-suffix = "connection"
    }
  }
}
Incoming requests Rate the rate of requests per server
OpenTelemetry
akka-http.http-server.requests
Prometheus
akka_http_http_server_requests
Elastic
http-servers.requests
Config
cinnamon.akka.http.meta.descriptor {
  server {
    requests {
      key = "requests"
      unit-type = "custom"
      unit-suffix = "request"
    }
  }
}
Outgoing response time Recorder the time for responses per server
OpenTelemetry
akka-http.http-server.response-time
Prometheus
akka_http_http_server_response_time_ns
Elastic
http-servers.response-time
Config
cinnamon.akka.http.meta.descriptor {
  server {
    response-time {
      key = "response-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 2xx Recorder the time for responses with 200-299 status codes per server
OpenTelemetry
akka-http.http-server.response-time-2xx
Prometheus
akka_http_http_server_response_time_2xx_ns
Elastic
http-servers.response-time-2xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    response-time-2xx {
      key = "response-time-2xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 3xx Recorder the time for responses with 300-399 status codes per server
OpenTelemetry
akka-http.http-server.response-time-3xx
Prometheus
akka_http_http_server_response_time_3xx_ns
Elastic
http-servers.response-time-3xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    response-time-3xx {
      key = "response-time-3xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 4xx Recorder the time for responses with 400-499 status codes per server
OpenTelemetry
akka-http.http-server.response-time-4xx
Prometheus
akka_http_http_server_response_time_4xx_ns
Elastic
http-servers.response-time-4xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    response-time-4xx {
      key = "response-time-4xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing response time 5xx Recorder the time for responses with 500-599 status codes per server
OpenTelemetry
akka-http.http-server.response-time-5xx
Prometheus
akka_http_http_server_response_time_5xx_ns
Elastic
http-servers.response-time-5xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    response-time-5xx {
      key = "response-time-5xx"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Outgoing responses Rate the rate of responses per server
OpenTelemetry
akka-http.http-server.responses
Prometheus
akka_http_http_server_responses
Elastic
http-servers.responses
Config
cinnamon.akka.http.meta.descriptor {
  server {
    responses {
      key = "responses"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing responses 2xx Rate the rate of responses with 200-299 status codes per server
OpenTelemetry
akka-http.http-server.responses-2xx
Prometheus
akka_http_http_server_responses_2xx
Elastic
http-servers.responses-2xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    responses-2xx {
      key = "responses-2xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing responses 3xx Rate the rate of responses with 300-399 status codes per server
OpenTelemetry
akka-http.http-server.responses-3xx
Prometheus
akka_http_http_server_responses_3xx
Elastic
http-servers.responses-3xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    responses-3xx {
      key = "responses-3xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing responses 4xx Rate the rate of responses with 400-499 status codes per server
OpenTelemetry
akka-http.http-server.responses-4xx
Prometheus
akka_http_http_server_responses_4xx
Elastic
http-servers.responses-4xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    responses-4xx {
      key = "responses-4xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}
Outgoing responses 5xx Rate the rate of responses with 500-599 status codes per server
OpenTelemetry
akka-http.http-server.responses-5xx
Prometheus
akka_http_http_server_responses_5xx
Elastic
http-servers.responses-5xx
Config
cinnamon.akka.http.meta.descriptor {
  server {
    responses-5xx {
      key = "responses-5xx"
      unit-type = "custom"
      unit-suffix = "response"
    }
  }
}

Akka Persistence (Classic / Typed)

Metric Type Description
Active entities Counter number of active entities (in-memory, excludes passivated entities)
OpenTelemetry
akka.entity.active-entities
Prometheus
akka_entity_active_entities
Elastic
entities.active-entities
Config
cinnamon.akka.persistence.meta {
  descriptor {
    active-entities {
      key = "active-entities"
    }
  }
}
Command in-stash time Recorder time that commands spend in the stash. See Akka Persistence internal stash for the details
OpenTelemetry
akka.command.in-stash-time
Prometheus
akka_command_in_stash_time_ns
Elastic
commands.in-stash-time
Config
cinnamon.akka.persistence.meta {
  descriptor {
    command-in-stash-time {
      key = "in-stash-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
RecoveryPermitter max permits GaugeLong max number of permits (set via Akka configuration)
OpenTelemetry
akka.recovery-permitter.max-permits
Prometheus
akka_recovery_permitter_max_permits
Elastic
recovery-permitter.max-permits
Config
cinnamon.akka.persistence.meta {
  descriptor {
    max-permits {
      key = "max-permits"
      unit-type = "custom"
      unit-suffix = "permit"
    }
  }
}
RecoveryPermitter pending entities Recorder number of entities waiting for a permit to start recovery
OpenTelemetry
akka.recovery-permitter.pending-entities
Prometheus
akka_recovery_permitter_pending_entities
Elastic
recovery-permitter.pending-entities
Config
cinnamon.akka.persistence.meta {
  descriptor {
    pending-entities {
      key = "pending-entities"
      unit-type = "custom"
      unit-suffix = "entity"
    }
  }
}
Persist failed Event Event created whenever a message persist fails. Event information: actor-ref — the actor failing to persist an event failure — the exception that occurred event — the event that failed if available sequence-number — the event sequence number
OpenTelemetry
akka.entity.persist-failed
Prometheus
akka_entity_persist_failed
Elastic
entities.persist-failed
Config
cinnamon.akka.persistence.meta {
  descriptor {
    persist-failed {
      key = "persist-failed"
    }
  }
}
Persist rejected Event Event created whenever a message persist is rejected. Event information: actor-ref — the actor failing to persist an event failure — the exception that occurred event — the event that failed if available sequence-number — the event sequence number
OpenTelemetry
akka.entity.persist-rejected
Prometheus
akka_entity_persist_rejected
Elastic
entities.persist-rejected
Config
cinnamon.akka.persistence.meta {
  descriptor {
    persist-rejected {
      key = "persist-rejected"
    }
  }
}
Persisted events Rate number of persisted events acknowledged by the journal
OpenTelemetry
akka.event.persisted-events
Prometheus
akka_event_persisted_events
Elastic
events.persisted-events
Config
cinnamon.akka.persistence.meta {
  descriptor {
    persisted-events {
      key = "persisted-events"
      unit-type = "custom"
      unit-suffix = "event"
    }
  }
}
Event persistence time Recorder time from the moment the persist/persistAll method was called to the moment when it was acknowledged by the journal
OpenTelemetry
akka.event.persistence-time
Prometheus
akka_event_persistence_time_ns
Elastic
events.persistence-time
Config
cinnamon.akka.persistence.meta {
  descriptor {
    event-persistence-time {
      key = "persistence-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Command processing time Recorder time for processing the command excluding time spent in the mailbox or stash
OpenTelemetry
akka.command.processing-time
Prometheus
akka_command_processing_time_ns
Elastic
commands.processing-time
Config
cinnamon.akka.persistence.meta {
  descriptor {
    command-processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Received commands Rate number of received commands
OpenTelemetry
akka.command.received-commands
Prometheus
akka_command_received_commands
Elastic
commands.received-commands
Config
cinnamon.akka.persistence.meta {
  descriptor {
    received-commands {
      key = "received-commands"
      unit-type = "custom"
      unit-suffix = "command"
    }
  }
}
Recovery failed Event Event created whenever a message replay fails. Event information: actor-ref — the actor trying to recover failure — the exception that occurred event — the event that failed if available recovery-failure-time — the time in nanoseconds
OpenTelemetry
akka.entity.recovery-failed
Prometheus
akka_entity_recovery_failed
Elastic
entities.recovery-failed
Config
cinnamon.akka.persistence.meta {
  descriptor {
    recovery-failed {
      key = "recovery-failed"
    }
  }
}
Recovery failure time Recorder failure recovery time for a persistent entity (in case of failure)
OpenTelemetry
akka.entity.recovery-failure-time
Prometheus
akka_entity_recovery_failure_time_ns
Elastic
entities.recovery-failure-time
Config
cinnamon.akka.persistence.meta {
  descriptor {
    recovery-failure-time {
      key = "recovery-failure-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Recovery time Recorder total time for recovery of a persistent entity (includes waiting for a recovery permit, and fetching and replaying events and snapshots from the journal)
OpenTelemetry
akka.entity.recovery-time
Prometheus
akka_entity_recovery_time_ns
Elastic
entities.recovery-time
Config
cinnamon.akka.persistence.meta {
  descriptor {
    recovery-time {
      key = "recovery-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
RecoveryPermitter used permits Recorder number of used permits
OpenTelemetry
akka.recovery-permitter.used-permits
Prometheus
akka_recovery_permitter_used_permits
Elastic
recovery-permitter.used-permits
Config
cinnamon.akka.persistence.meta {
  descriptor {
    used-permits {
      key = "used-permits"
      unit-type = "custom"
      unit-suffix = "permit"
    }
  }
}

Akka Projection

Metric Type Description
Handler Error Rate error rate during the envelope processing. If you use a handler strategy including a retry operation this number will increase
OpenTelemetry
akka-projection.projection.handler-error
Prometheus
akka_projection_projection_handler_error
Elastic
projections.handler-error
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    handler-error {
      key = "handler-error"
    }
  }
}
Handler Error Warning Event handler error event
OpenTelemetry
akka-projection.projection.handler-error-event
Prometheus
akka_projection_projection_handler_error_event
Elastic
projections.handler-error-event
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    handler-error-event {
      key = "handler-error-event"
    }
  }
}
Handler Success Counter number of processed envelopes or number of times an envelope handler ran successfully
OpenTelemetry
akka-projection.projection.handler-success
Prometheus
akka_projection_projection_handler_success
Elastic
projections.handler-success
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    handler-success {
      key = "handler-success"
      unit-type = "custom"
      unit-suffix = "envelope"
    }
  }
}
Offset Success Counter number of envelopes marked as processed or number of offset positions advanced
OpenTelemetry
akka-projection.projection.offset-success
Prometheus
akka_projection_projection_offset_success
Elastic
projections.offset-success
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    offset-success {
      key = "offset-success"
      unit-type = "custom"
      unit-suffix = "envelope"
    }
  }
}
Projection Count Counter number of running projections. This is a counter of instances so if you run a sharded projection the counter will indicate the number of shards.
OpenTelemetry
akka-projection.projection.projection-count
Prometheus
akka_projection_projection_projection_count
Elastic
projections.projection-count
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    projection-count {
      key = "projection-count"
    }
  }
}
Projection Failure Rate failure rate for projections. Failures indicate the projection required restarting
OpenTelemetry
akka-projection.projection.projection-failure
Prometheus
akka_projection_projection_projection_failure
Elastic
projections.projection-failure
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    projection-failure {
      key = "projection-failure"
    }
  }
}
Projection Failure Warning Event projection failure event
OpenTelemetry
akka-projection.projection.projection-failure-event
Prometheus
akka_projection_projection_projection_failure_event
Elastic
projections.projection-failure-event
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    projection-failure-event {
      key = "projection-failure-event"
    }
  }
}
Residence Time Recorder time (in milliseconds) spent since the creation of the envelope until its offset was committed
OpenTelemetry
akka-projection.projection.residence-time
Prometheus
akka_projection_projection_residence_time
Elastic
projections.residence-time
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    residence-time {
      key = "residence-time"
    }
  }
}
Service Time Recorder time (in nanoseconds) from the instant the envelope was considered ready until the projected data was available for reading. Note this time doesn’t include the period until data being readable and the offset being committed. As a consequence, when using at-least-once semantics, this metric will be reported for every envelope handler completing successfully (even if storing the offset failed and the envelope is retried)
OpenTelemetry
akka-projection.projection.service-time
Prometheus
akka_projection_projection_service_time_ns
Elastic
projections.service-time
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    service-time {
      key = "service-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Wait Time Recorder time (in milliseconds) an envelope spent on the queue/broker/journal. This is the time since the envelope was created until it was read, deserialized, and considered ready to be processed by the projection. This metric depends on the type of source since some sources don’t provide the creation timestamp. At the same time, some producers record the creation time in the producing process, while others record the timestamp on the database/broker when accepting the envelope. Note this metric is affected by existing time skew between nodes on your infrastructure
OpenTelemetry
akka-projection.projection.wait-time
Prometheus
akka_projection_projection_wait_time
Elastic
projections.wait-time
Config
cinnamon.akka.projection.meta.descriptor {
  projection {
    wait-time {
      key = "wait-time"
    }
  }
}

Akka Router

Metric Type Description
Router processing time Recorder statistics for the processing time of the router logic
OpenTelemetry
akka.message.router-processing-time
Prometheus
akka_message_router_processing_time_ns
Elastic
messages.router-processing-time
Config
cinnamon.akka.meta.descriptor {
  router {
    router-processing-time {
      key = "router-processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Router routed messages Rate the number of messages that routers have processed in the selected time frame
OpenTelemetry
akka.message.router-routed-messages
Prometheus
akka_message_router_routed_messages
Elastic
messages.router-routed-messages
Config
cinnamon.akka.meta.descriptor {
  router {
    router-routed-messages {
      key = "router-routed-messages"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}

Akka Stream

Metric Type Description
Akka Stream boundary demand Rate the rate of demand on an internal async boundary (includes batched demand requests)
OpenTelemetry
akka-stream.boundary.demand
Prometheus
akka_stream_boundary_demand
Elastic
boundaries.demand
Config
cinnamon.akka.stream.meta.descriptor {
  boundary {
    demand {
      key = "demand"
      unit-type = "custom"
      unit-suffix = "element"
    }
  }
}
Akka Stream boundary demand processing time Recorder the time for processing demand requests on an internal async boundary
OpenTelemetry
akka-stream.boundary.demand-processing-time
Prometheus
akka_stream_boundary_demand_processing_time_ns
Elastic
boundaries.demand-processing-time
Config
cinnamon.akka.stream.meta.descriptor {
  boundary {
    demand-processing-time {
      key = "demand-processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream boundary processing time Recorder the time for processing elements pushed downstream on an internal async boundary
OpenTelemetry
akka-stream.boundary.processing-time
Prometheus
akka_stream_boundary_processing_time_ns
Elastic
boundaries.processing-time
Config
cinnamon.akka.stream.meta.descriptor {
  boundary {
    processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream boundary throughput Rate the rate of events that have been processed on an internal async boundary
OpenTelemetry
akka-stream.boundary.throughput
Prometheus
akka_stream_boundary_throughput
Elastic
boundaries.throughput
Config
cinnamon.akka.stream.meta.descriptor {
  boundary {
    throughput {
      key = "throughput"
      unit-type = "custom"
      unit-suffix = "element"
    }
  }
}
Akka Stream connection demand Rate the rate of demand on a connection
OpenTelemetry
akka-stream.connection.demand
Prometheus
akka_stream_connection_demand
Elastic
connections.demand
Config
cinnamon.akka.stream.meta.descriptor {
  connection {
    demand {
      key = "demand"
      unit-type = "custom"
      unit-suffix = "element"
    }
  }
}
Akka Stream connection demand latency Recorder the time spent waiting for demand from downstream (the time from providing an element downstream with a push to starting to process demand from a pull)
OpenTelemetry
akka-stream.connection.demand-latency
Prometheus
akka_stream_connection_demand_latency_ns
Elastic
connections.demand-latency
Config
cinnamon.akka.stream.meta.descriptor {
  connection {
    demand-latency {
      key = "demand-latency"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection demand processing time Recorder the time for processing demands on a connection
OpenTelemetry
akka-stream.connection.demand-processing-time
Prometheus
akka_stream_connection_demand_processing_time_ns
Elastic
connections.demand-processing-time
Config
cinnamon.akka.stream.meta.descriptor {
  connection {
    demand-processing-time {
      key = "demand-processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection latency Recorder the time spent waiting for elements from upstream (the time from demanding an element upstream with a pull to starting to process an element from a push)
OpenTelemetry
akka-stream.connection.latency
Prometheus
akka_stream_connection_latency_ns
Elastic
connections.latency
Config
cinnamon.akka.stream.meta.descriptor {
  connection {
    latency {
      key = "latency"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection processing time Recorder the time for processing elements pushed downstream on a connection
OpenTelemetry
akka-stream.connection.processing-time
Prometheus
akka_stream_connection_processing_time_ns
Elastic
connections.processing-time
Config
cinnamon.akka.stream.meta.descriptor {
  connection {
    processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream connection throughput Rate the rate of elements that have been processed on a connection
OpenTelemetry
akka-stream.connection.throughput
Prometheus
akka_stream_connection_throughput
Elastic
connections.throughput
Config
cinnamon.akka.stream.meta.descriptor {
  connection {
    throughput {
      key = "throughput"
      unit-type = "custom"
      unit-suffix = "element"
    }
  }
}
Akka Stream flow active time Recorder the active time during the flow, provided by the Stopwatch, and allowing flow efficiency to be measured
OpenTelemetry
akka-stream.stream.flow-active-time
Prometheus
akka_stream_stream_flow_active_time_ns
Elastic
streams.flow-active-time
Config
cinnamon.akka.stream.meta.descriptor {
  stream {
    flow-active-time {
      key = "flow-active-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream flow efficiency Recorder the efficiency of end-to-end flows, based on how much time was spent actively processing as compared with waiting. Recorded as a percentage using active time / total flow time
OpenTelemetry
akka-stream.stream.flow-efficiency
Prometheus
akka_stream_stream_flow_efficiency
Elastic
streams.flow-efficiency
Config
cinnamon.akka.stream.meta.descriptor {
  stream {
    flow-efficiency {
      key = "flow-efficiency"
      unit-type = "custom"
      unit-suffix = "percent"
    }
  }
}
Akka Stream end-to-end flow time Recorder the time for elements to flow through the stream from sources to sinks (uses a Stopwatch internally)
OpenTelemetry
akka-stream.stream.flow-time
Prometheus
akka_stream_stream_flow_time_ns
Elastic
streams.flow-time
Config
cinnamon.akka.stream.meta.descriptor {
  stream {
    flow-time {
      key = "flow-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator demand Rate the rate of demand on an operator
OpenTelemetry
akka-stream.operator.demand
Prometheus
akka_stream_operator_demand
Elastic
operators.demand
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    demand {
      key = "demand"
      unit-type = "custom"
      unit-suffix = "element"
    }
  }
}
Akka Stream operator demand latency Recorder the time spent waiting for demand from downstream (the time from providing an element downstream with a push to starting to process demand from a pull)
OpenTelemetry
akka-stream.operator.demand-latency
Prometheus
akka_stream_operator_demand_latency_ns
Elastic
operators.demand-latency
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    demand-latency {
      key = "demand-latency"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator demand processing time Recorder the time for processing demands on an operator
OpenTelemetry
akka-stream.operator.demand-processing-time
Prometheus
akka_stream_operator_demand_processing_time_ns
Elastic
operators.demand-processing-time
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    demand-processing-time {
      key = "demand-processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator flow time Recorder the time for elements to flow through the stream from sources to being completed by an operator, providing a progress metric for flows
OpenTelemetry
akka-stream.operator.flow-time
Prometheus
akka_stream_operator_flow_time_ns
Elastic
operators.flow-time
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    flow-time {
      key = "flow-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operator latency Recorder the time spent waiting for elements from upstream (the time from demanding an element upstream with a pull to starting to process an element from a push)
OpenTelemetry
akka-stream.operator.latency
Prometheus
akka_stream_operator_latency_ns
Elastic
operators.latency
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    latency {
      key = "latency"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
OperatorFailure Event event when an operator fails
OpenTelemetry
akka-stream.operator.operator-failure
Prometheus
akka_stream_operator_operator_failure
Elastic
operators.operator-failure
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    operator-failure {
      key = "operator-failure"
    }
  }
}
Akka Stream operator processing time Recorder the time for processing elements through an operator
OpenTelemetry
akka-stream.operator.processing-time
Prometheus
akka_stream_operator_processing_time_ns
Elastic
operators.processing-time
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Akka Stream operators (running) Counter the number of operator instances currently running (as identified by reporting configuration)
OpenTelemetry
akka-stream.operator.running-operators
Prometheus
akka_stream_operator_running_operators
Elastic
operators.running-operators
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    running-operators {
      key = "running-operators"
      unit-type = "custom"
      unit-suffix = "operator"
    }
  }
}
Akka Stream operator throughput Rate the rate of elements that have been processed by an operator
OpenTelemetry
akka-stream.operator.throughput
Prometheus
akka_stream_operator_throughput
Elastic
operators.throughput
Config
cinnamon.akka.stream.meta.descriptor {
  operator {
    throughput {
      key = "throughput"
      unit-type = "custom"
      unit-suffix = "element"
    }
  }
}
Akka Stream operators (running) Counter the total number of operators currently running per stream (as identified by reporting configuration)
OpenTelemetry
akka-stream.stream.running-operators
Prometheus
akka_stream_stream_running_operators
Elastic
streams.running-operators
Config
cinnamon.akka.stream.meta.descriptor {
  stream {
    running-operators {
      key = "running-operators"
      unit-type = "custom"
      unit-suffix = "operator"
    }
  }
}
Akka Streams (running) Counter the number of streams currently running (as identified by reporting configuration)
OpenTelemetry
akka-stream.stream.running-streams
Prometheus
akka_stream_stream_running_streams
Elastic
streams.running-streams
Config
cinnamon.akka.stream.meta.descriptor {
  stream {
    running-streams {
      key = "running-streams"
      unit-type = "custom"
      unit-suffix = "stream"
    }
  }
}
Akka Stream throughput Rate the rate of elements being processed by a stream as a whole (measured at the sinks)
OpenTelemetry
akka-stream.stream.throughput
Prometheus
akka_stream_stream_throughput
Elastic
streams.throughput
Config
cinnamon.akka.stream.meta.descriptor {
  stream {
    throughput {
      key = "throughput"
      unit-type = "custom"
      unit-suffix = "element"
    }
  }
}

Ask Pattern

Metric Type Description
Asked requests Rate the number of messages that actors have been asked in the selected time frame
OpenTelemetry
akka.message.asked-requests
Prometheus
akka_message_asked_requests
Elastic
messages.asked-requests
Config
cinnamon.akka.ask-pattern.meta.descriptor {
  ask-pattern {
    asked-requests {
      key = "asked-requests"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Expired request event Event when request expires before the response arrived Event information: actor-ref — the actor being asked the message message — the message being sent timeout — the given timeout
OpenTelemetry
akka.actor.expired-request-event
Prometheus
akka_actor_expired_request_event
Elastic
actors.expired-request-event
Config
cinnamon.akka.ask-pattern.meta.descriptor {
  ask-pattern {
    expired-request-event {
      key = "expired-request-event"
    }
  }
}
Expired requests Rate the number of requests that timed out in the selected time frame grouped by specified timeout values
OpenTelemetry
akka.message.expired-requests
Prometheus
akka_message_expired_requests
Elastic
messages.expired-requests
Config
cinnamon.akka.ask-pattern.meta.descriptor {
  ask-pattern {
    expired-requests {
      key = "expired-requests"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Expired requests Rate the number of requests that timed out in the selected time frame grouped by specified timeout values
OpenTelemetry
akka.timeout.expired-requests
Prometheus
akka_timeout_expired_requests
Elastic
timeouts.expired-requests
Config
cinnamon.akka.ask-pattern.meta.descriptor {
  ask-pattern {
    expired-requests-by-timeout {
      key = "expired-requests"
      unit-type = "custom"
      unit-suffix = "msg"
    }
  }
}
Failed request event Event when the actor responses with akka.actor.Status.Failure (only Akka Classic) Event information: actor-ref — the actor being asked the message message — the message being sent cause — the exception returned with a failure response
OpenTelemetry
akka.actor.failed-request-event
Prometheus
akka_actor_failed_request_event
Elastic
actors.failed-request-event
Config
cinnamon.akka.ask-pattern.meta.descriptor {
  ask-pattern {
    failed-request-event {
      key = "failed-request-event"
    }
  }
}
Late response time Recorder the time from the moment the message was sent till the moment when the answer was received, counting only responses received after timeout
OpenTelemetry
akka.message.late-response-time
Prometheus
akka_message_late_response_time_ns
Elastic
messages.late-response-time
Config
cinnamon.akka.ask-pattern.meta.descriptor {
  ask-pattern {
    late-response-time {
      key = "late-response-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Success response time Recorder the time from the moment the message was sent till the moment when the answer was received and before the specified timeout, counting only successful responses
OpenTelemetry
akka.message.success-response-time
Prometheus
akka_message_success_response_time_ns
Elastic
messages.success-response-time
Config
cinnamon.akka.ask-pattern.meta.descriptor {
  ask-pattern {
    success-response-time {
      key = "success-response-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.

Cinnamon Stopwatch

Metric Type Description
Stopwatch active time Recorder the time spent actively in actor message processing, stream operators, or future callbacks, as compared with the time in actor mailboxes or waiting to be scheduled
OpenTelemetry
stopwatch.stopwatch-active-time
Prometheus
stopwatch_stopwatch_active_time_ns
Elastic
stopwatches.stopwatch-active-time
Config
cinnamon.akka.meta.descriptor {
  stopwatch {
    stopwatch-active-time {
      key = "stopwatch-active-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Stopwatch rate Rate the rate of stopwatch runs
OpenTelemetry
stopwatch.stopwatch-rate
Prometheus
stopwatch_stopwatch_rate
Elastic
stopwatches.stopwatch-rate
Config
cinnamon.akka.meta.descriptor {
  stopwatch {
    stopwatch-rate {
      key = "stopwatch-rate"
      unit-type = "custom"
      unit-suffix = "event"
    }
  }
}
Stopwatch time Recorder the total elapsed time between the start and stop endpoints of a flow
OpenTelemetry
stopwatch.stopwatch-time
Prometheus
stopwatch_stopwatch_time_ns
Elastic
stopwatches.stopwatch-time
Config
cinnamon.akka.meta.descriptor {
  stopwatch {
    stopwatch-time {
      key = "stopwatch-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
StopwatchTimeLimit Event stopwatch interval threshold exceeded event with the limit and actual time in nanos
OpenTelemetry
stopwatch.stopwatch-time-limit
Prometheus
stopwatch_stopwatch_time_limit
Elastic
stopwatches.stopwatch-time-limit
Config
cinnamon.akka.meta.descriptor {
  stopwatch {
    stopwatch-time-limit {
      key = "stopwatch-time-limit"
    }
  }
}

Java Future

Metric Type Description
Java Future completing time Recorder the time from creation to completing the CompletableFuture
OpenTelemetry
java-future.completing-time
Prometheus
java_future_completing_time_ns
Elastic
java-futures.completing-time
Config
cinnamon.java.future.meta.descriptor {
  future {
    completing-time {
      key = "completing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Java Futures created Rate the rate of CompletableFutures being created (for a given name)
OpenTelemetry
java-future.created-futures
Prometheus
java_future_created_futures
Elastic
java-futures.created-futures
Config
cinnamon.java.future.meta.descriptor {
  future {
    created-futures {
      key = "created-futures"
      unit-type = "custom"
      unit-suffix = "future"
    }
  }
}
Java Futures processed Rate the rate of futures/callbacks being processed (marked when the future/callback has finished running)
OpenTelemetry
java-future.processed-futures
Prometheus
java_future_processed_futures
Elastic
java-futures.processed-futures
Config
cinnamon.java.future.meta.descriptor {
  future {
    processed-futures {
      key = "processed-futures"
      unit-type = "custom"
      unit-suffix = "future"
    }
  }
}
Java Future processing time Recorder the time that futures/callbacks take to run
OpenTelemetry
java-future.processing-time
Prometheus
java_future_processing_time_ns
Elastic
java-futures.processing-time
Config
cinnamon.java.future.meta.descriptor {
  future {
    processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Java Futures scheduled Recorder the number of futures currently scheduled (includes both future async sources and completion callbacks)
OpenTelemetry
java-future.scheduled-futures
Prometheus
java_future_scheduled_futures
Elastic
java-futures.scheduled-futures
Config
cinnamon.java.future.meta.descriptor {
  future {
    scheduled-futures {
      key = "scheduled-futures"
      unit-type = "custom"
      unit-suffix = "future"
    }
  }
}
Java Future scheduling time Recorder the time that futures/callbacks are in scheduling (in executor queues waiting to run)
OpenTelemetry
java-future.scheduling-time
Prometheus
java_future_scheduling_time_ns
Elastic
java-futures.scheduling-time
Config
cinnamon.java.future.meta.descriptor {
  future {
    scheduling-time {
      key = "scheduling-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.

Scala Future

Metric Type Description
Scala Future completing time Recorder the time from creation to completing the underlying Promise for this Future
OpenTelemetry
scala-future.completing-time
Prometheus
scala_future_completing_time_ns
Elastic
scala-futures.completing-time
Config
cinnamon.scala.future.meta.descriptor {
  future {
    completing-time {
      key = "completing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Scala Futures created Rate the rate of Futures being created (for a given name)
OpenTelemetry
scala-future.created-futures
Prometheus
scala_future_created_futures
Elastic
scala-futures.created-futures
Config
cinnamon.scala.future.meta.descriptor {
  future {
    created-futures {
      key = "created-futures"
      unit-type = "custom"
      unit-suffix = "future"
    }
  }
}
Scala Futures processed Rate the rate of Futures/callbacks being processed (marked when the Future/callback has finished running)
OpenTelemetry
scala-future.processed-futures
Prometheus
scala_future_processed_futures
Elastic
scala-futures.processed-futures
Config
cinnamon.scala.future.meta.descriptor {
  future {
    processed-futures {
      key = "processed-futures"
      unit-type = "custom"
      unit-suffix = "future"
    }
  }
}
Scala Future processing time Recorder the time that Futures/callbacks take to run
OpenTelemetry
scala-future.processing-time
Prometheus
scala_future_processing_time_ns
Elastic
scala-futures.processing-time
Config
cinnamon.scala.future.meta.descriptor {
  future {
    processing-time {
      key = "processing-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Scala Futures scheduled Recorder the number of Futures currently scheduled (includes both Future runnables and Future callbacks)
OpenTelemetry
scala-future.scheduled-futures
Prometheus
scala_future_scheduled_futures
Elastic
scala-futures.scheduled-futures
Config
cinnamon.scala.future.meta.descriptor {
  future {
    scheduled-futures {
      key = "scheduled-futures"
      unit-type = "custom"
      unit-suffix = "future"
    }
  }
}
Scala Future scheduling time Recorder the time that Futures/callbacks are in scheduling (in execution context queues waiting to run)
OpenTelemetry
scala-future.scheduling-time
Prometheus
scala_future_scheduling_time_ns
Elastic
scala-futures.scheduling-time
Config
cinnamon.scala.future.meta.descriptor {
  future {
    scheduling-time {
      key = "scheduling-time"
      unit-type = "nanoseconds"
    }
  }
}
Unit conversion
OpenTelemetry duration metrics are converted to seconds by default.
Errors interacting with other agents
Project information