Lagom configuration
Lagom endpoints need to be selected for Telemetry using configuration. Cinnamon accesses the same configuration as used to configure an ActorSystem
. The default configuration is in application.conf
.
Server metrics
Server metrics are automatically enabled when Cinnamon’s Lagom dependency is on the classpath.
It is possible to turn server-metrics off for a server with configuration, see override and disable server metrics below.
Server metrics configuration
Endpoint metrics are not enabled by default. In other words, without any specific configuration, there will be no metrics for the endpoints.
Endpoint configuration settings are grouped per “host:port.” Cinnamon will inquire the configuration to determine how to set things up.
The configuration should define the following settings:
Setting | Explanation |
---|---|
host:port | The IP address and port the setting is valid for. Can be set to "*:*" to be valid for all hosts/ports. |
server-metrics | An optional setting (by default set to on ) that can be used to turn metrics off for server(s). |
paths | A section in which endpoint paths are defined with specific settings per endpoint. |
Each endpoint in the paths
section should define the following:
Setting | Explanation |
---|---|
endpoint path | The endpoint path to configure. A * wildcard can be used at the end to select all subpaths. |
metrics | Should be set to on , true or yes for metrics to be created. |
status-code-classes | Should be set to on , true or yes for endpoint metrics per status code class to be created. |
Server endpoint naming
By default, endpoint metrics created use the endpoint path as name for the metric. Let’s look at an example to clarify this feature:
Path directive | Service definition | Metric name |
---|---|---|
pathCall("/api/hello/:id", hello _) |
def hello(id: String) |
/api/hello/_id |
pathCall("/order/:orderId/item/:itemId", getItem _) |
def getItem(orderId: Long, itemId: String) |
/order/_orderId/item/_itemId |
Configuration conflict resolution
If there are several HTTP related configuration settings available on the classpath, e.g. Lagom and Akka HTTP, Lightbend Telemetry will merge all settings if possible. Should there be conflicting settings, the following order is applied: Lagom > Play > Akka HTTP.
Example configurations
Simple server configuration
The simplest configuration possible is the one that matches all paths, hosts and ports:
cinnamon.lagom.http.servers {
"*:*" {
paths {
"*" {
metrics = on
}
}
}
}
Configure specific endpoint paths
Below is an example configuration that is more specific than the simple server configuration:
cinnamon.lagom.http.servers {
"192.168.10.1:8080" {
paths {
"users/us/*" {
metrics = on
}
"users/se/*" {
metrics = on
}
"users/nz/*" {
metrics = on
}
}
}
}
The above configuration will create metrics for paths that map to users
in us
, se
, or nz
. Notice that all users in these domains are valid since we use *
as part of the paths. Furthermore, this configuration is only valid for a server started on IP 192.168.10.1
and port 8080
.
Override and disable server metrics
HTTP server configuration is selected from most specific to least specific port and host combination. This means that you can override wildcard configurations by having a more specific configuration section that have different settings.
Turn off all metrics for a particular host while still having them turned on for all other hosts:
cinnamon.lagom.http.servers {
"*:*" {
paths {
"*" {
metrics = on
}
}
},
"192.168.0.1:*" {
server-metrics = off
}
}
The server-metrics
setting is optional and if not present it will automatically be set to on
.
Enable endpoint metrics per status code class
Available since Cinnamon 2.13.0
Like overall server metrics, endpoint metrics (response rate and response time) can be recorded for each status code class (2xx, 3xx, 4xx, or 5xx). Use the status-code-classes
setting to enable these additional metrics for matching endpoint paths:
cinnamon.lagom.http.servers {
"*:*" {
paths {
"a/*" {
metrics = on
}
"b/*" {
metrics = on
status-code-classes = on
}
}
}
}
Client metrics configuration
Client metrics configuration settings are grouped per “host:port”. In the context of a Lagom service client we refer to “host:port” as a service. Each service can define which “paths” should generate metrics.
Let us look at a couple of client metrics configuration examples.
Simple client configuration
The simplest configuration possible is the one that matches all paths, hosts and ports:
cinnamon.lagom.http.clients {
"*:*" {
paths {
"*" {
metrics = on
}
}
}
}
With the above configuration all service calls will be monitored and metrics generated will be under the service name “:”.
Configure client service name
In the configuration below we use the setting service-name
which means that metrics will be generated in the context of “accountService” and “customerService”.
cinnamon.lagom.http.clients {
"192.168.10.1:*" {
service-name = "accountService"
paths {
"*" {
metrics = on
}
}
},
"192.168.10.10:8080" {
service-name = "customerService"
paths {
"*" {
metrics = on
}
}
}
}
Configure specific client paths
It is also possible to define what paths of a service that should generate and contribute to the metrics.
cinnamon.lagom.http.clients {
"192.168.10.1:8080" {
paths {
"customer/address/*" {
metrics = on
}
}
}
}
On the contrary to server endpoint metrics, in which unique metrics are created per path endpoint, any defined paths in client metrics configuration contribute to one overall metrics per service.
Circuit breaker configuration
To wire up Lagom circuit breakers telemetry in your Java application you must add the following line to the configuration file:
lagom.spi.circuit-breaker-metrics-class = "cinnamon.lagom.CircuitBreakerInstrumentation"
For Scala, cinnamon.lagom.CircuitBreakerInstrumentation
should be wired in your application. See the Scala example for details.
This instructs Lagom to provide data to Lightbend Telemetry rather than the default behavior. The information gathered include:
-
State of circuit breaker — closed, open, half-open
-
Latency
-
Throughput
-
Success and failure counts
Lagom circuit breakers
It is important to have an understanding of the circuit breakers in Lagom in order to interpret the data that is gathered. In Lagom each client talking to another service uses a circuit breaker. This basically means that the circuit breaker is on the client instead of the server (in a traditional sense). This means that if your code calls a service directly there will not be any information gathered about this call. Currently only service to service calls are instrumented. See the Lagom documentation for more information on service clients and circuit breakers.
The state of a circuit breaker is one of:
State | Interpretation | Data Representation |
---|---|---|
open | something is wrong | 1 |
half-open | things are going from bad to good | 2 |
closed | everything is fine | 3 |
The Data Representation
in the above table is the number associated with a certain state, i.e. this is what comes out of Lightbend Telemetry to any reporter listening to the data.