Elasticsearch

The Elasticsearch reporter works with versions 5.x, 6.x, 7.x and 8.x of Elasticsearch.

Cinnamon dependency

First make sure that your build is configured to use the Cinnamon Agent.

Reporter dependency

Add the core Coda Hale Metrics dependency and the Elasticsearch reporter dependency to your build:

sbt

copylibraryDependencies += Cinnamon.library.cinnamonCHMetrics
libraryDependencies += Cinnamon.library.cinnamonCHMetricsElasticsearchReporter

Maven

copy<dependency>
    <groupId>com.lightbend.cinnamon</groupId>
    <artifactId>cinnamon-chmetrics</artifactId>
    <version>2.20.3</version>
</dependency>
<dependency>
    <groupId>com.lightbend.cinnamon</groupId>
    <artifactId>cinnamon-chmetrics-elasticsearch-reporter</artifactId>
    <version>2.20.3</version>
</dependency>

Gradle

copydependencies {
  implementation group: 'com.lightbend.cinnamon', name: 'cinnamon-chmetrics', version: '2.20.3'
}
dependencies {
  implementation group: 'com.lightbend.cinnamon', name: 'cinnamon-chmetrics-elasticsearch-reporter', version: '2.20.3'
}

Reporter configuration

Below is the configuration needed to set up either version of the Elasticsearch reporter. The Reference tab shows all the configurable settings for the Elasticsearch reporter.

Required

copycinnamon.chmetrics {
  reporters += elasticsearch-reporter
}

Reference

copycinnamon.chmetrics {
  elasticsearch-reporter {

    # A list of hosts used to connect to elasticsearch.
    # Must be in the format `[http://][username:password@]hostname:port`.
    hosts = ["localhost:9200"]

    # Username and password to be used with all hosts (that don't define credentials directly).
    # Basic authentication is enabled when both username and password are non-empty.
    basic-auth {
      username = ""
      password = ""
    }

    # Additional fields to be included will all metrics.
    # Structured config data will be converted to structured JSON.
    # E.g. the following configuration:
    #   additional-fields {
    #     country = "UK"
    #     environment {
    #       role = "staging"
    #       run = 123456
    #     }
    #   }
    # will add those fields to every metric document in Elasticsearch.
    additional-fields {}

    # Milliseconds to wait for an established connection, before the next host in the list is tried.
    timeout = 1000

    # Defines how many metrics are sent per bulk requests.
    bulksize = 2500

    # The name of the index to write to, defaults to metrics.
    index = "cinnamon-metrics"

    # The date format to make sure to rotate to a new index.
    index-date-format = "yyyy-MM"

    # The field name of the timestamp (kibana default is @timestamp).
    timestamp-field-name = "@timestamp"

    # Frequency with which to report metrics.
    frequency = 10s

    # JSON files containing index mappings.
    # Default mapping will be applied if no file is specified.
    index-mapping-file-names = []

    # Whether to enable detailed debug logging for the Elasticsearch reporter.
    # This also requires enabling DEBUG in your logger for class:
    #   `com.lightbend.cinnamon.chmetrics.elasticsearch.CinnamonElasticsearchReporter`
    # AND setting the `akka.loglevel` configuration to DEBUG
    # Note: only use for troubleshooting.
    debug = off

    # Default to use legacy template api.
    # set to `true` to use the index template api.
    v8-composable-index-template = false

    percolation {
      # MetricFilter to define which metrics should be percolated.
      filter = "empty-filter"

      empty-filter {
        filter-class = "com.lightbend.cinnamon.chmetrics.elasticsearch.EmptyFilter"
      }

      # Implementation of the Notifier interface, which is executed upon a matching percolator.
      notifier = "empty-notifier"

      empty-notifier {
        notifier-class = "com.lightbend.cinnamon.chmetrics.elasticsearch.EmptyNotifier"
      }
    }
  }
}

HTTP authentication

To set credentials for Elasticsearch using HTTP Basic Authentication, configure the basic-auth section for the reporter:

copycinnamon.chmetrics {
  reporters += elasticsearch-reporter
  elasticsearch-reporter {
    hosts = ["https://somewhere.org:9200"]
    basic-auth {
      username = "elastic"
      password = "changeme"
    }
  }
}

You can specify that connections should be secured using SSL/TLS by including the https scheme.

The credentials can also be set individually on each host (and this will be selected over the global basic-auth configuration):

copycinnamon.chmetrics {
  reporters += elasticsearch-reporter
  elasticsearch-reporter {
    hosts = ["https://elastic:[email protected]:9200"]
  }
}

Host and application identifiers

The host and application identifiers reported to Elasticsearch are based on the shared Cinnamon metadata, and will default to the local host name and the Java main class. To configure these, use the cinnamon.host and cinnamon.application settings. For example:

copycinnamon {
  host = "somewhere"
  application = "awesome-sauce"
}

Additional fields

Additional fields can be added for all metrics reported to Elasticsearch. Configure the additional-fields section for the reporter:

copycinnamon.chmetrics {
  reporters += elasticsearch-reporter
  elasticsearch-reporter {
    additional-fields {
      country = "UK"
      environment {
        role = "staging"
        run = 123456
      }
    }
  }
}

With the configuration above, the following additional fields will be reported along with every metric document to Elasticsearch:

copy"country": "UK",
"environment": { "role": "staging", "run": 123456 }