Logging

Logging of Dead Letters

By default messages sent to dead letters are logged at info level. Existence of dead letters does not necessarily indicate a problem, but it might be, and therefore they are logged by default. After a few messages this logging is turned off, to avoid flooding the logs. You can disable this logging completely or adjust how many dead letters that are logged. During system shutdown it is likely that you see dead letters, since pending messages in the actor mailboxes are sent to dead letters. You can also disable logging of dead letters during shutdown.

akka {
  log-dead-letters = 10
  log-dead-letters-during-shutdown = on
}

To customize the logging further or take other actions for dead letters you can subscribe to the Event Stream.

Auxiliary logging options

Akka has a couple of configuration options for very low level debugging, that makes most sense in for developers and not for operations.

You almost definitely need to have logging set to DEBUG to use any of the options below:

akka {
  loglevel = "DEBUG"
}

This config option is very good if you want to know what config settings are loaded by Akka:

akka {
  # Log the complete configuration at INFO level when the actor system is started.
  # This is useful when you are uncertain of what configuration is used.
  log-config-on-start = on
}

If you want very detailed logging of user-level messages then wrap your actors' behaviors with akka.event.LoggingReceive and enable the receive option:

akka {
  actor {
    debug {
      # enable function of LoggingReceive, which is to log any received message at
      # DEBUG level
      receive = on
    }
  }
}

If you want very detailed logging of all automatically received messages that are processed by Actors:

akka {
  actor {
    debug {
      # enable DEBUG logging of all AutoReceiveMessages (Kill, PoisonPill et.c.)
      autoreceive = on
    }
  }
}

If you want very detailed logging of all lifecycle changes of Actors (restarts, deaths etc):

akka {
  actor {
    debug {
      # enable DEBUG logging of actor lifecycle changes
      lifecycle = on
    }
  }
}

If you want unhandled messages logged at DEBUG:

akka {
  actor {
    debug {
      # enable DEBUG logging of unhandled messages
      unhandled = on
    }
  }
}

If you want very detailed logging of all events, transitions and timers of FSM Actors that extend LoggingFSM:

akka {
  actor {
    debug {
      # enable DEBUG logging of all LoggingFSMs for events, transitions and timers
      fsm = on
    }
  }
}

If you want to monitor subscriptions (subscribe/unsubscribe) on the ActorSystem.eventStream:

akka {
  actor {
    debug {
      # enable DEBUG logging of subscription changes on the eventStream
      event-stream = on
    }
  }
}

Auxiliary remote logging options

If you want to see all messages that are sent through remoting at DEBUG log level: (This is logged as they are sent by the transport layer, not by the Actor)

akka {
  remote {
    # If this is "on", Akka will log all outbound messages at DEBUG level,
    # if off then they are not logged
    log-sent-messages = on
  }
}

If you want to see all messages that are received through remoting at DEBUG log level: (This is logged as they are received by the transport layer, not by any Actor)

akka {
  remote {
    # If this is "on", Akka will log all inbound messages at DEBUG level,
    # if off then they are not logged
    log-received-messages = on
  }
}

If you want to see message types with payload size in bytes larger than a specified limit at INFO log level:

akka {
  remote {
    # Logging of message types with payload size in bytes larger than
    # this value. Maximum detected size per message type is logged once,
    # with an increase threshold of 10%.
    # By default this feature is turned off. Activate it by setting the property to
    # a value in bytes, such as 1000b. Note that for all messages larger than this
    # limit there will be extra performance and scalability cost.
    log-frame-size-exceeding = 1000b
  }
}

Also see the logging options for TestKit: Tracing Actor Invocations.

Translating Log Source to String and Class

The rules for translating the source object to the source string and class which are inserted into the LogEvent during runtime are implemented using implicit parameters and thus fully customizable: simply create your own instance of LogSource[T] and have it in scope when creating the logger.

import akka.actor.ActorSystem
import akka.event.LogSource

object MyType {
  implicit val logSource: LogSource[AnyRef] = new LogSource[AnyRef] {
    def genString(o: AnyRef): String = o.getClass.getName
    override def getClazz(o: AnyRef): Class[_] = o.getClass
  }
}

class MyType(system: ActorSystem) {
  import MyType._
  import akka.event.Logging

  val log = Logging(system, this)
}

This example creates a log source which mimics traditional usage of Java loggers, which are based upon the originating object’s class name as log category. The override of getClazz is only included for demonstration purposes as it contains exactly the default behavior.

Turn Off Logging

To turn off logging you can configure the log levels to be OFF like this.

akka {
  stdout-loglevel = "OFF"
  loglevel = "OFF"
}

The stdout-loglevel is only in effect during system startup and shutdown, and setting it to OFF as well, ensures that nothing gets logged during system startup or shutdown.

Using the SLF4J API directly

If you use the SLF4J API directly in your application, remember that the logging operations will block while the underlying infrastructure writes the log statements.

This can be avoided by configuring the logging implementation to use a non-blocking appender. Logback provides AsyncAppender that does this. It also contains a feature which will drop INFO and DEBUG messages if the logging load is high.

Logging Thread, Akka Source and Actor System in MDC

Since the logging is done asynchronously the thread in which the logging was performed is captured in Mapped Diagnostic Context (MDC) with attribute name sourceThread. With Logback the thread name is available with %X{sourceThread} specifier within the pattern layout configuration:

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
  <encoder>
    <pattern>%date{ISO8601} %-5level %logger{36} %X{sourceThread} - %msg%n</pattern>
  </encoder>
</appender>

Another helpful facility is that Akka captures the actor’s address when instantiating a logger within it, meaning that the full instance identification is available for associating log messages e.g. with members of a router. This information is available in the MDC with attribute name akkaSource:

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
  <encoder>
    <pattern>%date{ISO8601} %-5level %logger{36} %X{akkaSource} - %msg%n</pattern>
  </encoder>
</appender>

Finally, the actor system in which the logging was performed is available in the MDC with attribute name sourceActorSystem:

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
  <encoder>
    <pattern>%date{ISO8601} %-5level %logger{36} %X{sourceActorSystem} - %msg%n</pattern>
  </encoder>
</appender>

For more details on what this attribute contains—also for non-actors—please see How to Log.

More accurate timestamps for log output in MDC

Akka's logging is asynchronous which means that the timestamp of a log entry is taken from when the underlying logger implementation is called, which can be surprising at first. If you want to more accurately output the timestamp, use the MDC attribute akkaTimestamp:

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
  <encoder>
    <pattern>%X{akkaTimestamp} %-5level %logger{36} %X{akkaSource} - %msg%n</pattern>
  </encoder>
</appender>

MDC values defined by the application

One useful feature available in Slf4j is MDC, Akka has a way for let the application specify custom values, you just need to get a specialized LoggingAdapter, the DiagnosticLoggingAdapter. In order to get it you will use the factory receiving an Actor as logSource:

// Within your Actor
val log: DiagnosticLoggingAdapter = Logging(this);

Once you have the logger, you just need to add the custom values before you log something. This way, the values will dologbe put in the SLF4J MDC right before appending the log and removed after.

val mdc = Map("requestId" -> 1234, "visitorId" -> 5678)
log.mdc(mdc)

// Log something
log.info("Starting new request")

log.clearMDC()

For convenience you can mixin the log member into actors, instead of defining it as above. This trait also lets you override def mdc(msg: Any): MDC for specifying MDC values depending on current message and lets you forget about the cleanup as well, since it already does it for you.

import Logging.MDC

final case class Req(work: String, visitorId: Int)

class MdcActorMixin extends Actor with akka.actor.DiagnosticActorLogging {
  var reqId = 0

  override def mdc(currentMessage: Any): MDC = {
    reqId += 1
    val always = Map("requestId" -> reqId)
    val perMessage = currentMessage match {
      case r: Req => Map("visitorId" -> r.visitorId)
      case _      => Map()
    }
    always ++ perMessage
  }

  def receive: Receive = {
    case r: Req => {
      log.info(s"Starting new request: ${r.work}")
    }
  }
}

Now, the values will be available in the MDC, so you can use them in the layout pattern:

<appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
  <encoder>
    <pattern>
      %-5level %logger{36} [req: %X{requestId}, visitor: %X{visitorId}] - %msg%n
    </pattern>
  </encoder>
</appender>

Using Markers

Some logging libraries allow, in addition to MDC data, attaching so called "markers" to log statements. These are used to filter out rare and special events, for example you might want to mark logs that detect some malicious activity and mark them with a SECURITY tag, and in your appender configuration make these trigger emails and other notifications immediately.

Markers are available through the LoggingAdapters, when obtained via Logging.withMarker. The first argument passed into all log calls then should be a akka.event.LogMarker.

The slf4j bridge provided by akka in akka-slf4j will automatically pick up this marker value and make it available to SLF4J. For example you could use it like this:

<pattern>%date{ISO8601} [%marker][%level] [%msg]%n</pattern>

A more advanced (including most Akka added information) example pattern would be:

<pattern>%date{ISO8601} level=[%level] marker=[%marker] logger=[%logger] akkaSource=[%X{akkaSource}] sourceActorSystem=[%X{sourceActorSystem}] sourceThread=[%X{sourceThread}] mdc=[ticket-#%X{ticketNumber}: %X{ticketDesc}] - msg=[%msg]%n----%n</pattern>