Durable Mailboxes

Durable Mailboxes

Overview

Akka supports a set of durable mailboxes. A durable mailbox is a replacement for the standard actor mailbox that is durable. What this means in practice is that if there are pending messages in the actor’s mailbox when the node of the actor resides on crashes, then when you restart the node, the actor will be able to continue processing as if nothing had happened; with all pending messages still in its mailbox.

None of these mailboxes implements transactions for current message. It’s possible if the actor crashes after receiving a message, but before completing processing of it, that the message could be lost.

Warning

IMPORTANT

None of these mailboxes work with blocking message send, i.e. the message send operations that are relying on futures; ? or ask. If the node has crashed and then restarted, the thread that was blocked waiting for the reply is gone and there is no way we can deliver the message.

The durable mailboxes currently supported are:

  • FileBasedMailbox – backed by a journaling transaction log on the local file system
  • RedisBasedMailbox – backed by Redis
  • ZooKeeperBasedMailbox – backed by ZooKeeper
  • BeanstalkBasedMailbox – backed by Beanstalkd
  • MongoBasedMailbox – backed by MongoDB

We’ll walk through each one of these in detail in the sections below.

You can easily implement your own mailbox. Look at the existing implementations for inspiration.

We are also discussing adding some of these durable mailboxes:

  • AmqpBasedMailbox – AMQP based mailbox (default RabbitMQ)
  • JmsBasedMailbox – JMS based mailbox (default ActiveMQ)
  • CassandraBasedMailbox – Cassandra based mailbox
  • CamelBasedMailbox – Camel based mailbox
  • SqlBasedMailbox – SQL based mailbox for general RDBMS (Postgres, MySQL, Oracle etc.)

Let us know if you have a wish for a certain priority order.

General Usage

The durable mailboxes and their configuration options reside in the akka.actor.mailbox package.

You configure durable mailboxes through the dispatcher. The actor is oblivious to which type of mailbox it is using.

In the configuration of the dispatcher you specify the fully qualified class name of the mailbox:

my-dispatcher {
  mailbox-type = akka.actor.mailbox.FileBasedMailboxType
}

Here is an example of how to create an actor with a durable dispatcher, in Scala:

import akka.actor.Props

    val myActor = system.actorOf(Props[MyActor].withDispatcher("my-dispatcher"), name = "myactor")

Corresponding example in Java:

import akka.actor.UntypedActorFactory;
import akka.actor.UntypedActor;
import akka.actor.Props;

    ActorRef myActor = system.actorOf(
        new Props().withDispatcher("my-dispatcher").withCreator(new UntypedActorFactory() {
          public UntypedActor create() {
            return new MyUntypedActor();
          }
        }), "myactor");

The actor is oblivious to which type of mailbox it is using.

This gives you an excellent way of creating bulkheads in your application, where groups of actors sharing the same dispatcher also share the same backing storage. Read more about that in the Dispatchers (Scala) documentation.

File-based durable mailbox

This mailbox is backed by a journaling transaction log on the local file system. It is the simplest to use since it does not require an extra infrastructure piece to administer, but it is usually sufficient and just what you need.

You configure durable mailboxes through the dispatcher, as described in General Usage with the following mailbox type.

Config:

my-dispatcher {
  mailbox-type = akka.actor.mailbox.FileBasedMailboxType
}

You can also configure and tune the file-based durable mailbox. This is done in the akka.actor.mailbox.file-based section in the Configuration.

#############################################
# Akka File Mailboxes Reference Config File #
#############################################

# This is the reference config file that contains all the default settings.
# Make your edits/overrides in your application.conf.
#
# For more information see <https://github.com/robey/kestrel/>

akka {
  actor {
    mailbox {
      file-based {
        # directory below which this queue resides
        directory-path = "./_mb"
        
        # attempting to add an item after the queue reaches this size (in items) will fail.
        max-items = 2147483647
        
        # attempting to add an item after the queue reaches this size (in bytes) will fail.
        max-size = 2147483647 bytes
        
        # attempting to add an item larger than this size (in bytes) will fail.
        max-item-size = 2147483647 bytes
        
        # maximum expiration time for this queue (seconds).
        max-age = 0s
        
        # maximum journal size before the journal should be rotated.
        max-journal-size = 16 MiB
        
        # maximum size of a queue before it drops into read-behind mode.
        max-memory-size = 128 MiB
        
        # maximum overflow (multiplier) of a journal file before we re-create it.
        max-journal-overflow = 10
        
        # absolute maximum size of a journal file until we rebuild it, no matter what.
        max-journal-size-absolute = 9223372036854775807 bytes
        
        # whether to drop older items (instead of newer) when the queue is full
        discard-old-when-full = on  
        
        # whether to keep a journal file at all
        keep-journal = on  
        
        # whether to sync the journal after each transaction
        sync-journal = off  
      }
    }
  }
}

Redis-based durable mailbox

This mailbox is backed by a Redis queue. Redis Is a very fast NOSQL database that has a wide range of data structure abstractions, one of them is a queue which is what we are using in this implementation. This means that you have to start up a Redis server that can host these durable mailboxes. Read more in the Redis documentation on how to do that.

You configure durable mailboxes through the dispatcher, as described in General Usage with the following mailbox type.

Config:

my-dispatcher {
  mailbox-type = akka.actor.mailbox.RedisBasedMailboxType
}

You also need to configure the IP and port for the Redis server. This is done in the akka.actor.mailbox.redis section in the Configuration.

##############################################
# Akka Redis Mailboxes Reference Config File #
##############################################

# This is the reference config file that contains all the default settings.
# Make your edits/overrides in your application.conf.
#
# for more information see <http://redis.io/>

akka {
  actor {
    mailbox {
      redis {
        # hostname of where the redis queue resides
        hostname = "127.0.0.1"
        
        # port at which the redis queue resides
        port = 6379
      }
    }
  }
}

ZooKeeper-based durable mailbox

This mailbox is backed by ZooKeeper. ZooKeeper is a centralized service for maintaining configuration information, naming, providing distributed synchronization, and providing group services This means that you have to start up a ZooKeeper server (for production a ZooKeeper server ensemble) that can host these durable mailboxes. Read more in the ZooKeeper documentation on how to do that.

You configure durable mailboxes through the dispatcher, as described in General Usage with the following mailbox type.

Config:

my-dispatcher {
  mailbox-type = akka.actor.mailbox.ZooKeeperBasedMailboxType
}

You also need to configure ZooKeeper server addresses, timeouts, etc. This is done in the akka.actor.mailbox.zookeeper section in the Configuration.

##################################################
# Akka ZooKepper Mailboxes Reference Config File #
##################################################

# This is the reference config file that contains all the default settings.
# Make your edits/overrides in your application.conf.
#
# For more information see <http://wiki.apache.org/hadoop/ZooKeeper>

akka {
  actor {
    mailbox {
      zookeeper {
        # host and port to connect to ZooKeeper
        server-addresses = "127.0.0.1:2181"
        
        # timeout after which an unreachable client is considered dead and its session is closed
        session-timeout = 60s
        
        # maximum wait period while connecting to ZooKeeper service
        connection-timeout = 60s
      }
    }
  }
}

Beanstalk-based durable mailbox

This mailbox is backed by Beanstalkd. Beanstalk is a simple, fast work queue. This means that you have to start up a Beanstalk server that can host these durable mailboxes. Read more in the Beanstalk documentation on how to do that.

You configure durable mailboxes through the dispatcher, as described in General Usage with the following mailbox type.

Config:

my-dispatcher {
  mailbox-type = akka.actor.mailbox.BeanstalkBasedMailboxType
}

You also need to configure the IP, and port, and so on, for the Beanstalk server. This is done in the akka.actor.mailbox.beanstalk section in the Configuration.

##################################################
# Akka Beanstalk Mailboxes Reference Config File #
##################################################

# This is the reference config file that contains all the default settings.
# Make your edits/overrides in your application.conf.
#
# for more information see <https://github.com/kr/beanstalkd/blob/v1.3/doc/protocol.txt>

akka {
  actor {
    mailbox {
      beanstalk {
        # hostname to connect to
        hostname = "127.0.0.1"
        
        # port to connect to
        port = 11300
        
        # wait period in case of a connection failure before reconnect
        reconnect-window = 5s
        
        # integer number of seconds to wait before putting the job in
        # the ready queue. The job will be in the "delayed" state during this time.
        message-submit-delay = 0s
        
        # time to run -- is an integer number of seconds to allow a worker
        # to run this job. This time is counted from the moment a worker reserves
        # this job. If the worker does not delete, release, or bury the job within
        # <ttr> seconds, the job will time out and the server will release the job.
        message-time-to-live = 120s
      }
    }
  }

}

MongoDB-based Durable Mailboxes

This mailbox is backed by MongoDB. MongoDB is a fast, lightweight and scalable document-oriented database. It contains a number of features cohesive to a fast, reliable & durable queueing mechanism which the Akka Mailbox takes advantage of.

Akka’s implementations of MongoDB mailboxes are built on top of the purely asynchronous MongoDB driver (often known as Hammersmith and com.mongodb.async) and as such are purely callback based with a Netty network layer. This makes them extremely fast & lightweight versus building on other MongoDB implementations such as mongo-java-driver and Casbah.

You configure durable mailboxes through the dispatcher, as described in General Usage with the following mailbox type.

Config:

my-dispatcher {
  mailbox-type = akka.actor.mailbox.MongoBasedMailboxType
}

You will need to configure the URI for the MongoDB server, using the URI Format specified in the MongoDB Documentation. This is done in the akka.actor.mailbox.mongodb section in the Configuration.

################################################
# Akka MongoDB Mailboxes Reference Config File #
################################################

# This is the reference config file that contains all the default settings.
# Make your edits/overrides in your application.conf.

akka {
  actor {
    mailbox {
      mongodb {

        # Any specified collection name will be used as a prefix for
        # collections that use durable mongo mailboxes.
        # Follow Mongo URI Spec - http://www.mongodb.org/display/DOCS/Connections
        uri = "mongodb://localhost/akka.mailbox"

        # Configurable timeouts for certain ops
        timeout {
          # time to wait for a read to succeed before timing out the future
          read = 3000ms
          # time to wait for a write to succeed before timing out the future
          write = 3000ms
        }
      }
    }
  }
}

You must specify a hostname (and optionally port) and at least a Database name. If you specify a collection name, it will be used as a ‘prefix’ for the collections Akka creates to store mailbox messages. Otherwise, collections will be prefixed with mailbox.

It is also possible to configure the timeout thresholds for Read and Write operations in the timeout block.

Contents