AWS DynamoDB

The AWS DynamoDB connector provides a flow for streaming DynamoDB requests. For more information about DynamoDB please visit the official documentation.

Project Info: Alpakka DynamoDB
Artifact
com.lightbend.akka
akka-stream-alpakka-dynamodb
1.0-M2
JDK versions
OpenJDK 8
Scala versions2.12.7, 2.11.12
JPMS module nameakka.stream.alpakka.aws.dynamodb
License
Readiness level
Community-driven
Since 0.5, 2017-01-13
Home pagehttps://doc.akka.io/docs/alpakka/current/
API documentation
Forums
Release notesIn the documentation
IssuesGithub issues
Sourceshttps://github.com/akka/alpakka

Artifacts

sbt
libraryDependencies += "com.lightbend.akka" %% "akka-stream-alpakka-dynamodb" % "1.0-M2"
Maven
<dependency>
  <groupId>com.lightbend.akka</groupId>
  <artifactId>akka-stream-alpakka-dynamodb_2.12</artifactId>
  <version>1.0-M2</version>
</dependency>
Gradle
dependencies {
  compile group: 'com.lightbend.akka', name: 'akka-stream-alpakka-dynamodb_2.12', version: '1.0-M2'
}

The table below shows direct dependencies of this module and the second tab shows all libraries it depends on transitively.

Direct dependencies
OrganizationArtifactVersionLicense
com.amazonawsaws-java-sdk-dynamodb1.11.476Apache License, Version 2.0
com.typesafe.akkaakka-http_2.1210.1.7Apache-2.0
com.typesafe.akkaakka-stream_2.122.5.19Apache License, Version 2.0
org.scala-langscala-library2.12.7BSD 3-Clause
Dependency tree
com.amazonaws    aws-java-sdk-dynamodb    1.11.476    Apache License, Version 2.0
    com.amazonaws    aws-java-sdk-core    1.11.476    Apache License, Version 2.0
        com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
        com.fasterxml.jackson.dataformat    jackson-dataformat-cbor    2.6.7    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
        commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
        joda-time    joda-time    2.8.1    Apache 2
        org.apache.httpcomponents    httpclient    4.5.5    Apache License, Version 2.0
            commons-codec    commons-codec    1.10    Apache License, Version 2.0
            commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
            org.apache.httpcomponents    httpcore    4.4.9    Apache License, Version 2.0
        software.amazon.ion    ion-java    1.0.2    The Apache License, Version 2.0
    com.amazonaws    aws-java-sdk-s3    1.11.476    Apache License, Version 2.0
        com.amazonaws    aws-java-sdk-core    1.11.476    Apache License, Version 2.0
            com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
            com.fasterxml.jackson.dataformat    jackson-dataformat-cbor    2.6.7    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
            commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
            joda-time    joda-time    2.8.1    Apache 2
            org.apache.httpcomponents    httpclient    4.5.5    Apache License, Version 2.0
                commons-codec    commons-codec    1.10    Apache License, Version 2.0
                commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
                org.apache.httpcomponents    httpcore    4.4.9    Apache License, Version 2.0
            software.amazon.ion    ion-java    1.0.2    The Apache License, Version 2.0
        com.amazonaws    aws-java-sdk-kms    1.11.476    Apache License, Version 2.0
            com.amazonaws    aws-java-sdk-core    1.11.476    Apache License, Version 2.0
                com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
                com.fasterxml.jackson.dataformat    jackson-dataformat-cbor    2.6.7    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
                commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
                joda-time    joda-time    2.8.1    Apache 2
                org.apache.httpcomponents    httpclient    4.5.5    Apache License, Version 2.0
                    commons-codec    commons-codec    1.10    Apache License, Version 2.0
                    commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
                    org.apache.httpcomponents    httpcore    4.4.9    Apache License, Version 2.0
                software.amazon.ion    ion-java    1.0.2    The Apache License, Version 2.0
            com.amazonaws    jmespath-java    1.11.476    Apache License, Version 2.0
                com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
        com.amazonaws    jmespath-java    1.11.476    Apache License, Version 2.0
            com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
    com.amazonaws    jmespath-java    1.11.476    Apache License, Version 2.0
        com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
com.typesafe.akka    akka-http_2.12    10.1.7    Apache-2.0
    com.typesafe.akka    akka-http-core_2.12    10.1.7    Apache-2.0
        com.typesafe.akka    akka-parsing_2.12    10.1.7    Apache-2.0
            org.scala-lang    scala-library    2.12.7    BSD 3-Clause
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    org.scala-lang    scala-library    2.12.7    BSD 3-Clause
com.typesafe.akka    akka-stream_2.12    2.5.19    Apache License, Version 2.0
    com.typesafe.akka    akka-actor_2.12    2.5.19    Apache License, Version 2.0
        com.typesafe    config    1.3.3    Apache License, Version 2.0
        org.scala-lang.modules    scala-java8-compat_2.12    0.8.0    BSD 3-clause
            org.scala-lang    scala-library    2.12.7    BSD 3-Clause
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    com.typesafe.akka    akka-protobuf_2.12    2.5.19    Apache License, Version 2.0
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    com.typesafe    ssl-config-core_2.12    0.3.6    Apache-2.0
        com.typesafe    config    1.3.3    Apache License, Version 2.0
        org.scala-lang.modules    scala-parser-combinators_2.12    1.1.1    BSD 3-clause
            org.scala-lang    scala-library    2.12.7    BSD 3-Clause
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    org.reactivestreams    reactive-streams    1.0.2    CC0
    org.scala-lang    scala-library    2.12.7    BSD 3-Clause
org.scala-lang    scala-library    2.12.7    BSD 3-Clause

Setup

This connector needs a DynamoClient instance in order to create stream operators. You can create the client yourself, or let the Akka Extension handle the lifecycle of the client for you.

Factories provided in the DynamoDb will use the client managed by the extension. The managed client will be created using the configuration resolved from reference.conf and with overrides from application.conf.

Example application.conf
akka.stream.alpakka.dynamodb {
 region = "eu-west-1"
 host = "localhost"
 port = 443
 tls = true
 parallelism = 32
 credentials {
   access-key-id = "dummy-access-key"
   secret-key-id = "dummy-secret-key"
 }
reference.conf
akka.stream.alpakka.dynamodb {
  # The AWS region
  region = ""

  # The AWS host
  host = ""

  # The AWS port, https is used as default scheme
  port = -1

  # Use Transport Level Security (https)
  tls = true

  # Max number of in flight requests from the AwsClient - must be a power of 2
  parallelism = 32

  # Optional Credentials. Used to define static credentials rather than use the DefaultAWSCredentialsProviderChain
  #  access-key-id = "dummy-access-key"
  #  secret-key-id = "dummy-secret-key"
  #}
}

If the credentials are not set in the configuration, connector will use the default credential provider chain provided by the DynamoDB Java SDK to retrieve credentials.

There is another set of factories under the DynamoDbExternal which take an instance of DynamoClient as a parameter. You might want to use a manually initiated client, if, for example, you need to use a custom credentials provider, which you may attach programmatically via the withCredentialsProvider method in DynamoSettings.

Scala
val provider: AWSCredentialsProvider = // ???
val settings = DynamoSettings("eu-west-1", "host")
  .withCredentialsProvider(provider)
Java
AWSCredentialsProvider provider = // ...
DynamoSettings settings =
    DynamoSettings.create("eu-west-1", "localhost").withCredentialsProvider(provider);

Usage

For simple operations you can issue a single request, and get back the result in a FutureCompletionStage.

Scala
import DynamoImplicits._
val listTablesResult: Future[ListTablesResult] =
  DynamoDb.single(new ListTablesRequest())
Java
final CompletionStage<ListTablesResult> listTablesResultFuture =
    DynamoDb.listTables(new ListTablesRequest(), system);

You can also get the response to a request as an element emitted from a Flow:

Scala
import DynamoImplicits._
val source: Source[String, NotUsed] = Source
  .single[CreateTable](new CreateTableRequest().withTableName("testTable"))
  .via(DynamoDb.flow)
  .map(_.getTableDescription.getTableArn)
Java
Source<String, NotUsed> tableArnSource =
    Source.single(new CreateTable(new CreateTableRequest().withTableName("testTable")))
        .via(DynamoDb.flow(system))
        .map(result -> (CreateTableResult) result)
        .map(result -> result.getTableDescription().getTableArn());

Some DynamoDB operations, such as Query and Scan, are paginated by nature. This is how you can get a stream of all result pages:

Scala
val scanPages: Source[ScanResult, NotUsed] =
  DynamoDb.source(new ScanRequest().withTableName("testTable"))
Java
Source<ScanResult, NotUsed> scanPages =
    DynamoDb.scanAll(new ScanRequest().withTableName("testTable"), system);

Running the example code

The code in this guide is part of runnable tests of this project. You are welcome to edit the code and run it in sbt.

Test code requires DynamoDB running in the background. You can start one quickly using docker:

docker-compose up dynamodb

Scala
sbt
> dynamodb/testOnly *Spec
Java
sbt
> dynamodb/testOnly *Test
Found an error in this documentation? The source code for this page can be found here. Please feel free to edit and contribute a pull request.