AWS DynamoDB

The AWS DynamoDB connector provides a flow for streaming DynamoDB requests. For more information about DynamoDB please visit the official documentation.

Project Info: Alpakka DynamoDB
Artifact
com.lightbend.akka
akka-stream-alpakka-dynamodb
1.1.2
JDK versions
OpenJDK 8
Scala versions2.12.7, 2.11.12, 2.13.0
JPMS module nameakka.stream.alpakka.aws.dynamodb
License
Readiness level
Since 0.5, 2017-01-13
Home pagehttps://doc.akka.io/docs/alpakka/current
API documentation
Forums
Release notesIn the documentation
IssuesGithub issues
Sourceshttps://github.com/akka/alpakka

Artifacts

sbt
libraryDependencies += "com.lightbend.akka" %% "akka-stream-alpakka-dynamodb" % "1.1.2"
Maven
<dependency>
  <groupId>com.lightbend.akka</groupId>
  <artifactId>akka-stream-alpakka-dynamodb_2.12</artifactId>
  <version>1.1.2</version>
</dependency>
Gradle
dependencies {
  compile group: 'com.lightbend.akka', name: 'akka-stream-alpakka-dynamodb_2.12', version: '1.1.2'
}

The table below shows direct dependencies of this module and the second tab shows all libraries it depends on transitively.

Direct dependencies
OrganizationArtifactVersionLicense
com.amazonawsaws-java-sdk-dynamodb1.11.476Apache License, Version 2.0
com.typesafe.akkaakka-http_2.1210.1.10Apache-2.0
com.typesafe.akkaakka-stream_2.122.5.23Apache License, Version 2.0
org.scala-langscala-library2.12.7BSD 3-Clause
Dependency tree
com.amazonaws    aws-java-sdk-dynamodb    1.11.476    Apache License, Version 2.0
    com.amazonaws    aws-java-sdk-core    1.11.476    Apache License, Version 2.0
        com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
        com.fasterxml.jackson.dataformat    jackson-dataformat-cbor    2.6.7    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
        commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
        joda-time    joda-time    2.8.1    Apache 2
        org.apache.httpcomponents    httpclient    4.5.5    Apache License, Version 2.0
            commons-codec    commons-codec    1.10    Apache License, Version 2.0
            commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
            org.apache.httpcomponents    httpcore    4.4.9    Apache License, Version 2.0
        software.amazon.ion    ion-java    1.0.2    The Apache License, Version 2.0
    com.amazonaws    aws-java-sdk-s3    1.11.476    Apache License, Version 2.0
        com.amazonaws    aws-java-sdk-core    1.11.476    Apache License, Version 2.0
            com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
            com.fasterxml.jackson.dataformat    jackson-dataformat-cbor    2.6.7    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
            commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
            joda-time    joda-time    2.8.1    Apache 2
            org.apache.httpcomponents    httpclient    4.5.5    Apache License, Version 2.0
                commons-codec    commons-codec    1.10    Apache License, Version 2.0
                commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
                org.apache.httpcomponents    httpcore    4.4.9    Apache License, Version 2.0
            software.amazon.ion    ion-java    1.0.2    The Apache License, Version 2.0
        com.amazonaws    aws-java-sdk-kms    1.11.476    Apache License, Version 2.0
            com.amazonaws    aws-java-sdk-core    1.11.476    Apache License, Version 2.0
                com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
                com.fasterxml.jackson.dataformat    jackson-dataformat-cbor    2.6.7    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
                commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
                joda-time    joda-time    2.8.1    Apache 2
                org.apache.httpcomponents    httpclient    4.5.5    Apache License, Version 2.0
                    commons-codec    commons-codec    1.10    Apache License, Version 2.0
                    commons-logging    commons-logging    1.1.3    The Apache Software License, Version 2.0
                    org.apache.httpcomponents    httpcore    4.4.9    Apache License, Version 2.0
                software.amazon.ion    ion-java    1.0.2    The Apache License, Version 2.0
            com.amazonaws    jmespath-java    1.11.476    Apache License, Version 2.0
                com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                    com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
        com.amazonaws    jmespath-java    1.11.476    Apache License, Version 2.0
            com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
                com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
    com.amazonaws    jmespath-java    1.11.476    Apache License, Version 2.0
        com.fasterxml.jackson.core    jackson-databind    2.6.7.2    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-annotations    2.6.0    The Apache Software License, Version 2.0
            com.fasterxml.jackson.core    jackson-core    2.6.7    The Apache Software License, Version 2.0
com.typesafe.akka    akka-http_2.12    10.1.10    Apache-2.0
    com.typesafe.akka    akka-http-core_2.12    10.1.10    Apache-2.0
        com.typesafe.akka    akka-parsing_2.12    10.1.10    Apache-2.0
            org.scala-lang    scala-library    2.12.7    BSD 3-Clause
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    org.scala-lang    scala-library    2.12.7    BSD 3-Clause
com.typesafe.akka    akka-stream_2.12    2.5.23    Apache License, Version 2.0
    com.typesafe.akka    akka-actor_2.12    2.5.23    Apache License, Version 2.0
        com.typesafe    config    1.3.3    Apache License, Version 2.0
        org.scala-lang.modules    scala-java8-compat_2.12    0.8.0    BSD 3-clause
            org.scala-lang    scala-library    2.12.7    BSD 3-Clause
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    com.typesafe.akka    akka-protobuf_2.12    2.5.23    Apache License, Version 2.0
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    com.typesafe    ssl-config-core_2.12    0.3.7    Apache-2.0
        com.typesafe    config    1.3.3    Apache License, Version 2.0
        org.scala-lang.modules    scala-parser-combinators_2.12    1.1.1    BSD 3-clause
            org.scala-lang    scala-library    2.12.7    BSD 3-Clause
        org.scala-lang    scala-library    2.12.7    BSD 3-Clause
    org.reactivestreams    reactive-streams    1.0.2    CC0
    org.scala-lang    scala-library    2.12.7    BSD 3-Clause
org.scala-lang    scala-library    2.12.7    BSD 3-Clause

Setup

Factories provided in the DynamoDb will use the client managed by the extension. The managed client will be created using the configuration resolved from reference.conf and with overrides from application.conf.

Example application.conf
akka.stream.alpakka.dynamodb {
 region = "eu-west-1"
 host = "localhost"
 port = 443
 tls = true
 parallelism = 32
 credentials {
   access-key-id = "dummy-access-key"
   secret-key-id = "dummy-secret-key"
 }
reference.conf
akka.stream.alpakka.dynamodb {
  # The AWS region
  region = ""

  # The AWS host
  host = ""

  # The AWS port, https is used as default scheme
  port = 443

  # Use Transport Level Security (https)
  tls = true

  # Max number of in flight requests from the AwsClient - must be a power of 2
  parallelism = 32

  # Optional max number of outstanding requests to the underlying host connection pool.
  # If unspecified, will be the same as the parallelism value. Must be a power of 2
  #max-open-requests =

  # Optional Credentials. Used to define static credentials rather than use the DefaultAWSCredentialsProviderChain
  #credentials {
  #  access-key-id = "dummy-access-key"
  #  secret-key-id = "dummy-secret-key"
  #}
}

If the credentials are not set in the configuration, connector will use the default credential provider chain provided by the DynamoDB Java SDK to retrieve credentials.

Usage

For simple operations you can issue a single request, and get back the result in a FutureCompletionStage.

Scala
val listTablesResult: Future[ListTablesResult] =
  DynamoDb.single(new ListTablesRequest())
Java
final Source<ListTablesResult, NotUsed> listTables =
    DynamoDb.listTables(new ListTablesRequest());

You can also get the response to a request as an element emitted from a Flow:

Scala
import akka.stream.alpakka.dynamodb.AwsOp._
val source: Source[String, NotUsed] = Source
  .single[CreateTable](new CreateTableRequest().withTableName("testTable"))
  .via(DynamoDb.flow)
  .map(_.getTableDescription.getTableArn)
Java
Source<String, NotUsed> tableArnSource =
    Source.single(AwsOp.create(new CreateTableRequest().withTableName("testTable")))
        .via(DynamoDb.flow())
        .map(r -> (CreateTableResult) r)
        .map(result -> result.getTableDescription().getTableArn());

Some DynamoDB operations, such as Query and Scan, are paginated by nature. This is how you can get a stream of all result pages:

Scala
val scanPages: Source[ScanResult, NotUsed] =
  DynamoDb.source(new ScanRequest().withTableName("testTable"))
Java
Source<ScanResult, NotUsed> scanPages =
    DynamoDb.scanAll(new ScanRequest().withTableName("testTable"));

A custom configured client can be used by attaching it as an attribute to the stream:

Scala
val settings = DynamoSettings(system).withRegion("custom-region")
val client = DynamoClient(settings)

val source: Source[ListTablesResult, NotUsed] =
  DynamoDb
    .source(new ListTablesRequest())
    .withAttributes(DynamoAttributes.client(client))
Java
final DynamoSettings settings = DynamoSettings.create(system).withRegion("custom-region");
final DynamoClient client = DynamoClient.create(settings, system, materializer);

final Source<ListTablesResult, NotUsed> source =
    DynamoDb.listTables(new ListTablesRequest())
        .withAttributes(DynamoAttributes.client(client));

Running the example code

The code in this guide is part of runnable tests of this project. You are welcome to edit the code and run it in sbt.

Test code requires DynamoDB running in the background. You can start one quickly using docker:

docker-compose up dynamodb

Scala
sbt
> dynamodb/testOnly *Spec
Java
sbt
> dynamodb/testOnly *Test
Found an error in this documentation? The source code for this page can be found here. Please feel free to edit and contribute a pull request.