Hadoop Distributed File System - HDFS
The connector offers Flows and Sources that interact with HDFS file systems.
For more information about Hadoop, please visit the Hadoop documentation.
Project Info: Alpakka HDFS | |
---|---|
Artifact | com.lightbend.akka
akka-stream-alpakka-hdfs
1.1.2
|
JDK versions | OpenJDK 8 |
Scala versions | 2.12.7, 2.11.12 |
JPMS module name | akka.stream.alpakka.hdfs |
License | |
Readiness level |
Since 0.20, 2018-07-04
|
Home page | https://doc.akka.io/docs/alpakka/current |
API documentation | |
Forums | |
Release notes | In the documentation |
Issues | Github issues |
Sources | https://github.com/akka/alpakka |
Artifacts
- sbt
libraryDependencies += "com.lightbend.akka" %% "akka-stream-alpakka-hdfs" % "1.1.2"
- Maven
<dependency> <groupId>com.lightbend.akka</groupId> <artifactId>akka-stream-alpakka-hdfs_2.12</artifactId> <version>1.1.2</version> </dependency>
- Gradle
dependencies { compile group: 'com.lightbend.akka', name: 'akka-stream-alpakka-hdfs_2.12', version: '1.1.2' }
The table below shows direct dependencies of this module and the second tab shows all libraries it depends on transitively.
- Direct dependencies
Organization Artifact Version License com.typesafe.akka akka-stream_2.12 2.5.23 Apache License, Version 2.0 org.apache.hadoop hadoop-client 3.1.1 Apache License, Version 2.0 org.scala-lang scala-library 2.12.7 BSD 3-Clause org.typelevel cats-core_2.12 1.6.0 MIT - Dependency tree
com.typesafe.akka akka-stream_2.12 2.5.23 Apache License, Version 2.0 com.typesafe.akka akka-actor_2.12 2.5.23 Apache License, Version 2.0 com.typesafe config 1.3.3 Apache License, Version 2.0 org.scala-lang.modules scala-java8-compat_2.12 0.8.0 BSD 3-clause org.scala-lang scala-library 2.12.7 BSD 3-Clause org.scala-lang scala-library 2.12.7 BSD 3-Clause com.typesafe.akka akka-protobuf_2.12 2.5.23 Apache License, Version 2.0 org.scala-lang scala-library 2.12.7 BSD 3-Clause com.typesafe ssl-config-core_2.12 0.3.7 Apache-2.0 com.typesafe config 1.3.3 Apache License, Version 2.0 org.scala-lang.modules scala-parser-combinators_2.12 1.1.1 BSD 3-clause org.scala-lang scala-library 2.12.7 BSD 3-Clause org.scala-lang scala-library 2.12.7 BSD 3-Clause org.reactivestreams reactive-streams 1.0.2 CC0 org.scala-lang scala-library 2.12.7 BSD 3-Clause org.apache.hadoop hadoop-client 3.1.1 Apache License, Version 2.0 org.apache.hadoop hadoop-annotations 3.1.1 Apache License, Version 2.0 org.apache.hadoop hadoop-common 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.woodstox woodstox-core 5.0.3 The Apache License, Version 2.0 org.codehaus.woodstox stax2-api 3.1.4 The BSD License com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.code.gson gson 2.2.4 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license com.google.re2j re2j 1.1 The Go license com.sun.jersey jersey-servlet 1.19 CDDL 1.1 commons-beanutils commons-beanutils 1.9.3 Apache License, Version 2.0 commons-collections commons-collections 3.2.2 Apache License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 commons-cli commons-cli 1.2 The Apache Software License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-collections commons-collections 3.2.2 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 commons-net commons-net 3.6 Apache License, Version 2.0 javax.servlet.jsp jsp-api 2.1 org.apache.avro avro 1.7.7 The Apache Software License, Version 2.0 com.thoughtworks.paranamer paranamer 2.3 BSD org.apache.commons commons-compress 1.4.1 The Apache Software License, Version 2.0 org.tukaani xz 1.0 Public Domain org.codehaus.jackson jackson-core-asl 1.9.13 The Apache Software License, Version 2.0 org.codehaus.jackson jackson-mapper-asl 1.9.13 The Apache Software License, Version 2.0 org.codehaus.jackson jackson-core-asl 1.9.13 The Apache Software License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.xerial.snappy snappy-java 1.0.5 The Apache Software License, Version 2.0 org.apache.commons commons-compress 1.4.1 The Apache Software License, Version 2.0 org.tukaani xz 1.0 Public Domain org.apache.commons commons-configuration2 2.1.1 Apache License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.commons commons-lang3 3.4 Apache License, Version 2.0 org.apache.commons commons-math3 3.1.1 The Apache Software License, Version 2.0 org.apache.curator curator-client 2.12.0 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.curator curator-recipes 2.12.0 The Apache Software License, Version 2.0 org.apache.curator curator-framework 2.12.0 The Apache Software License, Version 2.0 org.apache.curator curator-client 2.12.0 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-annotations 3.1.1 Apache License, Version 2.0 org.apache.hadoop hadoop-auth 3.1.1 Apache License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD commons-codec commons-codec 1.11 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.curator curator-framework 2.12.0 The Apache Software License, Version 2.0 org.apache.curator curator-client 2.12.0 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.httpcomponents httpclient 4.5.2 Apache License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.httpcomponents httpcore 4.4.4 Apache License, Version 2.0 org.apache.kerby kerb-simplekdc 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-admin 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-server 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-identity 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-xdr 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-client 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby token-provider 1.0.1 Apache License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.apache.htrace htrace-core4 4.1.0-incubating The Apache Software License, Version 2.0 org.apache.httpcomponents httpclient 4.5.2 Apache License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.httpcomponents httpcore 4.4.4 Apache License, Version 2.0 org.apache.kerby kerb-simplekdc 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-admin 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-server 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-identity 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-xdr 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-client 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby token-provider 1.0.1 Apache License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.codehaus.woodstox stax2-api 3.1.4 The BSD License org.eclipse.jetty jetty-servlet 9.3.19.v20170502 Apache Software License - Version 2.0 org.eclipse.jetty jetty-security 9.3.19.v20170502 Apache Software License - Version 2.0 org.eclipse.jetty jetty-webapp 9.3.19.v20170502 Apache Software License - Version 2.0 org.eclipse.jetty jetty-servlet 9.3.19.v20170502 Apache Software License - Version 2.0 org.eclipse.jetty jetty-security 9.3.19.v20170502 Apache Software License - Version 2.0 org.eclipse.jetty jetty-xml 9.3.19.v20170502 Apache Software License - Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-hdfs-client 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.squareup.okhttp okhttp 2.7.5 Apache 2.0 com.squareup.okio okio 1.6.0 Apache 2.0 org.apache.hadoop hadoop-mapreduce-client-core 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license org.apache.hadoop hadoop-hdfs-client 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.squareup.okhttp okhttp 2.7.5 Apache 2.0 com.squareup.okio okio 1.6.0 Apache 2.0 org.apache.hadoop hadoop-yarn-client 3.1.1 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.hadoop hadoop-yarn-common 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.jaxrs jackson-jaxrs-json-provider 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.jaxrs jackson-jaxrs-base 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.module jackson-module-jaxb-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.module jackson-module-jaxb-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license com.sun.jersey jersey-client 1.19 CDDL 1.1 com.sun.jersey jersey-core 1.19 CDDL 1.1 javax.ws.rs jsr311-api 1.1.1 CDDL License commons-cli commons-cli 1.2 The Apache Software License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 javax.servlet javax.servlet-api 3.1.0 CDDL + GPLv2 with classpath exception javax.xml.bind jaxb-api 2.2.11 CDDL 1.1 org.apache.commons commons-compress 1.4.1 The Apache Software License, Version 2.0 org.tukaani xz 1.0 Public Domain org.apache.hadoop hadoop-auth 3.1.1 Apache License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD commons-codec commons-codec 1.11 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.curator curator-framework 2.12.0 The Apache Software License, Version 2.0 org.apache.curator curator-client 2.12.0 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.httpcomponents httpclient 4.5.2 Apache License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.httpcomponents httpcore 4.4.4 Apache License, Version 2.0 org.apache.kerby kerb-simplekdc 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-admin 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-server 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-identity 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-xdr 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-client 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby token-provider 1.0.1 Apache License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-yarn-api 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 javax.xml.bind jaxb-api 2.2.11 CDDL 1.1 org.eclipse.jetty jetty-util 9.3.19.v20170502 Apache Software License - Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-mapreduce-client-jobclient 3.1.1 Apache License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license org.apache.hadoop hadoop-mapreduce-client-common 3.1.1 Apache License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license org.apache.hadoop hadoop-mapreduce-client-core 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license org.apache.hadoop hadoop-hdfs-client 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.squareup.okhttp okhttp 2.7.5 Apache 2.0 com.squareup.okio okio 1.6.0 Apache 2.0 org.apache.hadoop hadoop-yarn-client 3.1.1 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.hadoop hadoop-yarn-common 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.jaxrs jackson-jaxrs-json-provider 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.jaxrs jackson-jaxrs-base 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.module jackson-module-jaxb-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.module jackson-module-jaxb-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license com.sun.jersey jersey-client 1.19 CDDL 1.1 com.sun.jersey jersey-core 1.19 CDDL 1.1 javax.ws.rs jsr311-api 1.1.1 CDDL License commons-cli commons-cli 1.2 The Apache Software License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 javax.servlet javax.servlet-api 3.1.0 CDDL + GPLv2 with classpath exception javax.xml.bind jaxb-api 2.2.11 CDDL 1.1 org.apache.commons commons-compress 1.4.1 The Apache Software License, Version 2.0 org.tukaani xz 1.0 Public Domain org.apache.hadoop hadoop-auth 3.1.1 Apache License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD commons-codec commons-codec 1.11 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.curator curator-framework 2.12.0 The Apache Software License, Version 2.0 org.apache.curator curator-client 2.12.0 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.httpcomponents httpclient 4.5.2 Apache License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.httpcomponents httpcore 4.4.4 Apache License, Version 2.0 org.apache.kerby kerb-simplekdc 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-admin 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-server 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-identity 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-xdr 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-client 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby token-provider 1.0.1 Apache License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-yarn-api 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 javax.xml.bind jaxb-api 2.2.11 CDDL 1.1 org.eclipse.jetty jetty-util 9.3.19.v20170502 Apache Software License - Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-yarn-client 3.1.1 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.hadoop hadoop-yarn-common 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.jaxrs jackson-jaxrs-json-provider 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.jaxrs jackson-jaxrs-base 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.module jackson-module-jaxb-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.module jackson-module-jaxb-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-databind 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.fasterxml.jackson.core jackson-core 2.7.8 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license com.sun.jersey jersey-client 1.19 CDDL 1.1 com.sun.jersey jersey-core 1.19 CDDL 1.1 javax.ws.rs jsr311-api 1.1.1 CDDL License commons-cli commons-cli 1.2 The Apache Software License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 javax.servlet javax.servlet-api 3.1.0 CDDL + GPLv2 with classpath exception javax.xml.bind jaxb-api 2.2.11 CDDL 1.1 org.apache.commons commons-compress 1.4.1 The Apache Software License, Version 2.0 org.tukaani xz 1.0 Public Domain org.apache.hadoop hadoop-auth 3.1.1 Apache License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD commons-codec commons-codec 1.11 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.curator curator-framework 2.12.0 The Apache Software License, Version 2.0 org.apache.curator curator-client 2.12.0 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.httpcomponents httpclient 4.5.2 Apache License, Version 2.0 commons-codec commons-codec 1.11 Apache License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.apache.httpcomponents httpcore 4.4.4 Apache License, Version 2.0 org.apache.kerby kerb-simplekdc 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-admin 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-server 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-identity 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-xdr 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-client 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-common 1.0.1 Apache License, Version 2.0 commons-io commons-io 2.5 Apache License, Version 2.0 org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerb-crypto 1.0.1 Apache License, Version 2.0 org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby kerby-config 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.apache.kerby token-provider 1.0.1 Apache License, Version 2.0 com.nimbusds nimbus-jose-jwt 4.41.1 The Apache Software License, Version 2.0 com.github.stephenc.jcip jcip-annotations 1.0-1 Apache License, Version 2.0 net.minidev json-smart 2.3 The Apache Software License, Version 2.0 net.minidev accessors-smart 1.2 The Apache Software License, Version 2.0 org.ow2.asm asm 5.0.4 BSD org.apache.kerby kerb-core 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-pkix 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-asn1 1.0.1 Apache License, Version 2.0 org.apache.kerby kerby-util 1.0.1 Apache License, Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-yarn-api 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 javax.xml.bind jaxb-api 2.2.11 CDDL 1.1 org.eclipse.jetty jetty-util 9.3.19.v20170502 Apache Software License - Version 2.0 org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.slf4j slf4j-api 1.7.25 MIT License org.apache.hadoop hadoop-yarn-api 3.1.1 Apache License, Version 2.0 com.fasterxml.jackson.core jackson-annotations 2.7.8 The Apache Software License, Version 2.0 com.google.guava guava 11.0.2 The Apache Software License, Version 2.0 com.google.code.findbugs jsr305 3.0.0 The Apache Software License, Version 2.0 com.google.protobuf protobuf-java 2.5.0 New BSD license commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 javax.xml.bind jaxb-api 2.2.11 CDDL 1.1 org.apache.hadoop hadoop-yarn-client 3.1.1 Apache License, Version 2.0 commons-lang commons-lang 2.6 The Apache Software License, Version 2.0 commons-logging commons-logging 1.1.3 The Apache Software License, Version 2.0 org.scala-lang scala-library 2.12.7 BSD 3-Clause org.typelevel cats-core_2.12 1.6.0 MIT org.scala-lang scala-library 2.12.7 BSD 3-Clause org.typelevel cats-kernel_2.12 1.6.0 MIT org.scala-lang scala-library 2.12.7 BSD 3-Clause org.typelevel cats-macros_2.12 1.6.0 MIT org.scala-lang scala-library 2.12.7 BSD 3-Clause org.typelevel machinist_2.12 0.6.6 MIT org.scala-lang scala-library 2.12.7 BSD 3-Clause org.scala-lang scala-reflect 2.12.7 BSD 3-Clause org.scala-lang scala-library 2.12.7 BSD 3-Clause org.typelevel machinist_2.12 0.6.6 MIT org.scala-lang scala-library 2.12.7 BSD 3-Clause org.scala-lang scala-reflect 2.12.7 BSD 3-Clause org.scala-lang scala-library 2.12.7 BSD 3-Clause
Specifying a Hadoop Version
By default, HDFS connector uses Hadoop 3.1.1. If you are using a different version of Hadoop, you should exclude the Hadoop libraries from the connector dependency and add the dependency for your preferred version.
Set up client
Flows provided by this connector need a prepared org.apache.hadoop.fs.FileSystem
to interact with HDFS.
- Scala
-
import org.apache.hadoop.conf.Configuration import org.apache.hadoop.fs.FileSystem val conf = new Configuration() conf.set("fs.default.name", "hdfs://localhost:54310") val fs: FileSystem = FileSystem.get(conf)
- Java
-
Configuration conf = new Configuration(); conf.set("fs.default.name", "hdfs://localhost:54310"); fs = FileSystem.get(conf);
Writing
The connector provides three Flows. Each flow requires RotationStrategy
and SyncStrategy
to run. HdfsFlow
. HdfsFlow
.
The flows push OutgoingMessage
to a downstream.
Data Writer
Use HdfsFlow.data
to stream with FSDataOutputStream
without any compression.
- Scala
-
val flow = HdfsFlow.data( fs, SyncStrategy.count(500), RotationStrategy.size(1, FileUnit.GB), HdfsWritingSettings() )
- Java
-
Flow<HdfsWriteMessage<ByteString, NotUsed>, RotationMessage, NotUsed> flow = HdfsFlow.data( fs, SyncStrategy.count(500), RotationStrategy.size(1, FileUnit.GB()), settings);
Compressed Data Writer
First, create CompressionCodec
.
- Scala
-
val codec = new DefaultCodec() codec.setConf(fs.getConf)
- Java
-
DefaultCodec codec = new DefaultCodec(); codec.setConf(fs.getConf());
Then, use HdfsFlow.compress
to stream with CompressionOutputStream
and CompressionCodec
.
- Scala
-
val flow = HdfsFlow.compressed( fs, SyncStrategy.count(1), RotationStrategy.size(0.1, FileUnit.MB), codec, settings )
- Java
-
Flow<HdfsWriteMessage<ByteString, NotUsed>, RotationMessage, NotUsed> flow = HdfsFlow.compressed( fs, SyncStrategy.count(50), RotationStrategy.size(0.1, FileUnit.MB()), codec, settings);
Sequence Writer
Use HdfsFlow.sequence
to stream a flat file consisting of binary key/value pairs.
Without Compression
- Scala
-
val flow = HdfsFlow.sequence( fs, SyncStrategy.none, RotationStrategy.size(1, FileUnit.MB), settings, classOf[Text], classOf[Text] )
- Java
-
Flow<HdfsWriteMessage<Pair<Text, Text>, NotUsed>, RotationMessage, NotUsed> flow = HdfsFlow.sequence( fs, SyncStrategy.none(), RotationStrategy.size(1, FileUnit.MB()), SequenceFile.CompressionType.BLOCK, codec, settings, Text.class, Text.class);
With Compression
First, define a codec.
- Scala
-
val codec = new DefaultCodec() codec.setConf(fs.getConf)
- Java
-
DefaultCodec codec = new DefaultCodec(); codec.setConf(fs.getConf());
Then, create a flow.
- Scala
-
val flow = HdfsFlow.sequence( fs, SyncStrategy.none, RotationStrategy.size(1, FileUnit.MB), CompressionType.BLOCK, codec, settings, classOf[Text], classOf[Text] )
- Java
-
Flow<HdfsWriteMessage<Pair<Text, Text>, NotUsed>, RotationMessage, NotUsed> flow = HdfsFlow.sequence( fs, SyncStrategy.none(), RotationStrategy.size(1, FileUnit.MB()), settings, Text.class, Text.class);
Passing data through HdfsFlow
Use HdfsFlow.dataWithPassThrough
, HdfsFlow.compressedWithPassThrough
or HdfsFlow.sequenceWithPassThrough
.
When streaming documents from Kafka, you might want to commit to Kafka. The flow will emit two messages. For every input, it will produce WrittenMessage
and when it rotates, RotationMessage
.
Let’s say that we have these classes.
- Scala
-
case class Book(title: String) case class KafkaOffset(offset: Int) case class KafkaMessage(book: Book, offset: KafkaOffset)
- Java
-
public static class Book { final String title; Book(String title) { this.title = title; } } static class KafkaCommitter { List<Integer> committedOffsets = new ArrayList<>(); void commit(KafkaOffset offset) { committedOffsets.add(offset.offset); } } static class KafkaOffset { final int offset; KafkaOffset(int offset) { this.offset = offset; } } static class KafkaMessage { final Book book; final KafkaOffset offset; KafkaMessage(Book book, KafkaOffset offset) { this.book = book; this.offset = offset; } }
Then, we can stream with passThrough
.
- Scala
-
// We're going to pretend we got messages from kafka. // After we've written them to HDFS, we want // to commit the offset to Kafka val messagesFromKafka = List( KafkaMessage(Book("Akka Concurrency"), KafkaOffset(0)), KafkaMessage(Book("Akka in Action"), KafkaOffset(1)), KafkaMessage(Book("Effective Akka"), KafkaOffset(2)), KafkaMessage(Book("Learning Scala"), KafkaOffset(3)), KafkaMessage(Book("Scala Puzzlers"), KafkaOffset(4)), KafkaMessage(Book("Scala for Spark in Production"), KafkaOffset(5)) ) var committedOffsets = List[KafkaOffset]() def commitToKafka(offset: KafkaOffset): Unit = committedOffsets = committedOffsets :+ offset val resF = Source(messagesFromKafka) .map { kafkaMessage: KafkaMessage => val book = kafkaMessage.book // Transform message so that we can write to hdfs HdfsWriteMessage(ByteString(book.title), kafkaMessage.offset) } .via( HdfsFlow.dataWithPassThrough[KafkaOffset]( fs, SyncStrategy.count(50), RotationStrategy.count(4), HdfsWritingSettings().withNewLine(true) ) ) .map { message => message match { case WrittenMessage(passThrough, _) => commitToKafka(passThrough) case _ => () } message } .collect { case rm: RotationMessage => rm } .runWith(Sink.seq)
- Java
-
// We're going to pretend we got messages from kafka. // After we've written them to HDFS, we want // to commit the offset to Kafka List<KafkaMessage> messagesFromKafka = Arrays.asList( new KafkaMessage(new Book("Akka Concurrency"), new KafkaOffset(0)), new KafkaMessage(new Book("Akka in Action"), new KafkaOffset(1)), new KafkaMessage(new Book("Effective Akka"), new KafkaOffset(2)), new KafkaMessage(new Book("Learning Scala"), new KafkaOffset(3)), new KafkaMessage(new Book("Scala Puzzlers"), new KafkaOffset(4)), new KafkaMessage(new Book("Scala for Spark in Production"), new KafkaOffset(5))); final KafkaCommitter kafkaCommitter = new KafkaCommitter(); Flow<HdfsWriteMessage<ByteString, KafkaOffset>, OutgoingMessage<KafkaOffset>, NotUsed> flow = HdfsFlow.dataWithPassThrough( fs, SyncStrategy.count(50), RotationStrategy.count(4), HdfsWritingSettings.create().withNewLine(true)); CompletionStage<List<RotationMessage>> resF = Source.from(messagesFromKafka) .map( kafkaMessage -> { Book book = kafkaMessage.book; // Transform message so that we can write to hdfs\ return HdfsWriteMessage.create( ByteString.fromString(book.title), kafkaMessage.offset); }) .via(flow) .map( message -> { if (message instanceof WrittenMessage) { kafkaCommitter.commit(((WrittenMessage<KafkaOffset>) message).passThrough()); return message; } else { return message; } }) .collectType(RotationMessage.class) // Collect only rotation messages .runWith(Sink.seq(), materializer);
Configuration
We can configure the sink by HdfsWritingSettings
.
- Scala
-
val settings = HdfsWritingSettings() .withOverwrite(true) .withNewLine(false) .withLineSeparator(System.getProperty("line.separator")) .withPathGenerator(pathGenerator)
- Java
-
HdfsWritingSettings.create() .withOverwrite(true) .withNewLine(false) .withLineSeparator(System.getProperty("line.separator")) .withPathGenerator(pathGenerator);
File path generator
FilePathGenerator
provides a functionality to generate rotation path in HDFS.
- Scala
-
val pathGenerator = FilePathGenerator( (rotationCount: Long, timestamp: Long) => s"/tmp/alpakka/$rotationCount-$timestamp" )
- Java
-
BiFunction<Long, Long, String> func = (rotationCount, timestamp) -> "/tmp/alpakka/" + rotationCount + "-" + timestamp; FilePathGenerator pathGenerator = FilePathGenerator.create(func);
Rotation Strategy
RotationStrategy
provides a functionality to decide when to rotate files.
Sync Strategy
SyncStrategy
provides a functionality to decide when to synchronize the output.
Reading
Use HdfsSource
to read from HDFS. HdfsSource
. HdfsSource
.
Data Reader
- Scala
-
val source = HdfsSource.data(fs, path)
- Java
-
Source<ByteString, CompletionStage<IOResult>> source = HdfsSource.data(fs, path);
Compressed Data Reader
- Scala
-
val source = HdfsSource.compressed(fs, path, codec)
- Java
-
Source<ByteString, CompletionStage<IOResult>> source = HdfsSource.compressed(fs, path, codec);
Sequence Reader
- Scala
-
val source = HdfsSource.sequence(fs, path, classOf[Text], classOf[Text])
- Java
-
Source<Pair<Text, Text>, NotUsed> source = HdfsSource.sequence(fs, path, Text.class, Text.class);
Running the example code
The code in this guide is part of runnable tests of this project. You are welcome to edit the code and run it in sbt.
- Scala
-
sbt > hdfs/testOnly *.HdfsWriterSpec > hdfs/testOnly *.HdfsReaderSpec
- Java
-
sbt > hdfs/testOnly *.HdfsWriterTest > hdfs/testOnly *.HdfsReaderTest