Operators
Source operators
These built-in sources are available from akka.stream.scaladsl.Source
akka.stream.javadsl.Source
:
Operator | Description | |
---|---|---|
Source | actorRef | Materialize an ActorRef ; sending messages to it will emit them on the stream. |
Source | actorRefWithAck | Materialize an ActorRef ; sending messages to it will emit them on the stream. The source acknowledges reception after emitting a message, to provide back pressure from the source. |
ActorSource | actorRefWithAck | Materialize an ActorRef<T> ActorRef[T] ; sending messages to it will emit them on the stream. The source acknowledges reception after emitting a message, to provide back pressure from the source. |
Source | asSourceWithContext | Turns a Source into a SourceWithContext which can propagate a context per element along a stream. |
Source | asSubscriber | Integration with Reactive Streams, materializes into a org.reactivestreams.Subscriber . |
Source | combine | Combine several sources, using a given strategy such as merge or concat, into one source. |
Source | cycle | Stream iterator in cycled manner. |
Source | empty | Complete right away without ever emitting any elements. |
Source | failed | Fail directly with a user specified exception. |
Source | from | Stream the values of an Iterable . |
Source | fromCompletionStage | Send the single value of the CompletionStage when it completes and there is demand. |
Source | fromFuture | Send the single value of the Future when it completes and there is demand. |
Source | fromFutureSource | Streams the elements of the given future source once it successfully completes. |
Source | fromIterator | Stream the values from an Iterator , requesting the next value when there is demand. |
Source | fromPublisher | Integration with Reactive Streams, subscribes to a org.reactivestreams.Publisher . |
Source | fromSourceCompletionStage | Streams the elements of an asynchronous source once its given completion operator completes. |
Source | lazily | Defers creation and materialization of a Source until there is demand. |
Source | lazilyAsync | Defers creation and materialization of a CompletionStage until there is demand. |
Source | maybe | Materialize a Promise[Option[T]] CompletionStage that if completed with a Some[T] Optional will emit that T and then complete the stream, or if completed with None empty Optional complete the stream right away. |
Source | queue | Materialize a SourceQueue onto which elements can be pushed for emitting from the source. |
Source | range | Emit each integer in a range, with an option to take bigger steps than 1. |
Source | repeat | Stream a single object repeatedly |
Source | single | Stream a single object |
Source | tick | A periodical repetition of an arbitrary object. |
Source | unfold | Stream the result of a function as long as it returns a Some Optional . |
Source | unfoldAsync | Just like unfold but the fold function returns a Future CompletionStage . |
Source | unfoldResource | Wrap any resource that can be opened, queried for next element (in a blocking way) and closed using three distinct functions into a source. |
Source | unfoldResourceAsync | Wrap any resource that can be opened, queried for next element (in a blocking way) and closed using three distinct functions into a source. |
Source | zipN | Combine the elements of multiple streams into a stream of sequences. |
Source | zipWithN | Combine the elements of multiple streams into a stream of sequences using a combiner function. |
Sink operators
These built-in sinks are available from akka.stream.scaladsl.Sink
akka.stream.javadsl.Sink
:
Operator | Description | |
---|---|---|
Sink | actorRef | Send the elements from the stream to an ActorRef . |
Sink | actorRefWithAck | Send the elements from the stream to an ActorRef which must then acknowledge reception after completing a message, to provide back pressure onto the sink. |
Sink | asPublisher | Integration with Reactive Streams, materializes into a org.reactivestreams.Publisher . |
Sink | cancelled | Immediately cancel the stream |
Sink | combine | Combine several sinks into one using a user specified strategy |
Sink | fold | Fold over emitted element with a function, where each invocation will get the new element and the result from the previous fold invocation. |
Sink | foreach | Invoke a given procedure for each element received. |
Sink | foreachAsync | Invoke a given procedure asynchronously for each element received. |
Sink | foreachParallel | Like foreach but allows up to parallellism procedure calls to happen in parallel. |
Sink | fromSubscriber | Integration with Reactive Streams, wraps a org.reactivestreams.Subscriber as a sink. |
Sink | head | Materializes into a Future CompletionStage which completes with the first value arriving, after this the stream is canceled. |
Sink | headOption | Materializes into a Future[Option[T]] CompletionStage<Optional<T>> which completes with the first value arriving wrapped in Some Optional , or a None an empty Optional if the stream completes without any elements emitted. |
Sink | ignore | Consume all elements but discards them. |
Sink | last | Materializes into a Future CompletionStage which will complete with the last value emitted when the stream completes. |
Sink | lastOption | Materialize a Future[Option[T]] CompletionStage<Optional<T>> which completes with the last value emitted wrapped in an Some Optional when the stream completes. |
Sink | lazyInitAsync | Creates a real Sink upon receiving the first element. |
Sink | onComplete | Invoke a callback when the stream has completed or failed. |
Sink | preMaterialize | Materializes this Sink, immediately returning (1) its materialized value, and (2) a new Sink that can be consume elements ‘into’ the pre-materialized one. |
Sink | queue | Materialize a SinkQueue that can be pulled to trigger demand through the sink. |
Sink | reduce | Apply a reduction function on the incoming elements and pass the result to the next invocation. |
Sink | seq | Collect values emitted from the stream into a collection. |
Sink | setup | Defer the creation of a Sink until materialization and access ActorMaterializer and Attributes |
Sink | takeLast | Collect the last n values emitted from the stream into a collection. |
Additional Sink and Source converters
Sources and sinks for integrating with java.io.InputStream
and java.io.OutputStream
can be found on StreamConverters
. As they are blocking APIs the implementations of these operators are run on a separate dispatcher configured through the akka.stream.blocking-io-dispatcher
.
Be aware that asInputStream
and asOutputStream
materialize InputStream
and OutputStream
respectively as blocking API implementation. They will block the thread until data will be available from upstream. Because of blocking nature these objects cannot be used in mapMaterializeValue
section as it causes deadlock of the stream materialization process. For example, following snippet will fall with timeout exception:
...
.toMat(StreamConverters.asInputStream().mapMaterializedValue { inputStream =>
inputStream.read() // this could block forever
...
}).run()
Operator | Description | |
---|---|---|
StreamConverters | asInputStream | Create a sink which materializes into an InputStream that can be read to trigger demand through the sink. |
StreamConverters | asJavaStream | Create a sink which materializes into Java 8 Stream that can be run to trigger demand through the sink. |
StreamConverters | asOutputStream | Create a source that materializes into an OutputStream . |
StreamConverters | fromInputStream | Create a source that wraps an InputStream . |
StreamConverters | fromJavaStream | Create a source that wraps a Java 8 Stream . |
StreamConverters | fromOutputStream | Create a sink that wraps an OutputStream . |
StreamConverters | javaCollector | Create a sink which materializes into a Future CompletionStage which will be completed with a result of the Java 8 Collector transformation and reduction operations. |
StreamConverters | javaCollectorParallelUnordered | Create a sink which materializes into a Future CompletionStage which will be completed with a result of the Java 8 Collector transformation and reduction operations. |
File IO Sinks and Sources
Sources and sinks for reading and writing files can be found on FileIO
.
Operator | Description | |
---|---|---|
FileIO | fromFile | Emits the contents of a file. |
FileIO | fromPath | Emits the contents of a file from the given path. |
FileIO | toFile | Create a sink which will write incoming ByteString s to a given file. |
FileIO | toPath | Create a sink which will write incoming ByteString s to a given file path. |
Simple operators
These operators can transform the rate of incoming elements since there are operators that emit multiple elements for a single input (e.g. mapConcat
) or consume multiple elements before emitting one output (e.g. filter
). However, these rate transformations are data-driven, i.e. it is the incoming elements that define how the rate is affected. This is in contrast with detached operators which can change their processing behavior depending on being backpressured by downstream or not.
Operator | Description | |
---|---|---|
Source/Flow | alsoTo | Attaches the given Sink to this Flow , meaning that elements that pass through this Flow will also be sent to the Sink . |
Flow | asFlowWithContext | Turns a Flow into a FlowWithContext which can propagate a context per element along a stream. |
Source/Flow | collect | Apply a partial function to each incoming element, if the partial function is defined for a value the returned value is passed downstream. |
Source/Flow | collectType | Transform this stream by testing the type of each of the elements on which the element is an instance of the provided type as they pass through this processing step. |
Source/Flow | detach | Detach upstream demand from downstream demand without detaching the stream rates. |
Source/Flow | divertTo | Each upstream element will either be diverted to the given sink, or the downstream consumer according to the predicate function applied to the element. |
Source/Flow | drop | Drop n elements and then pass any subsequent element downstream. |
Source/Flow | dropWhile | Drop elements as long as a predicate function return true for the element |
Source/Flow | filter | Filter the incoming elements using a predicate. |
Source/Flow | filterNot | Filter the incoming elements using a predicate. |
Source/Flow | fold | Start with current value zero and then apply the current and next value to the given function. When upstream completes, the current value is emitted downstream. |
Source/Flow | foldAsync | Just like fold but receives a function that results in a Future CompletionStage to the next value. |
Source/Flow | grouped | Accumulate incoming events until the specified number of elements have been accumulated and then pass the collection of elements downstream. |
Source/Flow | intersperse | Intersperse stream with provided element similar to List.mkString . |
Flow | lazyInitAsync | Creates a real Flow upon receiving the first element by calling relevant flowFactory given as an argument. |
Source/Flow | limit | Limit number of element from upstream to given max number. |
Source/Flow | limitWeighted | Ensure stream boundedness by evaluating the cost of incoming elements using a cost function. |
Source/Flow | log | Log elements flowing through the stream as well as completion and erroring. |
Source/Flow | map | Transform each element in the stream by calling a mapping function with it and passing the returned value downstream. |
Source/Flow | mapConcat | Transform each element into zero or more elements that are individually passed downstream. |
Source/Flow | mapError | While similar to recover this operators can be used to transform an error signal to a different one without logging it as an error in the process. |
Source/Flow | recover | Allow sending of one last element downstream when a failure has happened upstream. |
Source/Flow | recoverWith | Allow switching to alternative Source when a failure has happened upstream. |
Source/Flow | recoverWithRetries | RecoverWithRetries allows to switch to alternative Source on flow failure. |
Source/Flow | reduce | Start with first element and then apply the current and next value to the given function, when upstream complete the current value is emitted downstream. |
Source/Flow | scan | Emit its current value, which starts at zero , and then apply the current and next value to the given function, emitting the next current value. |
Source/Flow | scanAsync | Just like scan but receives a function that results in a Future CompletionStage to the next value. |
Source/Flow | setup | Defer the creation of a Source/Flow until materialization and access ActorMaterializer and Attributes |
Source/Flow | sliding | Provide a sliding window over the incoming stream and pass the windows as groups of elements downstream. |
Source/Flow | statefulMapConcat | Transform each element into zero or more elements that are individually passed downstream. |
Source/Flow | take | Pass n incoming elements downstream and then complete |
Source/Flow | takeWhile | Pass elements downstream as long as a predicate function return true for the element include the element when the predicate first return false and then complete. |
Source/Flow | throttle | Limit the throughput to a specific number of elements per time unit, or a specific total cost per time unit, where a function has to be provided to calculate the individual cost of each element. |
Source/Flow | watch | Watch a specific ActorRef and signal a failure downstream once the actor terminates. |
Source/Flow | wireTap | Attaches the given Sink to this Flow as a wire tap, meaning that elements that pass through will also be sent to the wire-tap Sink , without the latter affecting the mainline flow. |
Flow operators composed of Sinks and Sources
Operator | Description | |
---|---|---|
Flow | fromSinkAndSource | Creates a Flow from a Sink and a Source where the Flow’s input will be sent to the Sink and the Flow ’s output will come from the Source. |
Flow | fromSinkAndSourceCoupled | Allows coupling termination (cancellation, completion, erroring) of Sinks and Sources while creating a Flow between them. |
Asynchronous operators
These operators encapsulate an asynchronous computation, properly handling backpressure while taking care of the asynchronous operation at the same time (usually handling the completion of a Future
CompletionStage
).
Operator | Description | |
---|---|---|
Source/Flow | ask | Use the ask pattern to send a request-reply message to the target ref actor. |
Source/Flow | mapAsync | Pass incoming elements to a function that return a Future CompletionStage result. |
Source/Flow | mapAsyncUnordered | Like mapAsync but Future CompletionStage results are passed downstream as they arrive regardless of the order of the elements that triggered them. |
Timer driven operators
These operators process elements using timers, delaying, dropping or grouping elements for certain time durations.
Operator | Description | |
---|---|---|
Source/Flow | delay | Delay every element passed through with a specific duration. |
Source/Flow | dropWithin | Drop elements until a timeout has fired |
Source/Flow | groupedWeightedWithin | Chunk up this stream into groups of elements received within a time window, or limited by the weight of the elements, whatever happens first. |
Source/Flow | groupedWithin | Chunk up this stream into groups of elements received within a time window, or limited by the number of the elements, whatever happens first. |
Source/Flow | initialDelay | Delays the initial element by the specified duration. |
Source/Flow | takeWithin | Pass elements downstream within a timeout and then complete. |
Backpressure aware operators
These operators are aware of the backpressure provided by their downstreams and able to adapt their behavior to that signal.
Operator | Description | |
---|---|---|
Source/Flow | batch | Allow for a slower downstream by passing incoming elements and a summary into an aggregate function as long as there is backpressure and a maximum number of batched elements is not yet reached. |
Source/Flow | batchWeighted | Allow for a slower downstream by passing incoming elements and a summary into an aggregate function as long as there is backpressure and a maximum weight batched elements is not yet reached. |
Source/Flow | buffer | Allow for a temporarily faster upstream events by buffering size elements. |
Source/Flow | conflate | Allow for a slower downstream by passing incoming elements and a summary into an aggregate function as long as there is backpressure. |
Source/Flow | conflateWithSeed | Allow for a slower downstream by passing incoming elements and a summary into an aggregate function as long as there is backpressure. |
Source/Flow | expand | Like extrapolate , but does not have the initial argument, and the Iterator is also used in lieu of the original element, allowing for it to be rewritten and/or filtered. |
Source/Flow | extrapolate | Allow for a faster downstream by expanding the last emitted element to an Iterator . |
Nesting and flattening operators
These operators either take a stream and turn it into a stream of streams (nesting) or they take a stream that contains nested streams and turn them into a stream of elements instead (flattening).
See the Substreams page for more detail and code samples.
Operator | Description | |
---|---|---|
Source/Flow | flatMapConcat | Transform each input element into a Source whose elements are then flattened into the output stream through concatenation. |
Source/Flow | flatMapMerge | Transform each input element into a Source whose elements are then flattened into the output stream through merging. |
Source/Flow | groupBy | Demultiplex the incoming stream into separate output streams. |
Source/Flow | prefixAndTail | Take up to n elements from the stream (less than n only if the upstream completes before emitting n elements) and returns a pair containing a strict sequence of the taken element and a stream representing the remaining elements. |
Source/Flow | splitAfter | End the current substream whenever a predicate returns true , starting a new substream for the next element. |
Source/Flow | splitWhen | Split off elements into a new substream whenever a predicate function return true . |
Time aware operators
Those operators operate taking time into consideration.
Operator | Description | |
---|---|---|
Source/Flow | backpressureTimeout | If the time between the emission of an element and the following downstream demand exceeds the provided timeout, the stream is failed with a TimeoutException . |
Source/Flow | completionTimeout | If the completion of the stream does not happen until the provided timeout, the stream is failed with a TimeoutException . |
Source/Flow | idleTimeout | If the time between two processed elements exceeds the provided timeout, the stream is failed with a TimeoutException . |
Source/Flow | initialTimeout | If the first element has not passed through this operators before the provided timeout, the stream is failed with a TimeoutException . |
Source/Flow | keepAlive | Injects additional (configured) elements if upstream does not emit for a configured amount of time. |
Fan-in operators
These operators take multiple streams as their input and provide a single output combining the elements from all of the inputs in different ways.
Operator | Description | |
---|---|---|
Source/Flow | concat | After completion of the original upstream the elements of the given source will be emitted. |
Source/Flow | interleave | Emits a specifiable number of elements from the original source, then from the provided source and repeats. |
Source/Flow | merge | Merge multiple sources. |
Source/Flow | mergeSorted | Merge multiple sources. |
Source/Flow | orElse | If the primary source completes without emitting any elements, the elements from the secondary source are emitted. |
Source/Flow | prepend | Prepends the given source to the flow, consuming it until completion before the original source is consumed. |
Source/Flow | zip | Combines elements from each of multiple sources into tuples Pair and passes the tuples pairs downstream. |
Source/Flow | zipLatest | Combines elements from each of multiple sources into tuples Pair and passes the tuples pairs downstream, picking always the latest element of each. |
Source/Flow | zipLatestWith | Combines elements from multiple sources through a combine function and passes the returned value downstream, picking always the latest element of each. |
Source/Flow | zipWith | Combines elements from multiple sources through a combine function and passes the returned value downstream. |
Source/Flow | zipWithIndex | Zips elements of current flow with its indices. |
Watching status operators
Operator | Description | |
---|---|---|
Source/Flow | monitor | Materializes to a FlowMonitor that monitors messages flowing through or completion of the operators. |
Source/Flow | watchTermination | Materializes to a Future CompletionStage that will be completed with Done or failed depending whether the upstream of the operators has been completed or failed. |
Actor interop operators
Operators meant for inter-operating between Akka Streams and Actors:
Operator | Description | |
---|---|---|
ActorSink | actorRef | Sends the elements of the stream to the given ActorRef<T> ActorRef[T] . |
ActorFlow | ask | Use the AskPattern to send each element as an ask to the target actor, and expect a reply back that will be sent further downstream. |
Error handling
For more background see the Error Handling in Streams section.
Operator | Description | |
---|---|---|
RestartSource | onFailuresWithBackoff | Wrap the given Source Source with a Source Source that will restart it when it fails using an exponential backoff. |
RestartFlow | onFailuresWithBackoff | Wrap the given Flow Flow with a Flow Flow that will restart it when it fails using an exponential backoff. Notice that this Flow Flow will not restart on completion of the wrapped flow. |
RestartSource | withBackoff | Wrap the given Source Source with a Source Source that will restart it when it fails or complete using an exponential backoff. |
RestartFlow | withBackoff | Wrap the given Flow Flow with a Flow Flow that will restart it when it fails or complete using an exponential backoff. |
RestartSink | withBackoff | Wrap the given Sink Sink with a Sink Sink that will restart it when it fails or complete using an exponential backoff. |