batchWeighted

Allow for a slower downstream by passing incoming elements and a summary into an aggregate function as long as there is backpressure and a maximum weight batched elements is not yet reached.

Backpressure aware operators

Signature

def batchWeighted[S](max: Long, costFn: Out ⇒ Long, seed: Out ⇒ S)(aggregate: (S, Out) ⇒ S): Repr[S]

Description

Allow for a slower downstream by passing incoming elements and a summary into an aggregate function as long as there is backpressure and a maximum weight batched elements is not yet reached. The weight of each element is determined by applying costFn. When the maximum total weight is reached and downstream still backpressures batch will also backpressure.

Will eagerly pull elements, this behavior may result in a single pending (i.e. buffered) element which cannot be aggregated to the batched value.

emits downstream stops backpressuring and there is a batched element available

backpressures batched elements reached the max weight limit of allowed batched elements & downstream backpressures

completes upstream completes and a “possibly pending” element was drained

Found an error in this documentation? The source code for this page can be found here. Please feel free to edit and contribute a pull request.