Alpakka Kafka 1.1.x

Note

In case you are browsing a specific version’s documentation: check out the latest release notes

Alpakka Kafka 1.1 changes the internals of how offset commits are sent to the Kafka broker. The new implementation is very beneficial for high-throughput consumers as committing is now connected to the regular polling of the Kafka consumer, which reduces pressure on Kafka.

It adds a new source for advanced usage: committablePartitionedManualOffsetSource which may be used when offsets are stored external to Kafka, but tools for consumer lag which rely on offsets being committed to Kafka are in use.

As a new experimental feature, offset committing is now possible without applying backpressure to the stream when commits are not processed at the same speed. This can be controlled via the delivery committer setting.

Exceptions from the Kafka consumer API are now properly passed back and fail the stream.

This goes along with a few improvements to the documentation.

1.1.0

Released: 2019-10-10

No changes affecting the functionality since 1.1.0-RC2.

The detailed list of changes is found in the 1.1.0 Github release listing.

1.1.0-RC2

Released: 2019-10-02

  • Committable partitioned source with manual offset seek support #908 by @seglo

The detailed list of changes is found in the 1.1.0-RC2 Github release listing.

This release is compiled and tested against Akka 2.5 for Scala 2.11, 2.12 and 2.13, Akka 2.6-M8 for Scala 2.12, and 2.13 and both Akka versions on Adopt OpenJDK 1.8 and 11.

If we do not get any reports that speak against it, we’ll release Alpakka Kafka 1.1.0 in a few weeks.

This release contains contributions by:

Author Commits Lines added Lines removed
ennru ennru 10 303 207
seglo seglo 1 249 25
raboof raboof 1 1 1

1.1.0-RC1

Released: 2019-09-05

Alpakka Kafka 1.1 changes the internals of how offset commits are sent to the Kafka broker. The new implementation is very beneficial for high-throughput consumers as committing is now connected to the regular polling of the Kafka consumer, which reduces pressure on Kafka.

As a new experimental feature, offset committing is now possible without applying backpressure to the stream when commits are not processed at the same speed. This can be controlled via the delivery committer setting.

Exceptions from the Kafka consumer API are now properly passed back and fail the stream.

This goes along with a few improvements to the documentation.

Notable changes since 1.0.5

  • Aggregate offsets and commit before poll #862
  • Special-casing single offset committing #868
  • Introduce setting for committing without backpressure #883
  • Capture exceptions from Kafka consumer and pass to involved stages #887

The detailed list of changes is found in the 1.1.0-RC1 Github release listing.

This release is compiled and tested against Akka 2.5 for Scala 2.11, 2.12 and 2.13, Akka 2.6-M6 for Scala 2.12, and 2.13 and both Akka versions on Adopt OpenJDK 1.8 and 11.

This release contains contributions by:

Author Commits Lines added Lines removed
ennru ennru 25 947 369
2m 2m 9 93 66
TimMoore TimMoore 2 10 33
bwmcadams bwmcadams 1 1 1
Found an error in this documentation? The source code for this page can be found here. Please feel free to edit and contribute a pull request.