Alpakka Kafka 1.0.x

1.0.5

Released: 2019-07-25

Notable changes since 1.0.4

• FlowWithContext support #780
• Added connection to broker status checker #674

This release is compiled and tested against Akka 2.5, 2.6 and Scala 2.11, 2.12, 2.13.0 on Adopt OpenJDK 8 and 11.

This release was made possible by:

Author	Commits	Lines added	Lines removed
2m	13	629	213
ennru	7	1327	203
tayvs	1	397	9
lomigmegard	1	1	1

1.0.4

Released: 2019-06-11

Alpakka Kafka 1.0.4 is released for Scala 2.13, 2.12 and 2.11.

Notable changes since 1.0.3

• Compile with Scala 2.13 #817
• Add Committer.batchFlow that emits CommittableOffsetBatch for every committed batch. #799

The detailed list of changes is found in the milestone.

This release is compiled and tested against Akka 2.5 and Scala 2.11, 2.12, 2.13.0 on Adopt OpenJDK 8 and 11.

This release was made possible by:

Author	Commits	Lines added	Lines removed
2m	30	2947	2735
ennru	3	966	301
mowczare	2	40	32
Jimmycheong	1	3	3

1.0.3

Released: 2019-05-09

Alpakka Kafka 1.0.3 fixes an important bug in the transactional support and improves the testkit for Java users.

Notable changes since 1.0.2

• Transactions: Avoid a timeout during draining #787 by Szymon Matejczyk, @szymonm
• Error handling: Signal exceptions from subscribe to stream, part of #772
• Testkit: Pull Embedded Kafka from new organisation id, part of #772
• Testkit: Improve Java API and make timeouts configurable via config #786
• Testkit: Specify the Docker image version for Testcontainers Kafka #783 as suggested by @miguelpuyol

The detailed list of changes is found in the milestone.

This release is compiled and tested against Akka 2.5 and Scala 2.11 and 2.12 on Adopt OpenJDK 1.8.

This release was made possible by:

Author	Commits	Lines added	Lines removed
2m	17	957	517
szymonm	2	29	17
ennru	2	15	6

1.0.2

Released: 2019-05-02

Alpakka Kafka 1.0.2 makes the transactional support more reliable.

Notable changes since 1.0.1

• Improvements for transactional support
  • Transactions: Add offsets to be committed directly after producer.send #752
  • Fix transaction offsets for transactional producer #742 by Szymon Matejczyk, @szymonm
• Do not fail commits during a rebalance (avoiding CommitFailedException) #755
• Testcontainers support in the Alpakka Kafka testkit #775

The detailed list of changes is found in the milestone.

This release is compiled and tested against Akka 2.5 and Scala 2.11 and 2.12 on Adopt OpenJDK 1.8.

This release was made possible by:

Author	Commits	Lines added	Lines removed
ennru	23	2165	1453
2m	4	147	59
szymonm	3	669	192
fancywriter	1	5	3

1.0.1

Released: 2019-03-07

Alpakka Kafka 1.0.1 upgrades the Apache Kafka client to 2.1.1, as version 2.1.0 contained a few annoying bugs which disturb proper operations.

Most notably: Kubernetes - Kafka clients are resolving DNS entries only one time (KAFKA-7755)

Full Apache Kafka 2.1.1 release notes.

The upgrade to the Apache Kafka client 2.1.1 is the only change compared to release 1.0.

This release is compiled and tested against Akka 2.5 and Scala 2.11 and 2.12.

This release was made possible by:

Author	Commits	Lines added	Lines removed
Philippus	1	1	1
raboof	1	1	1

1.0

Released: 2019-02-28

Final release of Alpakka Kafka 1.0! Theses release notes contain the need-to-know features and changes for Alpakka Kafka since release 0.22.

A bit of “history”

Alpakka Kafka has made a long journey. It started off as reactive kafka with a first release in January 2015 when Akka Streams were still experimental and created by Krzysiek Ciesielski, @kciesielski. The team at SoftwareMill were the first stewards of the project.

In 2016 the Akka team at Lightbend (at that time still called Typesafe) took over the responsibility to take care of it and started offering commercial support from version 0.16 (released May 2017). They were assisted by Alexey Romanchuk, @13h3r during that time. The Akka team has participated significantly in the creation of the Reactive Streams industry standard and Akka Streams has been one of its first implementations.

In May 2018 Lightbend started the Alpakka team to steward the Alpakka project and Alpakka Kafka. When the work of improving the tests and the code for Alpakka Kafka kicked off, we dropped the name it had for a while Akka Streams Kafka.

Apache Kafka has made an incredible journey during this period. From in-house project at LinkedIn via being open-sourced to becoming the integration backbone of many micro-service architectures.

The need for streaming data has been the key driver for all these endeavours.

Features

Alpakka Kafka provides Apache Kafka connectivity for Akka Streams. It supports consuming messages from Kafka into Akka Streams with at-most-once, at-least-once and transactional semantics, and supports producing messages to Kafka.

Once consumed messages are in the Akka Stream, the whole flexibility of all Akka Stream operators becomes available.

Alpakka Kafka achieves back-pressure for consuming by automatically pausing and resuming its Kafka subscriptions. When there is no downstream demand for more data, the consumer will not read any data. Any other communication with the Kafka broker (heartbeats, committing, rebalancing) will still continue.

Alpakka Kafka 1.0 uses the Apache Kafka Java client 2.1.0 internally.

Most important changes since 0.22

• Upgrade to Kafka client 2.1.0 #660. This upgrade makes it possible to use of the zstandard compression (with Kafka 2.1 brokers). Use Kafka client 2.x poll API #614.

• No more WakeupException! The Kafka client API 2.x allows for specifying a timeout when polling the Kafka broker, thus we do not need to use the cranky tool of Kafka’s WakeupExceptions to be sure not to block a precious thread. The settings to configure wake-ups are not used anymore.

• Alpakka Kafka consumers don’t fail for non-responding Kafka brokers anymore (as they used to to after a number of WakeupExceptions).

• New Committer.sink and Committer.flow for standardised committing #622 and #644

• Commit with metadata #563 and #579

• Java APIs for all settings classes #616

• Alpakka Kafka testkit