Apache Kafka crashes, failures, partition reassignment. Trasparency is a requirement for distributed systems

“…your partitions will be reassigned when your app crashes, so you won’t even be able to detect the failure later unless you carefully track it …”

This was about the Kafka simple consumer from here http://engineering.onlive.com/2013/12/12/didnt-use-kafka/

Also from official documentations: “… For manually assigning replicas to…”.

We look for comments and discussion about trasparency in Kafka.

It can be on these issues as well on other ones:

Performance transparency: Client programs should continue to perform satisfactorily while the load on the service varies within a specified range.
A major task of many middleware platforms is to make remote invocations look like local ones.
Is the programmer protected from the details of data representation and marshalling?
A common requirement when data are replicated is for replication transparency. That is, clients should not normally have to be aware that multiple physical copies of data exist. As far as clients are concerned, data are organized as individual logical objects and they identify only one item in each case when they request an operation to be performed. Furthermore, clients expect operations to return only one set of values.

Is this provided by Kafka?

Leave a Reply

Your email address will not be published. Required fields are marked *