Producers and Consumers subscribe to a stream of records
Producer -> Streaming Platform -> Consumer
The producer and consumer here are independent of each other meaning the producer has no clue about which consumer is going to read the message.
Stores the stream of events so that it can be replayed if neccessary.
Traditional Messaging System
Transient Message Persistance: Once the messages are read by the consumers, then the messages will be removed from the message broker.
It is the brokers responsibility to keep track of consumed messages and remove them from the broker when the messages are read.
It can target a specific consumer to read the message from the broker.
Does not follow the principles of distributed systems.
Kafka Streaming Platform
Saves the messages in a file system where Kafka is installed.
Stores events based on a retention time. Events are immutable.
Consumers Responsibility to keep track of consumed messages.
Any consumer can access a message from the broker.
Kafka is built on top of the core principles of a distributed system.
Transportation
Retail
Banking