Does Storm guarantee data processing?
Last reviewed:
Yes. Storm tracks every tuple through the topology and replays it from the spout if it is not fully processed within a timeout. That gives at-least-once processing by default, and messages are only replayed when something fails. For exactly-once semantics, Storm's higher-level Trident API processes data in batches and stores state transactionally.
Storm guarantees every tuple will be fully processed. One of Storm's core mechanisms is the ability to track the lineage of a tuple as it makes its way through the topology in an extremely efficient way. Read more about how this works here.
Storm's basic abstractions provide an at-least-once processing guarantee, the same guarantee you get when using a queueing system. Messages are only replayed when there are failures.
Using Trident, a higher level abstraction over Storm's basic abstractions, you can achieve exactly-once processing semantics.