Amazon Kinesis vs. IBM Streams (discontinued)

Amazon Kinesis is a great replacement for Kafka and it works better whenever the components of the solution are AWS based. Best if extended fan-out is not required, but still price-performance ratio is very good for simplifying maintenance.

I would go with a different option if the systems to be connected are legacy, for instance in the case of traditional messaging clients.

Incentivized

Like the name says, it is good for streaming data and analyzing. It is great to look at tuples at a fast rate, filtering, calling other sources to enrich data, can call APIs, etc. Could do better for ingest use cases, can do better with guaranteed delivery, etc.

Incentivized

• Processing huge loads of data
• Integrating well with IoT Platform on Amazon
• Integration with overall AWS Ecosystem
• Scalability

Incentivized

• IBM Streams is well suited for providing wire-speed real-time end-to-end processing with sub-millisecond latency.
• Streams is amazingly computationally efficient. In other words, you can typically do much more processing with a given amount of hardware than other technologies. In a recent linear-road benchmark Streams based application was able to provide greater capability than the Hadoop-based implementation using 10x less hardware. So even when latency isn't critical, using Streams might still make sense for reducing operational cost.
• Streams comes out of the box with a large and comprehensive set of tested and optimized toolkits. Leveraging these toolkits not only reduces the development time and cost but also helps reduce project risk by eliminating the need for custom code which likely has not seen as much time in test or production.
• In addition to the out of the box toolkits, there is an active developer community contributing additional specialized packages.

Incentivized

• Not a queue system, so little visibility into "backlog" if there is any
• Confusing terminology to make sure events aren't missed
• Sometimes didn't seem to trigger Lambda functions, or dropped events when a lot came in

Incentivized

• Documentation could be more extensive, with more examples, although overall this is not too bad compared to some of the alternative solutions.
• Seems expensive to use in production.

Incentivized

The documentation was confusing and lacked examples. The streams suddenly stopped working with no explanation and there was no information in the logs. All these were more difficult when dealing with enhanced fan-out. In fact, we were about to abort the usage of Kinesis due to a misunderstanding with enhanced fan-out.

Incentivized

The main benefit was around set up - incredibly easy to just start using Kinesis. Kinesis is a real-time data processing platform, while Kafka is more of a message queue system. If you only need a message queue from a limited source, Kafka may do the job. More complex use cases, with low latency, higher volume of data, real time decisions and integration with multiple sources and destination at a decent price, Kinesis is better.

Incentivized

There are well explained tutorials to get the user started. If you are looking for business application ideas, the user community offers a diversity of applications. It is very easy to launch applications on the cloud and can integrate with other analytic tools available on Watson Studio. It takes away the burden of the technology so that users can focus on business innovations.

Incentivized

• Caused us to need to re-engineer some basic re-try logic
• Caused us to drop some content without knowing it
• Made monitoring much more difficult
• We eventually switched back to SQS because Kinesis is not the same as a Queue system

Incentivized

• Ability to do more with less
• Admins and data analyst can now focus on more thinking tasks
• No negative impacts yet

Incentivized