Apache Flink vs. Cloudera Data Platform

In well-suited scenarios, I would recommend using Apache Flink when you need to perform real-time analytics on streaming data, such as monitoring user activities, analyzing IoT device data, or processing financial transactions in real-time. It is also a good choice in scenarios where fault tolerance and consistency are crucial. I would not recommend it for simple batch processing pipelines or for teams that aren't experienced, as it might be overkill, and the steep learning curve may not justify the investment.

Incentivized

I have seen that Cloudera Data Platform is well suited for large batch processes. It works really well for our indication analyses that are performed by the actuaries. I feel that rapid streaming operations may be a situation where additional technology would be needed to provide for a robust solution.

Incentivized

• Low latency Stream Processing, enabling real-time analytics
• Scalability, due its great parallel capabilities
• Stateful Processing, providing several built-in fault tolerance systems
• Flexibility, supporting both batch and stream processing

Incentivized

• Scales
• Highly available

Incentivized

• Python/SQL API, since both are relatively new, still misses a few features in comparison with the Java/Scala option
• Steep Learning Curve, it's documentation could be improved to something more user-friendly, and it could also discuss more theoretical concepts than just coding
• Community smaller than other frameworks

Incentivized

• Constantly changing costs
• Log visibility

Incentivized

We have utilized Cloudera support quite frequently and are very satisfied with the capability and responsiveness of that team. Often, the new features delivered with the platform give us an opportunity to mature the way we're doing things, and the support team have been valuable in developing those new patterns.

Incentivized

Apache Spark is more user-friendly and features higher-level APIs. However, it was initially built for batch processing and only more recently gained streaming capabilities. In contrast, Apache Flink processes streaming data natively. Therefore, in terms of low latency and fault tolerance, Apache Flink takes the lead. However, Spark has a larger community and a decidedly lower learning curve.

Incentivized

IBM's offering of the Cloud Pak for Data has been a moving target and difficult to compare to Cloudera Data Platform. We have implemented our solution on Amazon Web Services, which appears to be supported by IBM at this point, but the migration would be very expensive for us to endeavor.

Incentivized

• Allowed for real-time data recovery, adding significant value to the busines
• Enabled us to create new internal tools that we couldn't find in the market, becoming a strategic asset for the business
• Enhanced the overall technical capability of the team

Incentivized

• Reduced operational costs
• Speed to market

Incentivized