Apache Flume is well suited when the use case is log data ingestion and aggregate only, for example for compliance of configuration management. It is not well suited where you need a general-purpose real-time data ingestion pipeline that can receive log data and other forms of data streams (eg IoT, messages).
Well suited for my big data related project or a static data set analysis especially for uploading huge dataset to the cluster.
But had some issues with connecting IoT real-time data and feeding to Power BI. It might be my understanding please take it as a mere comment rather than a suggestion.
Easier pricing and plug-and-play like you see with AWS and Azure, it would be nice from a budgeting and billing standpoint, as well as better support for the administration.
Bundling of the Cloud Object Storage should be included with the Analytics Engine.
The inability to add your own Hadoop stack components has made some transfers a little more complex.
Apache Flume is open-source so support is limited. Never the less, it has great documentation and best practices documents from their end-users so it is not hard to use, setup and configure.
Apache Flume is a very good solution when your project is not very complex at transformation and enrichment, and good if you have an external management suite like Cloudera, Hortonworks, etc. But it is not a real EAI or ETL like AB Initio or Attunity so you need to know exactly what you want. On the other hand being an opensource project give Apache a lot of room to personalize thanks to its plug-able architecture and has a very nice performance having a very low CPU and Memory footprint, a single server can do the job on many occasions, as opposed to the multi-server architecture of paid products.
We initially wanted to go with Google BigQuery, mainly for the name recognition. However, the pricing and support structure led us to seek alternatives, which pointed us to IBM. Apache Spark was also in the running, but here IBM's domination in the industry made the choice a no-brainer. As previously stated, the support received was not quite what we expected, but was adequate.
This product has allowed us to gather analytics data across multiple platforms so we can view and analyze the data from different workflows, all in one place.
IBM Analytics has allowed us to scale on demand which allows us to capture more and more data, thus increasing our ROI.
The convenience of the ability to access and administer the product via multiple interfaces has allowed our administrators to ensure that the application is making a positive ROI for our business users and partners.
