Apache Flume vs. Databricks Data Intelligence Platform vs. IBM Cognos Analytics

Apache Flume is well suited when the use case is log data ingestion and aggregate only, for example for compliance of configuration management. It is not well suited where you need a general-purpose real-time data ingestion pipeline that can receive log data and other forms of data streams (eg IoT, messages).

Incentivized

Medium to Large data throughput shops will benefit the most from Databricks Spark processing. Smaller use cases may find the barrier to entry a bit too high for casual use cases. Some of the overhead to kicking off a Spark compute job can actually lead to your workloads taking longer, but past a certain point the performance returns cannot be beat.

Incentivized

Well suited: Financial reporting - It can handle complex, pixel perfect, muti-page reports with scheduled delivery to stakeholders (like sales report by region on quarterly periodicity) Operational dashboard across departments - It can combine multiple data sources (ERP, CRM, excels etc) with filters, and embedded AI insights Less appropriate: Live dashboards - As stated earlier as well, IBM Cognos Analytics doesn't suit well for live dashboards or event driven data. For ex: live web traffic data or IOT device data, etc Data science - Although IBM Cognos Analytics is great tool for data exploration but it should not be used as a substitute for Python or R, which has edge over advanced modelling and stats based workflows like predictive modelling or clustering

Incentivized

• Multiple sources of data (sources) and destinations (sinks) that allows you to move data form and to any relevant data storage
• It is very easy to setup and run
• Very open to personalization, you can create filters, enrichment, new sources and destinations

Incentivized

• Process raw data in One Lake (S3) env to relational tables and views
• Share notebooks with our business analysts so that they can use the queries and generate value out of the data
• Try out PySpark and Spark SQL queries on raw data before using them in our Spark jobs
• Modern day ETL operations made easy using Databricks. Provide access mechanism for different set of customers

Incentivized

• We can make dozens of dispatchers all focusing on different types of workloads.
• Friendly user interface, without the need for coding or complicated editing.
• Highly functionality reporting tools.
• We can easily create trigger when a certain threshold are met sending reports or alerts to needed parties.

Incentivized

• It is very specific for log data ingestion so it is pretty hard to use for anything else besides log data
• Data replication is not built in and needs to be added on top of Apache Flume (not a hard job to do though)

Incentivized

• Sometimes, when multiple jobs depend on each other in different environments, it is not always easy to see the full workflow in one place.
• It is sometimes difficult to determine which job or cluster contributes more to the overall cost.
• For beginners, cluster configuration may be a little difficult. So more recommendation in the platform can help.

Incentivized

• IBM Cognos Analytics enables customer data segmentation, which is essential for marketing, improving and streamlining purchasing behavior and preferences. This helps companies create more targeted and effective marketing campaigns.
• Our clients Through data analysis, we can identify and observe trends in the behavior of other clients, allowing us to anticipate needs and adjust strategies to avoid consequences.

Incentivized

For an existing solution, renewing licenses does provide a good return on investment. Additionally, while rolling out scorecards and dashboards with little adhoc capabilities, to end users, cognos is very easily scalable. It also allows to create a solution that has a mix of OLAP and relational data-sources, which is a limitation with other tools. Synchronizing with existing security setup is easy too.

Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured.

in terms of graph generation and interaction it could improve their UI and UX

Incentivized

We have a strong user base (3500 users) that are highly utilizing this tool. Basic users are able to consume content within the applied security model. We have a set of advanced users that really push the limits of Cognos with Report and Query Studio. These users have created a lot of personal content and stored it in 'My Reports'. Users enjoy this flexibility.

Reports can typically be viewed through any browser that can access the server, so the availability is ultimately up to what the company utilizing it is comfortable with allowing, though report development tends to be more picky about browsers and settings as mentioned above. It also has an optional iPad app and general mobile browsing support, but dashboards lack the mobile compatibility. What keeps it from getting a higher score is the desktop tools that are vital to the development process. The compatibility with only Windows when the server has a wide range of compatibility can be a real sore point for a company that outfits its employees exclusively with Mac or Linux machines. Of course, if they are planning on outsourcing the development anyways, it's a rather moot point

Overall no major complaints but it doesn't handle DMR (Dimensionally Modeled for Relational) very well. DMR modelling is a capability that IBM Cognos Framework Manager provides allowing you to specify dimensional information for relational metadata and allows for OLAP-style queries. However, the capability is not very efficient and, for example, if I'm using only 2 columns on a 20-column model, the software is not smart enough to exclude 18 columns and the query side gets progressively larger and larger until it's effectively unusable.

Apache Flume is open-source so support is limited. Never the less, it has great documentation and best practices documents from their end-users so it is not hard to use, setup and configure.

Incentivized

One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.

Why is their web application not working as fast as you think it should? They never know, and it is always a a bunch of shots in the dark to find out. Trying to download software from them is like trying to find a book at the library before computers were invented.

Incentivized

Onsite training provided by IBM Cognos was effective and as expected. They did not perform training with our data which was a bit difficult for our end-users.

The online courses they offer are thorough and presented in such a way that someone who isn't already familiar with the general design methodologies used in this field will be capable of making a good design. The training environments are provided as a fully self contained virtual machine with everything needed already to create the environments. We've had some persisting issues with the environments becoming unavailable, but support has been responsive when these issues arise and straightening them out for us

Make sure that any custom tables that you have, are built into your metadata packages. You can still access them via SQL queries in Cognos, but it is much easier to have them as a part of the available metadata packages.

Apache Flume is a very good solution when your project is not very complex at transformation and enrichment, and good if you have an external management suite like Cloudera, Hortonworks, etc. But it is not a real EAI or ETL like AB Initio or Attunity so
you need to know exactly what you want. On the other hand being an opensource project give Apache a lot of room to personalize thanks to its plug-able architecture and has a very nice performance having a very low CPU and Memory footprint, a single server can do the job on many occasions, as opposed to the multi-server architecture of paid products.

Incentivized

The most important differentiating factor for Databricks Lakehouse Platform from these other platforms is support for ACID transactions and the time travel feature. Also, native integration with managed MLflow is a plus. EMR, Cloudera, and Hortonworks are not as optimized when it comes to Spark Job Execution. Other platforms need to be self-managed, which is another huge hassle.

Incentivized

Power BI is stronger for quick ad-hoc analysis and dashboards, but IBM Cognos Analytics is better when consistency, precision, and mass distribution matter. Tableau is best for interactive analysis, while IBM Cognos Analytics is better for standardized, repeatable enterprise reporting. Sigma shines for customizable dashboards and drill-down analysis while IBM Cognos Analytics holds an edge in data discovery and visualization.

Incentivized

The Cognos architecture is well suited for scalability. However, the architecture must be designed with scalability in mind from day one of the implementation. We recently upgraded from 10.1 to 10.2.1 and took the opportunity to revamp our architecture. It is now poised for future growth and scalability.

• Flume has simplified a lot many of our ingest procedures, easier to deploy and integrate than a classical EAI, reducing the time to market
• But opposed to EAIs if the project starts to grow in complexity Apache Flume project may not be as suitable

Incentivized

• The ability to spin up a BIG Data platform with little infrastructure overhead allows us to focus on business value not admin
• DB has the ability to terminate/time out instances which helps manage cost.
• The ability to quickly access typical hard to build data scenarios easily is a strength.

Incentivized

• We use the tool for data modeling as it helps in predictive data analysis for complex data, which is very similar to real-life scenarios.
• Options of customizing & scheduling reports as per our requirements basis.
• Has mobile application which works seamless.
• API integration is not upto the mark with very limited options.
• Licensing & Maintenance can go from cheap to expensive depending on the scope.

Incentivized