Apache Airflow vs. OpenText Vertica vs. Snowflake

Airflow is well-suited for data engineering pipelines, creating scheduled workflows, and working with various data sources. You can implement almost any kind of DAG for any use case using the different operators or enforce your operator using the Python operator with ease. The MLOps feature of Airflow can be enhanced to match MLFlow-like features, making Airflow the go-to solution for all workloads, from data science to data engineering.

Incentivized

Vertica as a data warehouse to deliver analytics in-house and even to your client base on scale is not rivaled anywhere in the market. Frankly, in my experience it is not even close to equaled. Because it is such a powerful data warehouse, some people attempt to use it as a transactional database. It certainly is not one of those. Individual row inserts are slow and do not perform well. Deletes are a whole other story. RDBMS it is definitely not. OLAP it rocks.

Incentivized

Snowflake is well suited when you have to store your data and you want easy scalability and increase or decrease the storage per your requirement. You can also control the computing cost, and if your computing cost is less than or equal to 10% of your storage cost, then you don't have to pay for computing, which makes it cost-effective as well.

Incentivized

• Apache Airflow is one of the best Orchestration platforms and a go-to scheduler for teams building a data platform or pipelines.
• Apache Airflow supports multiple operators, such as the Databricks, Spark, and Python operators. All of these provide us with functionality to implement any business logic.
• Apache Airflow is highly scalable, and we can run a large number of DAGs with ease. It provided HA and replication for workers. Maintaining airflow deployments is very easy, even for smaller teams, and we also get lots of metrics for observability.

Incentivized

• Extremely fast query performance - Vertica is one of the fastest query engines out there.
• Scales to TBs - Scales reasonably well up to 10-20 nodes and 10 - 100s of TB of data.
• Easy to Use - Fairly easy to user, we made quite some headway with just 1 person running it for a while.

Incentivized

• Snowflake scales appropriately allowing you to manage expense for peak and off peak times for pulling and data retrieval and data centric processing jobs
• Snowflake offers a marketplace solution that allows you to sell and subscribe to different data sources
• Snowflake manages concurrency better in our trials than other premium competitors
• Snowflake has little to no setup and ramp up time
• Snowflake offers online training for various employee types

Incentivized

• UI/Dashboard can be updated to be customisable, and jobs summary in groups of errors/failures/success, instead of each job, so that a summary of errors can be used as a starting point for reviewing them.
• Navigation - It's a bit dated. Could do with more modern web navigation UX. i.e. sidebars navigation instead of browser back/forward.
• Again core functional reorg in terms of UX. Navigation can be improved for core functions as well, instead of discovery.

Incentivized

• Could use some work on better integrating with cloud providers and open source technologies. For AWS you will find an AMI in the marketplace and recently a connector for loading data from S3 directly was created. With last release, integration with Kafka was added that can help.
• Managing large workloads (concurrent queries) is a bit challenging.
• Having a way to provide an estimate on the duration for currently executing queries / etc. can be helpful. Vertica provides some counters for the query execution engine that are helpful but some may find confusing.
• Unloading data over JDBC is very slow. We've had to come up with alternatives based on vsql, etc. Not a very clean, official on how to unload data.

Incentivized

• Add constraints for views and not just for tables
• Do not force customers to renew for same or higher amount to avoid loosing unused credits. Already paid credits should not expire (at least within a reasonable time frame), independent of renewal deal size.

Incentivized

SnowFlake is very cost effective and we also like the fact we can stop, start and spin up additional processing engines as we need to. We also like the fact that it's easy to connect our SQL IDEs to Snowflake and write our queries in the environment that we are used to

Incentivized

For its capability to connect with multicloud environments. Access Control management is something that we don't get in all the schedulers and orchestrators. But although it provides so many flexibility and options to due to python , some level of knowledge of python is needed to be able to build workflows.

Incentivized

Because the fact that you can query tons of data in a few seconds is incredible, it also gives you a lot of functions to format and transform data right in your query, which is ideal when building data models in BI tools like Power BI, it is available as a connector in the most used BI tools worldwide.

Incentivized

I haven't had any recent opportunity to reach out to Vertica support. From what I remember, I believe whenever I reached out to them the experience was smooth.

Incentivized

We have had terrific experiences with Snowflake support. They have drilled into queries and given us tremendous detail and helpful answers. In one case they even figured out how a particular product was interacting with Snowflake, via its queries, and gave us detail to go back to that product's vendor because the Snowflake support team identified a fault in its operation. We got it solved without lots of back-and-forth or finger-pointing because the Snowflake team gave such detailed information.

Incentivized

Multiple DAGs can be orchestrated simultaneously at varying times, and runs can be reproduced or replicated with relative ease. Overall, utilizing Apache Airflow is easier to use than other solutions now on the market. It is simple to integrate in Apache Airflow, and the workflow can be monitored and scheduling can be done quickly using Apache Airflow. We advocate using this tool for automating the data pipeline or process.

Incentivized

Vertica performs well when the query has good stats and is tuned well. Options for GUI clients are ugly and outdated. IO optimized: it's a columnar store with no indexing structures to maintain like traditional databases. The indexing is achieved by storing the data sorted on disk, which itself is run transparently as a background process.

Incentivized

I have had the experience of using one more database management system at my previous workplace. What Snowflake provides is better user-friendly consoles, suggestions while writing a query, ease of access to connect to various BI platforms to analyze, [and a] more robust system to store a large amount of data. All these functionalities give the better edge to Snowflake.

Incentivized

• Impact Depends on number of workflows. If there are lot of workflows then it has a better usecase as the implementation is justified as it needs resources , dedicated VMs, Database that has a cost
• Donot use it if you have very less usecases

Incentivized

• Positive impact on ROI by being able to get customer insights in real-time.
• Positive ROI through reduced time to set-up and maintain Vertica instances.

Incentivized

• With separate compute and storage feature, the queries get executed quickly and it improves our overall productivity.
• Earlier we were using a different product for analytical purposes, but with Snowflake's in-built analytical feature we are now able to save money.
• Snowflake is cost efficient, features like auto suspend for compute resources helped to control the costs.

Incentivized