Azure Data Lake Storage vs. Google Cloud Dataflow vs. Nasuni

Azure Data Lake is an absolutely essential piece of a modern data and analytics platform. Over the past 2 years, our usage of Azure Data Lake as a reporting source has continued to grow and far exceeds more traditional sources like MS SQL, Oracle, etc.

Incentivized

It is best in cases where you have batch as well as streaming data. Also in some cases where you have batch data right now and in future you will get streaming data. In those cases Dataflow is very good. Also in cases where most of your infra is on GCP. It might not be good when you already are on AWS or Azure. And also you want in-depth control over security and management. Then you can directly use Apache beam over Dataflow.

Incentivized

Well suited if you have a lot of data that doesn't need to be stored and read right away. I think even if you don't have much data, you can still use it for it's intended purpose to great effect, but think of it as the more data you have, the even better it will work. I don't think it would be particularly useful if you already have a slick file restore system in place and you don't need to store your data elsewhere with redundancy.

Incentivized

• Setting up Azure Data Lake Storage account, container is quite easy
• Access from anywhere and easy maintenance
• Integration with Azure Data Factory service for end to end pipeline is pretty easy
• Can store Any form of data (Structured, Unstructured, Semi) in faster manner

Incentivized

• Streaming, Real time work load
• Batch processing
• Auto scaling
• flexible pricing

• The management console is extremely simple and easy to navigate, making common tasks easy to do.
• Our storage appliance is configured to snapshot data several times an hour, making the risk of data loss very low.
• Data restores are very intuitive, and take seconds to initiate regardless of whether it is one file or 300GB of data. We have successfully restored many Gigs of data in minutes.

Incentivized

• study for the certifications also to have them as a reference for work when you have any questions about applying a configuration to the equipment.
• The Internet interface is simple and easy to use. Capacity is good and it's good that HP continues to innovate with this technology

Incentivized

• More templates for Bigquery and App Engine. There is only limited options for templates so the things we use can limit.
• I would like native connectors for Excel (XLSX) to reduce the need for custom wrappers in financial pipelines.
• Debugging Google Cloud Dataflow using only logs in Cloud Logging can be overwhelming sometimes, and it’s not always obvious which specific element in the flow caused a failure. IT uses a lot of time.

Incentivized

• Better analytics when issues arise.
• Better onboarding documentation.
• Real-time monitoring.

Incentivized

It really saved a lot of time and it's flexibility really can give you infra which is future-proof for most of the use cases may it be streaming or batch data. And with this you can avoid use of resource-heavy big data offerings.

Incentivized

As I mentioned, the user interface is amazing and straight forward. It's very easy to learn how to configure and restore files. I would like a bit more reporting, especially in terms of live reporting and monitoring. The support is great when you have a question on how to do something, which helps with usability.

Incentivized

Again, it may have a little to do with the size and speed of your own environment, but we've been nothing but pleased with the speed of access of the files - even pulling old files from the cloud storage. Recovery of huge and many data files is a bit slow if you don't have the specs of the filer up to snuff.

Incentivized

Azure Data Lake Storage from a functionality perspective is a much easier solution to work with. It's implementation from Amazon EMR went smooth, and continued usage is definitely better. However, Amazon EMR was significantly cheaper overall between the high transaction fees and cost of storage due to growth. The two both have their advantages and disadvantages, but the functionality of Azure Data Lake Storage outweighed it's cost

Incentivized

Google Cloud Dataproc Cloud Datafusion

The technical support and escalation path for Nasuni is much more reliable and efficient. No getting transferred to various teams. Often times, the person who answers your call is able to resolve your issue. If they cannot, they get the case assigned to the appropriate engineer right away. Time to close has always been very good.

Incentivized

• Instead of having separate pools of storage for data we are now operating on a single layer platform which has cut down on time spent on maintaining those separate pools.
• We have had more of an ROI with the scalability as we are able to control costs of storage when need be.
• We are able to operate in a more streamlined approach as we are able to stay within the Azure suite of products and integrate seamlessly with the rest of the applications in our cloud-based infrastructure

Incentivized

• cost saving from managing our own data center for ETL servers
• consumption based pricing
• with auto scaling feature, we were able to expand components to support work load

• Dramatically reduced time spent managing our storage platform. Quotas and reporting tools take all the guesswork out of data growth. Updates are easy to deploy. Time freed up can be used for more user-facing activities that we consider more valuable to the organization.
• The overall stability of the platform has been very good. We have been running on the same hardware for the past four years without any performance issues.

Incentivized