Overall Satisfaction with Azure Databricks
We are leveraging Databricks capabilities in various use cases. For instance, to design a tailor made change data capture that keep track of users account details and keep it updated in delta lake. We have also designed numerous ETL processes which is scheduled to provide data to data analytics on strict delivery timelines. Moreover, the workspaces is integrated with other Azure services such as Azure Synapse Analytics, Azure data lake, Azure Data Factory. Some of our Databricks are triggered by Azure Data Factory.
- Consistently great performance when dealing with huge scale data with the help of spark architecture
- Magic commands such as spark sql, pyspark, scala . This comes really handy in day to day work
- Integration with other Azure services is super smooth and robust
- Their pipeline workflow orchestration is pretty primitive. Lacks some common features
- Workspace UI and navigation requires steep learning curve
- Personally, I am not fond of their autosave feature. Its dangerous for production level notebooks scripts