Azure Databricks vs. Vertex AI

Centralised notebooks are out directly into production. This can lead to poorly engineered code. It is very good for fast queries and our data team are always able to provide what we ask for. It is a big cost to our business so it is important it runs efficiently and returns on our investment.

Incentivized

In my regular activity, Vertex AI is missing some of the True Positive Alerts due to the ML training and needs to train more data sets, after it has reduced the false positives. To find the Zero day Vulnerability it has low accuracy and sometimes it misses the true positives. Once we have trained with the large data set, it came up with good results.

Incentivized

• Data Processing and Transformations based on Spark
• Delta Lakehouse when clubbed with an external cloud storage
• Governance using Unity Catalog to unify IAM
• Delta Live Tables is a product, which although relatively newer, has a great potential with the visuals of a pipeline.

Incentivized

• Vertex AI comes with support for LOTs of LLMs out of the box
• MLOps tools are available that help to standardize operational aspects
• Document AI is an out of the box feature that works just perfectly for our use cases of automating lots to tedious data extraction tasks from images as well as papers

Incentivized

• Intuitive interface
• Ease of use
• Providing FAQ or QRGs

Incentivized

• Customization of AutoML models - A must needed capability to be able to tweak hyperparameters and also working with different models
• Model Explainability -Providing more comprehensive explanations about how models are utilizing features could be very beneficial
• Model versioning and experiments tracking - Enhancing the versioning capability could be good for end users

Incentivized

The developers are able to switch between Python and SQL in the Notebook which allows the collaboration of SQL analyst and Data scientist. The integration of Mosaic AI allows users to write complex codes in natural languages. Unity catalog has centralized the security and governance features and simplified the process of maintaining it

Incentivized

Google is always top notch with their security and user interface performance. We use Google's entire suite in our business anyways, so using Vertex became second nature very quickly. I will say, though, that Google does need to come down on the price somewhat with their token allocation. Also, their UI is very robust, so it does require some time for training to really master it.

Incentivized

I have found Azure Databricks to be much better than Snowflake for handling bigger, diverse data types. Snowflake is much simpler and better for smaller warehousing. The real time processing is much better in Azure Databricks and we have much more language options. Snowflake is more expensive but simpler to use. Both are great for different needs.

Incentivized

We tend to adapt and use the platform that suits the customers needs the best. We return to Vertex AI because it is the most in-depth option out there so we can configure it any which way they want. However, it is not quick to market and constantly changing or updating it's feature-set. This makes it suitable for bigger customers that have the capital and time to spend on a bigger project that is well researched and not quick to market like some of the other options that feel like a light-version of this.

Incentivized

• The support team is amazing, they help you at every stage of the projects, from sales to delivery.
• On a framework level, it has had an amazing impact and has reduced the clients overall data platform costs by a staggering 65%
• There has been a 40% Manual work requirement on average for the clients when they move to Azure Databricks Data Platform

Incentivized

• It is pay as you go model so it'll save more cost of your org. In our case previously we used to incurred 1-2L/Month now we are reduced it to 80k-1L.
• It'll help you save your model training & model selection time as it provides pre-trained models in autoML.
• It'll help you in terms of Security wherein we can use row level security access to authorized persons.

Incentivized