Likelihood to Recommend If you need a managed big data megastore, which has native integration with highly optimized
Apache Spark Engine and native integration with MLflow, go for Databricks Lakehouse Platform. The Databricks Lakehouse Platform is a breeze to use and analytics capabilities are supported out of the box. You will find it a bit difficult to manage code in notebooks but you will get used to it soon.
Read full review Ideal for daily standard ETL use cases whether the data is sourced from / transferred to the native connectors (like SQL Server) or FTP. Best if the company uses MS suite of tools. There are better options in the market for chaining tasks where you want a custom flow of executions depending on the outcome of each process or if you want advanced functionality like API connections, etc.
Read full review Pros Process raw data in One Lake (S3) env to relational tables and views Share notebooks with our business analysts so that they can use the queries and generate value out of the data Try out PySpark and Spark SQL queries on raw data before using them in our Spark jobs Modern day ETL operations made easy using Databricks. Provide access mechanism for different set of customers Read full review Ease of use - can be used with no prior experience in a relatively short amount of time. Flexibility - provides multiple means of accomplishing tasks to be able to support virtually any scenario. Performance - performs well with default configurations but allows the user to choose a multitude of options that can enhance performance. Resilient - supports the configuration of error handling to prevent and identify breakages. Complete suite of configurable tools. Read full review Cons Connect my local code in Visual code to my Databricks Lakehouse Platform cluster so I can run the code on the cluster. The old databricks-connect approach has many bugs and is hard to set up. The new Databricks Lakehouse Platform extension on Visual Code, doesn't allow the developers to debug their code line by line (only we can run the code). Maybe have a specific Databricks Lakehouse Platform IDE that can be used by Databricks Lakehouse Platform users to develop locally. Visualization in MLFLOW experiment can be enhanced Read full review SSIS has been a bit neglected by Microsoft and new features are slow in coming. When importing data from flat files and Excel workbooks, changes in the data structure will cause the extracts to fail. Workarounds do exist but are not easily implemented. If your source data structure does not change or rarely changes, this negative is relatively insignificant. While add-on third-party SSIS tools exist, there are only a small number of vendors actively supporting SSIS and license fees for production server use can be significant especially in highly-scaled environments. Read full review Likelihood to Renew Some features should be revised or improved, some tools (using it with Visual Studio) of the toolbox should be less schematic and somewhat more flexible. Using for example, the CSV data import is still very old-fashioned and if the data format changes it requires a bit of manual labor to accept the new data structure
Read full review Usability Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured. in terms of graph generation and interaction it could improve their UI and UX
Read full review SQL Server Integration Services is a relatively nice tool but is simply not the ETL for a global, large-scale organization. With developing requirements such as NoSQL data, cloud-based tools, and extraordinarily large databases, SSIS is no longer our tool of choice.
Read full review Performance Raw performance is great. At times, depending on the machine you are using for development, the IDE can have issues. Deploying projects is very easy and the tool set they give you to monitor jobs out of the box is decent. If you do very much with it you will have to write into your projects performance tracking though.
Read full review Support Rating One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.
Read full review The support, when necessary, is excellent. But beyond that, it is very rarely necessary because the user community is so large, vibrant and knowledgable, a simple Google query or forum question can answer almost everything you want to know. You can also get prewritten script tasks with a variety of functionality that saves a lot of time.
Read full review Implementation Rating The implementation may be different in each case, it is important to properly analyze all the existing infrastructure to understand the kind of work needed, the type of software used and the compatibility between these, the features that you want to exploit, to understand what is possible and which ones require integration with third-party tools
Read full review Alternatives Considered Compared to
Synapse &
Snowflake , Databricks provides a much better development experience, and deeper configuration capabilities. It works out-of-the-box but still allows you intricate customisation of the environment. I find Databricks very flexible and resilient at the same time while
Synapse and
Snowflake feel more limited in terms of configuration and connectivity to external tools.
Read full review I had nothing to do with the choice or install. I assume it was made because it's easy to integrate with our SQL Server environment and free. I'm not sure of any other enterprise level solution that would solve this problem, but I would likely have approached it with traditional scripting. Comparably free, but my own familiarity with trad scripts would be my final deciding factor. Perhaps with some further training on SSIS I would have a different answer.
Read full review Return on Investment The ability to spin up a BIG Data platform with little infrastructure overhead allows us to focus on business value not admin DB has the ability to terminate/time out instances which helps manage cost. The ability to quickly access typical hard to build data scenarios easily is a strength. Read full review Data integrity across various products allows unify certain processes inside the organization and save funds by reducing human labour factor. Automated data unification allows us plan our inputs better and reduce over-warehousing by overbuying The employee number, responsible for data management was reduced from 4 to 1 person Read full review ScreenShots