Databricks offers the Databricks Lakehouse Platform (formerly the Unified Analytics Platform), a data science platform and Apache Spark cluster manager. The Databricks Unified Data Service provides a platform for data pipelines, data lakes, and data platforms.
$0.07
Per DBU
GoCD
Score8 out of 10
N/A
GoCD, from ThoughtWorks in Chicago, is an application lifecycle management and development tool.
Medium to Large data throughput shops will benefit the most from Databricks Spark processing. Smaller use cases may find the barrier to entry a bit too high for casual use cases. Some of the overhead to kicking off a Spark compute job can actually lead to your workloads taking longer, but past a certain point the performance returns cannot be beat.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Previously, our team used Jenkins. However, since it's a shared deployment resource we don't have admin access. We tried GoCD as it's open source and we really like. We set up our deployment pipeline to run whenever codes are merged to master, run the unit test and revert back if it doesn't pass. Once it's deployed to the staging environment, we can simply do 1-click to deploy the appropriate version to production. We use this to deploy to an on-prem server and also AWS. Some deployment pipelines use custom Powershell script for.Net application, some others use Bash script to execute the docker push and cloud formation template to build elastic beanstalk.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Pipeline-as-Code works really well. All our pipelines are defined in yml files, which are checked into SCM.
The ability to link multiple pipelines together is really cool. Later pipelines can declare a dependency to pick up the build artifacts of earlier ones.
Agents definition is really great. We can define multiple different kinds of environments to best suit our diverse build systems.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured.
in terms of graph generation and interaction it could improve their UI and UX
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.
The most important differentiating factor for Databricks Lakehouse Platform from these other platforms is support for ACID transactions and the time travel feature. Also, native integration with managed MLflow is a plus. EMR, Cloudera, and Hortonworks are not as optimized when it comes to Spark Job Execution. Other platforms need to be self-managed, which is another huge hassle.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
GoCD is easier to setup, but harder to customize at runtime. There's no way to trigger a pipeline with custom parameters.
Jenkins is more flexible at runtime. You can define multiple user-provided parameters so when user needs to trigger a build, there's a form for him/her to input the parameters.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Settings.xml need to be backed up periodically. It contains all the settings for your pipelines! We accidentally deleted before and we have to restore and re-create several missing pipelines
More straight forward use of API and allows filtering e.g., pull all pipelines triggered after this date
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
