Databricks Data Intelligence Platform vs. Twilio Segment

Medium to Large data throughput shops will benefit the most from Databricks Spark processing. Smaller use cases may find the barrier to entry a bit too high for casual use cases. Some of the overhead to kicking off a Spark compute job can actually lead to your workloads taking longer, but past a certain point the performance returns cannot be beat.

Incentivized

Best suited: - Merging emails coming from: Facebook leads forms, Unbounce or landing pages forms, Google forms, any other kind of lead generation tool and bundling all that information together for a single user "profile". - Passing events generated in multiple applications by the same user (product selected in web, product discarded in cart, etc) and delivering those events into other applications (like a CRM) Less appropriate: - Reading/updating data directly from segment from a frontend application

Incentivized

• Process raw data in One Lake (S3) env to relational tables and views
• Share notebooks with our business analysts so that they can use the queries and generate value out of the data
• Try out PySpark and Spark SQL queries on raw data before using them in our Spark jobs
• Modern day ETL operations made easy using Databricks. Provide access mechanism for different set of customers

Incentivized

• Multi-platform. Segment has easy integrations in many different web, backend, and app platforms/frameworks. We use the Segment SDK in Android and iOS as well as our node.js backend.
• Segment is fairly affordable for early-stage companies that are trying out different analytics software. The "developer" plan is free and is suitable for most companies with products that have a small user base.
• The UI is great! It is extremely intuitive and easy-to-learn, and this made it take very little time to integrate this software into our analytics and marketing workflows.

Incentivized

• Connect my local code in Visual code to my Databricks Lakehouse Platform cluster so I can run the code on the cluster. The old databricks-connect approach has many bugs and is hard to set up. The new Databricks Lakehouse Platform extension on Visual Code, doesn't allow the developers to debug their code line by line (only we can run the code).
• Maybe have a specific Databricks Lakehouse Platform IDE that can be used by Databricks Lakehouse Platform users to develop locally.
• Visualization in MLFLOW experiment can be enhanced

Incentivized

• More and richer sources. For example, MailChimp is a source but the data you get from MailChimp is quite limited. I ended up writing my own scripts to take better advantage of MailChimp's API because Segment's integration was lacking.
• Better examples on how to set up event tracking. Pageview tracking is easy enough, but it would be nice if they had a sample app and corresponding code for it and showed you, via Git commits, how to add various kinds of events.

Incentivized

Because it is an amazing platform for designing experiments and delivering a deep dive analysis that requires execution of highly complex queries, as well as it allows to share the information and insights across the company with their shared workspaces, while keeping it secured.

in terms of graph generation and interaction it could improve their UI and UX

Incentivized

One of the best customer and technology support that I have ever experienced in my career. You pay for what you get and you get the Rolls Royce. It reminds me of the customer support of SAS in the 2000s when the tools were reaching some limits and their engineer wanted to know more about what we were doing, long before "data science" was even a name. Databricks truly embraces the partnership with their customer and help them on any given challenge.

Over the period it took us to set up, we kept going back to their enablement team to help us with the setup, and they were always ready and were very helpful in the entire process. Even with their documentation, they took the time out to help us work through the process. We've never had a message/email unanswered for more than an hour on working days.

Incentivized

The most important differentiating factor for Databricks Lakehouse Platform from these other platforms is support for ACID transactions and the time travel feature. Also, native integration with managed MLflow is a plus. EMR, Cloudera, and Hortonworks are not as optimized when it comes to Spark Job Execution. Other platforms need to be self-managed, which is another huge hassle.

Incentivized

We chose Twilio Segment for the good API integration and node resources, I would use Ontraport again, particularly if I didn't have the requirements for API and development/platform integration. Certainly the set up and management is easy and seamless with both the API and the user interface to use depending on circumstances and requirements.

Incentivized

• The ability to spin up a BIG Data platform with little infrastructure overhead allows us to focus on business value not admin
• DB has the ability to terminate/time out instances which helps manage cost.
• The ability to quickly access typical hard to build data scenarios easily is a strength.

Incentivized

• Segment has enabled us to get a full view of our front end activity, join it to our back-end activity, and get full visibility into our funnels and user activity.
• Segment lets us send events to ad tools with a full audit trail so all the numbers line up.
• Segment also brings data from other sources into our data warehouse, saving our data engineering time from building commodity connectors.