Apache Spark is a multi-language engine for executing data engineering, data science, and machine learning on single-node machines or clusters.
N/A
Dataloader.io
Score 10.0 out of 10
N/A
Dataloader.io delivers a cloud based solution to import and export information from Salesforce.
$99
per month
Presto
Score 10.0 out of 10
N/A
Presto is an open source SQL query engine designed to run queries on data stored in Hadoop or in traditional databases.
Teradata supported development of Presto followed the acquisition of Hadapt and Revelytix.
All the above systems work quite well on big data transformations whereas Spark really shines with its bigger API support and its ability to read from and write to multiple data sources. Using Spark one can easily switch between declarative versus imperative versus functional …
I think Presto is one of the best solutions out there today at the cutting edge for interactive query analysis. One of the challenges is presto is a niche tool for the interactive query use case and doesn't have the knobs and whistles as much as Spark. In the foreseeable future …
Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks. Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.
Replacing data. If we've put something in a category or a bucket that is no longer named that anymore because we've evolved with the times and we want to rebrand everything, it makes it way easier to do a quick import with the new terms.
Presto is for interactive simple queries, where Hive is for reliable processing. If you have a fact-dim join, presto is great..however for fact-fact joins presto is not the solution.. Presto is a great replacement for proprietary technology like Vertica
Extracting Salesforce attachments in original file format! I do not know of a tool that can do this better, or more efficiently! This is a huge benefit to companies that would like to extract attachments from Salesforce for tasks like data migrations.
Cross-object data extract within one file. You can pull data from related objects as long as there is a populated lookup from the object you are extracting, to another object (Child or Parent).
UI is simple and requires very little to no training. Given the acquisition of Mulesoft by Salesforce, I would not be surprised if DataLoader.IO is rolled out as the new global data loading tool for Salesforce.
Linking, embedding links and adding images is easy enough.
Once you have become familiar with the interface, Presto becomes very quick & easy to use (but, you have to practice & repeat to know what you are doing - it is not as intuitive as one would hope).
Organizing & design is fairly simple with click & drag parameters.
At the moment, I can't find a way to rename jobs. This would be useful to organize what was previously created hastily by techs in a rush.
A preview of the job, especially upserts, would take a great deal of stress away from some of us (especially those who are not so confident in their ETL practice).
A native vlookup equivalent may be a welcome addition.
Presto was not designed for large fact fact joins. This is by design as presto does not leverage disk and used memory for processing which in turn makes it fast.. However, this is a tradeoff..in an ideal world, people would like to use one system for all their use cases, and presto should get exhaustive by solving this problem.
Resource allocation is not similar to YARN and presto has a priority queue based query resource allocation..so a query that takes long takes longer...this might be alleviated by giving some more control back to the user to define priority/override.
UDF Support is not available in presto. You will have to write your own functions..while this is good for performance, it comes at a huge overhead of building exclusively for presto and not being interoperable with other systems like Hive, SparkSQL etc.
It is easy to use and doesn't require a security token, so I enjoy using it. It also doesn't require any download or installation, which is sometimes a blocker to gettingthings done if the company has limits. also, the dataloader.io is easy for other people to pick up, so others can have visibility into the data jobs that have occurred
If the team looking to use Apache Spark is not used to debug and tweak settings for jobs to ensure maximum optimizations, it can be frustrating. However, the documentation and the support of the community on the internet can help resolve most issues. Moreover, it is highly configurable and it integrates with different tools (eg: it can be used by dbt core), which increase the scenarios where it can be used
Dataloader definitely skews towards a more technical userbase. Users should be adept at manipulating data in spreadsheets and decipher JSON formatted error messaging. Additionally, there is a good amount of time need to set up the environment to map to the pertinent fields we are trying to adjust. While I would not recommend the typical account manager to use Dataloader, a typical operations manager should have no issue.
1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.
The utility itself is very self-explanatory and has enough information to guide you through the process. It has an intuitive experience for those familiar with data loading/exporting utilities. Outside of this, they have a Zendesk help center to log support requests and provide documentation to help guide you troubleshoot any issues that may be occurring.
Spark in comparison to similar technologies ends up being a one stop shop. You can achieve so much with this one framework instead of having to stitch and weave multiple technologies from the Hadoop stack, all while getting incredibility performance, minimal boilerplate, and getting the ability to write your application in the language of your choosing.
I have used salesforce inspector also for operations like import and export of data from custom objects but it doesn't work well when you have data in huge numbers. Instead of using Salesforce Inspector, one should go for Dataloader.io if the number of records is huge to be dealt with.
Presto is good for a templated design appeal. You cannot be too creative via this interface - but, the layout and options make the finalized visual product appealing to customers. The other design products I use are for different purposes and not really comparable to Presto.
HUGE time saving. When we need to clean or review data, we used to have to do it line by line. This can do the work within excel and make cleanup/management an afternoons work as opposed to a week.
Rollback what you did/change/deleted is relatively simple if you remember to back up the data you are manipulating.
