SingleStore aims to deliver the world’s fastest distributed SQL database for data-intensive applications: SingleStoreDB, which combines transactional + analytical workloads in a single platform.
The in-memory tech is the best in class. If perf. is the hard limit then this is the No.1 to go. When data scales up, the cost of maintaining a big cluster with memory keep increasing (both the infrastructure cost and licensing cost) becomes high. Also the balance between …
I have tried using CSV as a back-end storage, yet I/O is very heavy, direct transit from spark to SingleStore DB (formerly MemSQL) in memory really beats.
Well suited: To most of the local run of datasets and non-prod systems - scalability is not a problem at all. Including data from multiple types of data sources is an added advantage. MLlib is a decently nice built-in library that can be used for most of the ML tasks. Less appropriate: We had to work on a RecSys where the music dataset that we used was around 300+Gb in size. We faced memory-based issues. Few times we also got memory errors. Also the MLlib library does not have support for advanced analytics and deep-learning frameworks support. Understanding the internals of the working of Apache Spark for beginners is highly not possible.
SingleStore HTAP engine is well suited for real-time analytics, fast ingestion, scaling OLTP system like MySQL. When you need to run reports or perform aggregates on billions of rows and you get result in seconds, you cannot get this experience with other OLTP engines. I wish DBtLab was a little more developer and supported for SingleStore. This would allow to perform better data transformation. You can use stored procedures, but DBTLabs has become a standard for dimensional modeling in data warehousing projects. This is probably why SingleStore has trouble piercing in the data warehouse world. It is definately capable to compete with Snowflake when it comes to scalability, query performance, data compression, but Snowflake has ravaged the data warehouse market in few years and large corporations have already invested lots of money in migrating into Snowflake. The SingleStore community needs to grow. Everyone who uses SingleStore loves it.
We wish the product had better support for High Availability of the aggregator. Currently the indexes generated by the two different aggregators are not in the same sequential space and so our apps have more burden to deal with HA.
More tools for debugging issues such as high memory usage would be good.
The price was the one that kept us away from purchasing for the first few years. Now we are able to afford due to a promotion that gives it at 25% of the list price. Not sure if we'll continue after the promotion offer expires in another 2 years.
The only thing I dislike about spark's usability is the learning curve, there are many actions and transformations, however, its wide-range of uses for ETL processing, facility to integrate and it's multi-language support make this library a powerhouse for your data science solutions. It has especially aided us with its lightning-fast processing times.
[Until it is] supported on AWS ECS containers, I will reserve a higher rating for SingleStore. Right now it works well on EC2 and serves our current purpose, [but] would look forward to seeing SingleStore respond to our urge of feature in a shorter time period with high quality and security.
SingleStore's performance is incredible. Our predictive algorithms went from taking 24-48 HOURS down to 15 minutes allowing our team to run those much more frequently. Previously, we were limited to about 60 requests per minute due to table locks. Implementing columnstore on SingleStore allowed us to receive 1000 requests per minute.
1. It integrates very well with scala or python. 2. It's very easy to understand SQL interoperability. 3. Apache is way faster than the other competitive technologies. 4. The support from the Apache community is very huge for Spark. 5. Execution times are faster as compared to others. 6. There are a large number of forums available for Apache Spark. 7. The code availability for Apache Spark is simpler and easy to gain access to. 8. Many organizations use Apache Spark, so many solutions are available for existing applications.
Very responsive to trouble tickets - Often, I think, the SingleStore's monitoring systems have already alerted the engineers by the time I get around to writing a ticket (about 10 - 20 mins after we see a problem). I feel like things are escalated nicely and SingleStore takes resolving trouble tickets seriously. Also SingleStore follows up after incidents to with a post mortem and actionable takaways to improve the product. Very satisfied here.
We allowed 2-3 months for a thorough evaluation. We saw pretty quickly that we were likely to pick SingleStore, so we ported some of our stored procedures to SingleStore in order to take a deeper look. Two SingleStore people worked closely with us to ensure that we did not have any blocking problems. It all went remarkably smoothly.
All the above systems work quite well on big data transformations whereas Spark really shines with its bigger API support and its ability to read from and write to multiple data sources. Using Spark one can easily switch between declarative versus imperative versus functional type programming easily based on the situation. Also it doesn't need special data ingestion or indexing pre-processing like Presto. Combining it with Jupyter Notebooks (https://github.com/jupyter-incubator/sparkmagic), one can develop the Spark code in an interactive manner in Scala or Python
Timescale was the biggest alternative option we looked at for SingleStore, however the requirement to learn a new syntax (due to not being SQL compatible) was our biggest pain point. Supporting a new language would require alterations to the Laravel framework, as this only offered SQL integration out of the box. This alteration would be time consuming and would limit our scope to future hiring due to the new syntax.
Faster turn around on feature development, we have seen a noticeable improvement in our agile development since using Spark.
Easy adoption, having multiple departments use the same underlying technology even if the use cases are very different allows for more commonality amongst applications which definitely makes the operations team happy.
Performance, we have been able to make some applications run over 20x faster since switching to Spark. This has saved us time, headaches, and operating costs.
As the overall performance and functionality were expanded, we are able to deliver our data much faster than before, which increases the demand for data.
Metadata is available in the platform by default, like metadata on the pipelines. Also, the information schema has lots of metadata, making it easy to load our assets to the data catalog.
