If you need a SQL-capable database-like solution that is file-based and embeddable in your existing Java Virtual Machine processes, Apache Derby is an open-source, zero cost, robust and performant option. You can use it to store structured relational data but in small files that can be deployed right alongside with your solution, such as storing a set of relational master data or configuration settings inside your binary package that is deployed/installed on servers or client machines.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
I find Qubole is well suited for getting started analyzing data in the cloud without being locked in to a specific cloud vendor's tooling other than the underlying filesystem. Since the data itself is not isolated to any Qubole cluster, it can be easily be collected back into a cloud-vendor's specific tools for further analysis, therefore I find it complementary to any offerings such as Amazon EMR or Google DataProc.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Apache Derby is SMALL. Compared to an enterprise scale system such as MSSQL, it's footprint is very tiny, and it works well as a local database.
The SPEED. I have found that Apache Derby is very fast, given the environment I was developing in.
Based in JAVA (I know that's an obvious thing to say), but Java allows you to write some elegant Object Oriented structures, thus allowing for fast, Agile test cases against the database.
Derby is EASY to implement and can be accessed from a console with little difficulty. Making it appropriate for everything from small embedded systems (i.e. just a bash shell and a little bit of supporting libraries) to massive workstations.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Providing an open selection of all cloud provider instance types with no explanation as to their ideal use cases causes too much confusion for new users setting up a new cluster. For example, not everyone knows that Amazon's R or X-series models are memory optimized, while the C and M-series are for general computation.
I would like to see more ETL tools provided other than DistCP that allow one to move data between Hadoop Filesystems.
From the cluster administration side, onboarding of new users for large companies seems troublesome, especially when trying to create individual cluster per team within the company. Having the ability to debug and share code/queries between users of other teams / clusters should also be possible.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Personally, I have no issues using Amazon EMR with Hue and Zeppelin, for example, for data science and exploratory analysis. The benefits to using Qubole are that it offers additional tooling that may not be available in other cloud providers without manual installation and also offers auto-terminating instances and scaling groups.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
SQLite is another open-source zero-cost file-based SQL-capable database solution and is a good alternative to Apache Derby, especially for non-Java-based solutions. We chose Apache Derby as it is Java-based, and so is the solution we embedded it in. However, SQLite has a similar feature set and is widely used in the industry to serve the same purposes for native solutions such as C or C++-based products.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Qubole was decided on by upper management rather than these competitive offerings. I find that Databricks has a better Spark offering compared to Qubole's Zeppelin notebooks.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Being Open source, the resources spent on the purchase of the product are ZERO.
Contrary to popular belief, open source software CAN provide support, provided that the developers/contributors are willing to answer your emails.
Overall, the ROI was positive: being able to experiment with an open source technology that could perform on par with the corporate products was promising, and gave us much information about how to proceed in the future.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
We like to say that Qubole has allowed for "data democratization", meaning that each team is responsible for their own set of tooling and use cases rather than being limited by versions established by products such as Hortonworks HDP or Cloudera CDH
One negative impact is that users have over-provisioned clusters without realizing it, and end up paying for it. When setting up a new cluster, there are too many choices to pick from, and data scientists may not understand the instance types or hardware specs for the datasets they need to operate on.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
