Apache Hive vs. Db2 vs. PostgreSQL

Software work execution is on a large scale, it is good to use for new projects or organizational changes, data lineage mapping has always been dubious but this one has had good results. You can store and synchronize data from different departments, the storage process can be manual but it is best automated.

Incentivized

I have primarily used it as the basis for a SIS - but I have migrated more than a few systems from there database systems to DB2 (Filemaker, MySQL, etc.). DB2 does have a better structural approach, as opposed to Filemaker, which allows for more data consistency, but this can also lead to an inflexibility that can sometimes be counterintuitive when attempting to compensate for the flexibility of the work environment as Schools tend to have an all in one approach.

Incentivized

PostgreSQL is best used for structured data, and best when following relational database design principles. I would not use PostgreSQL for large unstructured data such as video, images, sound files, xml documents, web-pages, especially if these files have their own highly variable, internal structure.

Incentivized

• Apache Hive allows use to write expressive solutions to complex problems thanks to its SQL-like syntax.
• Relatively easy to set up and start using.
• Very little ramp-up to start using the actual product, documentation is very thorough, there is an active community, and the code base is constantly being improved.

Incentivized

• While we query a large set of data, the results are generally available within a minute or so.
• Always reliable - I have never experienced an application going down.
• It is easy to write queries and find tables and columns.
• We can log in smoothly without any headaches.

Incentivized

• It works well with external data sources and runs on platforms with stable performance.
• Clients can rest assured that their personal information will be safe and secure.
• Many forums discuss setup and usage, and most are free.
• Adding tooling applications to a computer is unlimited.
• PostgreSQL runs on many OS platforms and supports ANSI SQL, stored procedures, and triggers.

• Some queries, particularly complex joins, are still quite slow and can take hours
• Previous jobs and queries are not stored sometimes
• Switching to Impala can sometimes be time-consuming (i.e. the system hangs, or is slow to respond).
• Sometimes, directories and tables don't load properly which causes confusion

Incentivized

• Learning curve for DB resources - Improvements to UI or native command line built-ins can help with increasing efficiencies for DB resources
• Better resource utilization monitoring and recommendations
• Continue to adopt support for modern frameworks and languages making it easier for organizations to see making Db2 the easy first choice

Incentivized

• Clearer indications on what is the query plan, to optimize the query
• More out of the box, Postgres specific, SQL functions
• It would be nice to have a more visual aid of the relationship between all tables, but possibly this depend more on the UI used

Incentivized

Since I do not know the second data warehouse solution that integrate with HDFS as well as Hive.

The DB2 database is a solid option for our school. We have been on this journey now for 3-4 years so we are still adapting to what it can do. We will renew our use of DB2 because we don’t see. Major need to change. Also, changing a main database in a school environment is a major project, so we’ll avoid that if possible.

Incentivized

As a needed software for day to day development activities

Incentivized

Hive is a very good big data analysis and ad-hoc query platform, which supports scaling also. The BI processes can be easily integrated with Hadoop via the Hive. It can deal with a much larger data set that traditional RDBMS can not. It is a "must-have" component of the big data domain.

Incentivized

You have to be well versed in using the technology, not only from a GUI interface but from a command line interface to successfully use this software to its fullest.

Incentivized

Postgresql is the best tool out there for relational data so I have to give it a high rating when it comes to analytics, data availability and consistency, so on and so forth. SQL is also a relatively consistent language so when it comes to building new tables and loading data in from the OLTP database, there are enough tools where we can perform ETL on a scalable basis.

Incentivized

I have never had DB2 go down unexpectedly. It just works solidly every day. When I look at the logs, sometimes DB2 has figured out there was a need to build an index. Instead of waiting for me to do it, the database automatically created the index for me. At my current company, we have had zero issues for the past 8 years. We have upgrade the server 3 times and upgraded the OS each time and the only thing we saw was that DB2 got better and faster. It is simply amazing.

Incentivized

PostgreSQL's availability is top notch. Apart from connection time-out for an idle user, the database is super reliable.

Incentivized

The performances are exceptional if you take care to maintain the database. It is a very powerful tool and at the same time very easy to use. In our installation, we expect a DB machine on the mainframe with access to the database through ODBC connectors directly from branch servers, with fabulous end users experience.

Incentivized

The data queries are relatively quick for a small to medium sized table. With complex joins, and a wide and deep table however, the performance of the query has room for improvement.

Incentivized

Apache Hive is a FOSS project and its open source. We need not definitely comment on anything about the support of open source and its developer community. But, it has got tremendous developer support, awesome documentation. I would justify the fact that much support can be gathered from the community backup.

Incentivized

Easily the best product support team. :) Whenever we have questions, they have answered those in a timely manner and we like how they go above and beyond to help.

Incentivized

There are several companies that you can contract for technical support, like EnterpriseDB or Percona, both first level in expertise and commitment to the software.

But we do not have contracts with them, we have done all the way from googling to forums, and never have a problem that we cannot resolve or pass around. And for dozens of projects and more than 15 years now.

Incentivized

the material was very clear and all subjects have been handled

Incentivized

The online training is request based. Had there been recorded videos available online for potential users to benefit from, I could have rated it higher. The online documentation however is very helpful. The online documentation PDF is downloadable and allows users to pace their own learning. With examples and code snippets, the documentation is great starting point.

Incentivized

db2 work well with the application, also the replication tool can keep it up

The online documentation of the PostgreSQL product is elaborate and takes users step by step.

Incentivized

Besides Hive, I have used Google BigQuery, which is costly but have very high computation speed. Amazon Redshift is the another product, I used in my recent organisation. Both Redshift and BigQuery are managed solution whereas Hive needs to be managed

Incentivized

DB2 was more scalable and easily configurable than other products we evaluated and short listed in terms of functionality and pricing. IBM also had a good demo on premise and provided us a sandbox experience to test out and play with the product and DB2 at that time came out better than other similar products.

Incentivized

Although the competition between the different databases is increasingly aggressive in the sense that they provide many improvements, new functionalities, compatibility with complementary components or environments, in some cases it requires that it be followed within the same family of applications that performs the company that develops it and that is not all bad, but being able to adapt or configure different programs, applications or other environments developed by third parties apart is what gives PostgreSQL a certain advantage and this diversification in the components that can be joined with it, is the reason why it is a great option to choose.

Incentivized

By
using DB2 only to support my IzPCA activities, my knowledge here
is somewhat limited.

Anyway,
from what I was able to understand, DB2 is extremely scallable.

Maybe the information below could serve as an example of scalability.

Customer have an huge mainframe environment, 13x z15 CECs, around
80 LPARs, and maybe more than 50 Sysplexes (I am not totally sure about this
last figure...)

Today
we have 7 IzPCA
databases, each one in a distinct Syplex.

Plans
are underway to have, at the end, an small LPAR, with only one DB2 sub-system,
and with only one database, then transmit the data from a lot of other LPARs,
and then process all the data in this only one database.

The
IzPCA collect process (read the data received, manipulate it, and insert rows
in the tables) today is a huge process, demanding many elapsed
hours, and lots of CPU.

Almost
100% of the tables are PBR type, insert jobs run in parallel, but in 4 of the 7
database, it is a really a huge and long process.

Combining
the INSERTs loads from the 7 databases in only one will be impossible.......,,,,

But,
IzPCA recently introduced a new feature, called "Continuous
Collector".

By
using that feature, small amounts of data will be transmited to the central
LPAR at every 5 minutes (or even less), processed immediately,in
a short period of time, and with small use of CPU,
instead of one or two transmissions by day, of very large amounts of data and
the corresponding collect jobs occurring only once or twice a day, with long
elapsed times, and huge comsumption of CPU

I
suspect the total CPU seconds consumed will be more or less the same in
both cases, but in the new method it will occur in small bursts
many times a day!!

Incentivized

The DB is reliable, scalable, easy to use and resolves most DB needs

Incentivized

• Apache hive is secured and scalable solution that helps in increasing the overall organization productivity.
• Apache hive can handle and process large amount of data in a sufficient time manner.
• It simplifies writing SQL queries, hence helping the organization as most companies use SQL for all query jobs.

Incentivized

• Negative: Difficult and manual deployment
• Negative: Missing assistants from common monitoring metrics
• Positive: Stability
• Positive: Performance
• Positive: Resiliency and high availability (HADR)
• Positive: Data Replication (Q-Rep)
• Positive: Interaction with storage subsystems for backups (TSM, SVC)
• Positive: Gigantic monitoring features in the form of table functions

Incentivized

• Easy to administer so our DevOps team has only ever used minimal time to setup, tune, and maintain.
• Easy to interface with so our Engineering team has only ever used minimal time to query or modify the database. Getting the data is straightforward, what we do with it is the bigger concern.
• It's free. You can't beat that.

Incentivized