Apache Hive vs. Google BigQuery

Software work execution is on a large scale, it is good to use for new projects or organizational changes, data lineage mapping has always been dubious but this one has had good results. You can store and synchronize data from different departments, the storage process can be manual but it is best automated.

Incentivized

Google BigQuery is great for being the central datastore and entry point of data if you're on GCP. It seamlessly integrates with other Google products, meaning you can ingest data from other Google products with ease and little technical knowledge, and all of it is near real-time. Being serverless, BigQuery will scale with you, which means you don't have to worry about contention or spikes in demand/storage. This can, however, mean your costs can run away quickly or mount up at short notice.

Incentivized

• Apache Hive allows use to write expressive solutions to complex problems thanks to its SQL-like syntax.
• Relatively easy to set up and start using.
• Very little ramp-up to start using the actual product, documentation is very thorough, there is an active community, and the code base is constantly being improved.

Incentivized

• First and foremost - Google BigQuery is great at quickly analyzing large amounts of data, which helps us understand things like customer behavior or product performance without waiting for a long time.
• It is very easy to use. Anyone in our team can easily ask questions about our data using simple language, like asking ChatGPT a question. This means everyone can find important information from our data without needing to be a data expert.
• It plays nicely with other tools we use, so we can seamlessly connect it with things like Google Cloud Storage for storing data or Data Studio for creating visual reports. This makes our work smoother and helps us collaborate better across different tasks.

Incentivized

• Some queries, particularly complex joins, are still quite slow and can take hours
• Previous jobs and queries are not stored sometimes
• Switching to Impala can sometimes be time-consuming (i.e. the system hangs, or is slow to respond).
• Sometimes, directories and tables don't load properly which causes confusion

Incentivized

• It is challenging to predict costs due to BigQuery's pay-per-query pricing model. User-friendly cost estimation tools, along with improved budget alerting features, could help users better manage and predict expenses.
• The BigQuery interface is less intuitive. A more user-friendly interface, enhanced documentation, and built-in tutorial systems could make BigQuery more accessible to a broader audience.

Incentivized

Since I do not know the second data warehouse solution that integrate with HDFS as well as Hive.

We have to use this product as its a 3rd party supplier choice to utilise this product for their data side backend so will not be likely we will move away from this product in the future unless the 3rd party supplier decides to change data vendors.

Incentivized

Hive is a very good big data analysis and ad-hoc query platform, which supports scaling also. The BI processes can be easily integrated with Hadoop via the Hive. It can deal with a much larger data set that traditional RDBMS can not. It is a "must-have" component of the big data domain.

Incentivized

web UI is easy and convenient. Many RDBMS clients such as aqua data studio, Dbeaver data grid, and others connect. Range of well-documented APIs available. The range of features keeps expanding, increasing similar features to traditional RDBMS such as Oracle and DB2

Apache Hive is a FOSS project and its open source. We need not definitely comment on anything about the support of open source and its developer community. But, it has got tremendous developer support, awesome documentation. I would justify the fact that much support can be gathered from the community backup.

Incentivized

BigQuery can be difficult to support because it is so solid as a product. Many of the issues you will see are related to your own data sets, however you may see issues importing data and managing jobs. If this occurs, it can be a challenge to get to speak to the correct person who can help you.

Incentivized

Besides Hive, I have used Google BigQuery, which is costly but have very high computation speed. Amazon Redshift is the another product, I used in my recent organisation. Both Redshift and BigQuery are managed solution whereas Hive needs to be managed

Incentivized

PowerBI can connect to GA4 for example but the data processing is more complicated and it takes longer to create dashboards. Azure is great once the data import has been configured but it's not an easy task for small businesses as it is with BigQuery.

Incentivized

None so far. Very satisfied with the transparency on contract terms and pricing model.

Google Support has kindly provide individual support and consultants to assist with the integration work. In the circumstance where the consultants are not present to support with the work, Google Support Helpline will always be available to answer to the queries without having to wait for more than 3 days.

• Apache hive is secured and scalable solution that helps in increasing the overall organization productivity.
• Apache hive can handle and process large amount of data in a sufficient time manner.
• It simplifies writing SQL queries, hence helping the organization as most companies use SQL for all query jobs.

Incentivized

• Previously, running complex queries on our on-premise data warehouse could take hours. Google BigQuery processes the same queries in minutes. We estimate it saves our team at least 25% of their time.
• We can target our marketing campaigns very easily and understand our customer behaviour. It lets us personalize marketing campaigns and product recommendations and experience at least a 20% improvement in overall campaign performance.
• Now, we only pay for the resources we use. Saved $1 million annually on data infrastructure and data storage costs compared to our previous solution.

Incentivized