Seamless and near real-time integration for GCP users.
September 21, 2024
Seamless and near real-time integration for GCP users.
Score 8 out of 10
Vetted Review
Verified User
Overall Satisfaction with Google BigQuery
BigQuery fits naturally into the GCP estate, seamlessly integrating with GSheets and allowing users to ingest data into BigQuery with little to no technical knowledge. This is beneficial for GForm data, which is naturally presented via GSheet. With the real-time integration GSheets has with BigQuery, this form of data can then be in your data stack in mere seconds. This is really helpful when we want to collect feedback data or review a process using GForms as the method of data collection.
Pros
- Realtime integration with Google Sheets.
- GSheet data can be linked to a BigQuery table and the data in that sheet is ingested in realtime into BigQuery. It's a live 'sync' which means it supports insertions, deletions, and alterations. The only limitation here is the schema'; this remains static once the table is created.
- Seamless integration with other GCP products.
- A simple pipeline might look like this:-
- GForms -> GSheets -> BigQuery -> Looker
- It all links up really well and with ease.
- One instance holds many projects.
- Separating data into datamarts or datameshes is really easy in BigQuery, since one BigQuery instance can hold multiple projects; which are isolated collections of datasets.
Cons
- Can't customise compute methods.
- In some cases, it's nice to limit a specific query, user, database, or process to a specific compute engine. This helps standardize costs, run time, and the fallout on shared resources. BigQuery's ease of use makes it simple for most users but difficult to customize for power uses.
- Potential for vendor lock-in.
- If your data stack is heavily reliant on integrations with other GCP products, you'll find it hard to move. Even if you just move from BigQuery to another supplier, you'll need to set up integrations from your current GCP products to the external vendor. Or find a vendor that integrates with GCP well.
- Autoscaling might cause unexpected costs.
- Since BigQuery charges on storage, compute, and streaming inserts, if any of these become unexpectedly in-demand (especially compute), your costs may skyrocket without much notice.
- The time it takes from a user submitting a feedback form to the data being available in a dashboard is mere hours, and this is mostly due to the scheduling of the transformation pipeline itself. This means that stakeholders can review the data being collected in near real-time.
- Not having to worry about scaling up compute clusters means that during periods of heavy usage, we don't need to pay attention too much, other than keeping an eye on the costs.
- The low technical bar for entry when ingesting GForm data into BigQuery means that many simple, repetitive tasks can be outsourced to the data consumers themselves, freeing up developer time.
We actually use Snowflake and BigQuery in tandem because they both currently meet various needs. Redshift, however, has barely been used since our migration away from it. In the case of both Snowflake and BigQuery, they beat Redshift by a long shot. The main reasons are their clearer UI, easier integration, modern architectural design, and generally more efficient data storage and computing process.
Do you think Google BigQuery delivers good value for the price?
Yes
Are you happy with Google BigQuery's feature set?
Yes
Did Google BigQuery live up to sales and marketing promises?
I wasn't involved with the selection/purchase process
Did implementation of Google BigQuery go as expected?
I wasn't involved with the implementation phase
Would you buy Google BigQuery again?
Yes
Comments
Please log in to join the conversation