Connect micro services in your platform by using PubSub
Use Cases and Deployment Scope
We use Google Cloud Pub/Sub to communicate between micro services. By using a Pub/Sub topic, we can delegate not so time critical tasks to be processed asynchronously by another micro service.
Using a BigQuery subscription, we can automatically write events sent to a Pub/Sub topic into a BigQuery table.
Pros
- Push subscription that can trigger a HTTP endpoint
- Supports retries
- BigQuery subscription for writing into a BigQuery table
Cons
- Allow write and read of messages more than 10 MB
- Support rate at which HTTP endpoints are triggered to reduce scale requirements on downstream services
- Support replay of messages
Return on Investment
- Using a BigQuery subscription reduces development effort.
- Since there is no rate control built into Pub/Sub it can result in overload of downstream services (including databases) when the incoming traffic is bursty in nature. We had to built a custom solution to address this scenario.
- Helps integration of services spread across multiple cloud environments.
Usability
Alternatives Considered
Amazon Simple Queue Service (SQS) and Apache Kafka
Other Software Used
AlloyDB, ExpressVPN, Sprinto, Google BigQuery, Firebase, Sentry
