Amazon Simple Notification Service (SNS) vs. Google Cloud Dataflow

The Amazon SNS service is well suited to support event notifications, monitoring applications, workflow systems, time-sensitive information updates, and mobile applications that generates or consumes notifications. It can be used to relay time-critical events to mobile applications and devices. It provides significant advantages to developers who build mobile applications that rely on real-time events. It is not well suited for hybrid cross platform mobile application frameworks at this juncture. An optimal version to meet the needs of a cross platform mobile developer is needed as generally the frameworks are not meant to manage real-time events. It is also not suited for cases where the queue management needs improvement or requires special workflows/tooling.

Incentivized

It is best in cases where you have batch as well as streaming data. Also in some cases where you have batch data right now and in future you will get streaming data. In those cases Dataflow is very good. Also in cases where most of your infra is on GCP. It might not be good when you already are on AWS or Azure. And also you want in-depth control over security and management. Then you can directly use Apache beam over Dataflow.

Incentivized

• Built in for quicker setup within AWS ecosystem.
• Trusted as you control the users and configuration via IAM and easy access controls.
• Can be sent to S3 simple storage or for long term storage if required.
• Can be used in many regions, same configs.

Incentivized

• Streaming, Real time work load
• Batch processing
• Auto scaling
• flexible pricing

• At times you receive access denied errors which are annoying.
• Rarely do you receive internal failure errors where you can't access the information. It is rare but it does happen.
• You are required to add an MWS Authentication Token every so often. I wish it would pull that information automatically for you so you don't have to go searching for it.

Incentivized

• More templates for Bigquery and App Engine. There is only limited options for templates so the things we use can limit.
• I would like native connectors for Excel (XLSX) to reduce the need for custom wrappers in financial pipelines.
• Debugging Google Cloud Dataflow using only logs in Cloud Logging can be overwhelming sometimes, and it’s not always obvious which specific element in the flow caused a failure. IT uses a lot of time.

Incentivized

It is useful for applications developed using event driven architecture. It helps in tracking and logging the events in a very timely and efficient manner. The dashboards are a little difficult to implement. But overall it is very easy to integrate with other AWS services like Lambda, API GW, S3 and DynamoDB. The permissions to access should be resolved before using it.

Incentivized

It really saved a lot of time and it's flexibility really can give you infra which is future-proof for most of the use cases may it be streaming or batch data. And with this you can avoid use of resource-heavy big data offerings.

Incentivized

Amazon Simple Notification Sevices (SNS) support depends on your usage pattern and definitely on your support plan as an enterprise with AWS. Before reaching out to support you should read their documentation, as they have mentioned almost all the common issues and their solutions there. However, for specific issues, they generally respond in 1-2 business days.

Incentivized

Amazon’s SNS is incredibly easy to set up compared to the
more powerful, but complex, Kafka flavours.

SNS’s core advantages are –

· no setup/no maintenance

· either a queue (SQS) or a topic (SNS)

· various limitations (on size, how long a message lives, etc)

· limited throughput: you can do batch and concurrent requests, but still
achieving high throughputs would be expensive

· SNS has notifications for email, SMS, SQS, HTTP built-in.

· no "message stream" concept Overall, it would be the best choice to get into the concepts of Pub/Sub concepts as although it has limitations it can provide significant capabilities and solutions

Incentivized

Google Cloud Dataproc Cloud Datafusion

• Amazon Simple Notification Service (SNS) saved us a lot of extra coding time by providing straightforward functionality we needed in our ad campaign automation tool.
• Amazon Simple Notification Service (SNS) allows us to maintain a consistent, serverless model within our applications.

Incentivized

• cost saving from managing our own data center for ETL servers
• consumption based pricing
• with auto scaling feature, we were able to expand components to support work load