Aerospike vs. AWS Data Pipeline

We were developing an advertisement time auction application, where we had to store the client's personal details, advertisement-related details, location, and many other details. Moreover, we required a promotion, cookies, and a few more details from the front end. All this information is heavy in terms of size and cannot be lost if the server crash. So, we required an extremely fast disk database with high scalability and low throughput.

Incentivized

AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals. With AWS Data Pipeline, you can regularly access your data where it’s stored, transform and process it at scale, and efficiently transfer the results to AWS services such as Amazon S3, Amazon RDS, Amazon DynamoDB, and Amazon EMR.

Incentivized

• Low latency
• Stable
• Highly configurable
• increasing features

Incentivized

• Helps you easily create complex data processing workloads
• Fault tolerant
• Highly available

Incentivized

• Load balancing per network segments.
• Reduction in price.
• Cross datacenter replication usage isn't so straightforward. Sometimes cross dc replication can have issues of bad data..

Incentivized

• Pipeline Stuck in Pending Status
• Pipeline Component Stuck in Waiting for Runner Status
• EMR Cluster Fails With Error

Incentivized

If money isn't an issue, and you're not on the cloud, then I'd go with Aerospike. If you're the cloud ie, aws or azure, then i'd stick with dynamoDB or Cosmos then. Aerospike is definitely not something you want to put into the cloud. It doesn't work well w/ cross regions. If cross DC, you'll have to write some stuff for data integrity checks.

Incentivized

We were in dilemma in deciding the database and it was the first time we were using Aerospike. Eventually, everything went as expected and resolved the client's requirement along with positive feedback and appreciation

Incentivized

You pay for the level of support.

Incentivized

Aerospike is much more performant than MongoDB, however there is much greater community adoption and support for mongo

Incentivized

AWS data pipelines are easy to use over data factory for data engineers

Incentivized

• increased response time
• minimal managerial resource required
• developer can start using with shallow learning curve

Incentivized

• Easy to use
• Data engineers are able to create the data pipelines quickly and effectively
• Scalable and Fault tolerant

Incentivized