AWS Data Pipeline vs. MySQL

AWS Data Pipeline is a web service that helps you reliably process and move data between different AWS compute and storage services, as well as on-premises data sources, at specified intervals. With AWS Data Pipeline, you can regularly access your data where it’s stored, transform and process it at scale, and efficiently transfer the results to AWS services such as Amazon S3, Amazon RDS, Amazon DynamoDB, and Amazon EMR.

Incentivized

From my own perspective and the tasks that I perform on a daily basis, MySQL is perfect. It has a reasonable footprint, is fast enough and offers the security and flexibility I need. Everyone has their preferred applications and, no doubt, for larger data warehouses or more intensive applications, MySQL may have its limits, but for the area that I operate in, it's a great match.

Incentivized

• Helps you easily create complex data processing workloads
• Fault tolerant
• Highly available

Incentivized

• Security: is embedded at each level in MySQL. Authentication mechanisms are in place for configuring user access and even service account access to applications. MySQL is secure enough under the hood to store your sensitive information. Also, additional plugins are available that sit on top of MySQL for even tighter security.
• Widely adopted: MySQL is used across the industry and is trusted the most. Therefore, if you face any problems, simply Google it and you shall land in plenty of forums. This is a great relief as when you are in a need of help, you can find it right in your browser.
• Lightweight application: MySQL is not a heavy application. However, the data you store in the database can get heavy with time, but as in the configuration and MySql application files, those are not very heavy and can easily be installed on legacy systems as well.

Incentivized

• Pipeline Stuck in Pending Status
• Pipeline Component Stuck in Waiting for Runner Status
• EMR Cluster Fails With Error

Incentivized

• Although you can add the data you require as more and more data is added, the fixity of it becomes more critical.
• As the demand, size, and use of the system increase, you may also need to change or acquire more equipment on your servers, although this is an internal inconvenience for the company.

For teaching Databases and SQL, I would definitely continue to use MySQL. It provides a good, solid foundation to learn about databases. Also to learn about the SQL language and how it works with the creation, insertion, deletion, updating, and manipulation of data, tables, and databases. This SQL language is a foundation and can be used to learn many other database related concepts.

Incentivized

I give MySQL a 9/10 overall because I really like it but I feel like there are a lot of tech people who would hate it if I gave it a 10/10. I've never had any problems with it or reached any of its limitations but I know a few people who have so I can't give it a 10/10 based on those complaints.

The support staff is friendly, knowledgeable, and efficient. I only had to get part way through my explanations before they had a solution. They will walk you through a fix or actually connect in and fix the problem for you--or would if you can allow it. I've done it both ways with them. They are always forthcoming with 'how to do this if it happens again' information. I love working with MySQL support.

1. Estimate your data size. 2. Test, test, and test.

Incentivized

AWS data pipelines are easy to use over data factory for data engineers

Incentivized

MongoDB has a dynamic schema for how data is stored in 'documents' whereas MySQL is more structured with tables, columns, and rows. MongoDB was built for high availability whereas MySQL can be a challenge when it comes to replication of the data and making everything redundant in the event of a DR or outage.

Incentivized

• Easy to use
• Data engineers are able to create the data pipelines quickly and effectively
• Scalable and Fault tolerant

Incentivized

• As it is an open source solution through community solution, we can use it in a multitude of projects without cost license
• The acquisition by Oracle makes you need to contract support for the enterprise version
• If you have knowledge about oracle databases, you can get more out of the enterprise version

Incentivized