Good Cloud ETL solution Tool helps in Building Complex Data Pipeline (real time and batch processing both)
Use Cases and Deployment Scope
Informatica Cloud Service is used in several departments. In upcoming months we will start using it in more departments. We already had datapipeline setup using Python spark and other paid tool. But most department are migrating the data pipeline and creating good ETL model following SCD type 1,2 Load.
We are migrating most of our complex data pipeline to informatica one.
Reason to migrate is for cloud solution ETL. [The] main advantage of cloud ETL [is that] it can be more scalable and cost-effective solution: when we require more storage and more resource we can pay more, so now we can just pay for what we need.
Informatica also has wide user base so even getting the right talent is not an issue for our organization, if someone used Informatica he/she can easily start with Informatica Cloud service. In the future, the Cloud will be excellent solution for resource and cost optimization.
Pros
- Building Data pipeline in very easy (mapping and workflow section is highly configurable and mostly with all option)
- Easy debugging mode, you can backtrack and check which transformation has issue
- Easy workflow trigger using shell script, python or control-M
- Cloud base solution is future which will be very helpful for scalable solution
Cons
- Our previous solution is pure in house development, which is based on free [solutions]. So Informatica is costly w.r.t freeware.
- Sometimes team architecture is based on AWS so they are not going to switch on Informatica solution
Likelihood to Recommend
If your company does not have a scalable ETL solution, the current data pipeline is just data migration using my SQL or python API, you should switch to a good ETL solution - you can achieve [this with] Informatica Intelligent Cloud Services.
[Well suited] if you want to use good data warehousing concepts in your practical warehouse, no hardcore developer is required. Ease in debugging.
In the future (next 5 Years) most process will be setup on cloud base solution. Reason of cloud solution is [it's] flexible resources, pay for what you use, no in-house storage processing required, and the availability of talented people in [the job] market [who know] Informatica.
