Microsoft's Azure Data Factory is a service built for all data integration needs and skill levels. It is designed to allow the user to easily construct ETL and ELT processes code-free within the intuitive visual environment, or write one's own code. Visually integrate data sources using more than 80 natively built and maintenance-free connectors at no added cost. Focus on data—the serverless integration service does the rest.
N/A
Fivetran
Score 8.4 out of 10
N/A
Fivetran replicates applications, databases, events and files into a high-performance data warehouse, after a five minute setup. The vendor says their standardized cloud pipelines are fully managed and zero-maintenance. The vendor says Fivetran began with a realization: For modern companies using cloud-based software and storage, traditional ETL tools badly underperformed, and the complicated configurations they required often led to project failures. To streamline and accelerate…
$0.01
Microsoft Visual Studio Code
Score 9.3 out of 10
N/A
Microsoft offers Visual Studio Code, an open source text editor that supports code editing, debugging, IntelliSense syntax highlighting, and other features.
Azure Data Factory helps us automate to schedule jobs as per customer demands to make ETL triggers when the need arises. Anyone can define the workflow with the Azure Data Factory UI designer tool and easily test the systems. It helped us automate the same workflow with …
Fivetran is more intuitive and easier to use than code-based ETL/ELT tools. The data modelling Fivetran performs makes the data more usable more quickly. Fivetran's dbt support and integration is unique.
Best scenario is for ETL process. The flexibility and connectivity is outstanding. For our environment, SAP data connectivity with Azure Data Factory offers very limited features compared to SAP Data Sphere. Due to the limited modelling capacity of the tool, we use Databricks for data modelling and cleaning. Usage of multiple tools could have been avoided if adf has modelling capabilities.
Fivetran's business model justifies the use-case where we require data from a single source basically a lot of data but if the requirement is not on the heavier side, Fivetran comes to costly operation when compared to its peers. Otherwise, I'll recommend Fivetran for stability and update and seamless service provider.
As a general workhorse IDE, Microsoft Visual Studio Codee is unmatched. Building on the early success of applications such as Atom, it has long been the standard for electron based IDEs. It can be outshone using IDEs that are dedicated to particular platforms, such as Microsoft Visual Studio Code for .net and the Jetbrains IDEs for Java, Python and others. For remote collaborative development, something like Zed is ahead of VSCode live share, which can be quite flakey.
Granularity of Errors: Sometimes, Azure Data Factory provides error messages that are too generic or vague for us, making it challenging to pinpoint the exact cause of a pipeline failure. Enhanced error messages with more actionable details would greatly assist us as users in debugging their pipelines.
Pipeline Design UI: In my experience, the visual interface for designing pipelines, especially when dealing with complex workflows or numerous activities, can become cluttered. I think a more intuitive and scalable design interface would improve usability. In my opinion, features like zoom, better alignment tools, or grouping capabilities could make managing intricate designs more manageable.
Native Support: While Azure Data Factory does support incremental data loads, in my experience, the setup can be somewhat manual and complex. I think native and more straightforward support for Change Data Capture, especially from popular databases, would simplify the process of capturing and processing only the changed data, making regular data updates more efficient
The customization of key combinations should be more accessible and easier to change
The auxiliary panels could be minimized or as floating tabs which are displayed when you click on them
A monitoring panel of resources used by Microsoft Visual Studio Code or plugins and extensions would help a lot to be able to detect any malfunction of these
Solid tool that provides everything you need to develop most types of applications. The only reason not a 10 is that if you are doing large distributed teams on Enterprise level, Professional does provide more tools to support that and would be worth the cost.
So far product has performed as expected. We were noticing some performance issues, but they were largely Synapse related. This has led to a shift from Synapse to Databricks. Overall this has delayed our analytic platform. Once databricks becomes fully operational, Azure Data Factory will be critical to our environment and future success.
Very easy and intuitive to setup and maintain as there usually are not that many options. Very well documented (e.g. how to setup each connector, how the schema looks like, any specific features of this connector etc.). Also the operation is intuitive, e.g. you have status pages, log pages, configuration pages etc. for each connector.
Microsoft Visual Studio Code earns a 10 for its exceptional balance of power and simplicity. Its intuitive interface, robust extension ecosystem, and integrated terminal streamline development. With seamless Git integration and highly customizable settings, it adapts perfectly to any workflow, making complex coding tasks feel effortless for beginners and experts alike.
Overall, Microsoft Visual Studio Code is pretty reliable. Every so often, though, the app will experience an unexplained crash. Since it is a stand-alone app, connectivity or service issues don't occur in my experience. Restarting the app seems to always get around the problem, but I do make sure to save and backup current work.
It runs pretty well and gets our data from point A to point cluster quickly enough. Honestly, it's not something I think about unless it breaks and that's pretty rare.
Microsoft Visual Studio Code is pretty snappy in performance terms. It launches quickly, and tasks are performed quickly. I don't have a lot of integrations other than CoPilot, but I suspect that if the integration partner is provisioned appropriately that any performance impact would be pretty minimal. It doesn't have a lot of bells and whistles (unless you start adding plugins left and right).
We have not had need to engage with Microsoft much on Azure Data Factory, but they have been responsive and helpful when needed. This being said, we have not had a major emergency or outage requiring their intervention. The score of seven is a representation that they have done well for now, but have not proved out their support for a significant issue
Active development means filing a bug on the GitHub repo typically gets you a response within 4 days. There are plugins for almost everything you need, whether it be linting, Vim emulation, even language servers (which I use to code in Scala). There is well-maintained official documentation. The only thing missing is forums. The closest thing is GitHub issues, which typically has the answers but is hard to sift through -- there are currently 78k issues.
Azure Data Factory helps us automate to schedule jobs as per customer demands to make ETL triggers when the need arises. Anyone can define the workflow with the Azure Data Factory UI designer tool and easily test the systems. It helped us automate the same workflow with programming languages like Python or automation tools like ansible. Numerous options for connectivity be it a database or storage account helps us move data transfer to the cloud or on-premise systems.
We never seriously considered using anything else. Our data engineers had used Fivetran extensively in previous roles so when it came time to make a decision, there wasn't much of a process. They gladly signed the contract with Fivetran pretty quickly.
Visual Studio Code stacks up nicely against Visual Studio because of the price and because it can be installed without admin rights. We don't exclusively use Visual Studio Code, but rather use Visual Studio and Visual Studio code depending on the project and which version of source control the given project is wired up to.
It is easily deployed with our Jamf Pro instance. There is actually very little setup involved in getting the app deployed, and it is fairly well self-contained and does not deploy a large amount of associated files. However, it is not particularly conducive to large project, multi-developer/department projects that involve some form of central integration.
