Appium vs. Katalon vs. Xamarin

1. It's open source which supports range of languages, operating systems and languages. Well suited for Android and IOS mobile automation. Supports all kinds of apps, which makes it flexible and robust mobile testing tool 2. It is less appropriate where we need intercept network call to verify the API calls. Extensive coding experience is required to work Appium

Incentivized

Katalon TestOps is well suited for an oraganization who want to organize their automation collections and have everything in one place. It provides good insight on test runs, failures and also helps in defect tracking. It is extremely easy to schedule and run test suites automatically. With the help of test history, it becomes easier to triage the report and identify if it is a product issue or a test flakiness. Also with the automation run scheduling features with custom options, user do not have to manually trigger the runs and just run the scheduler to do the job.

Incentivized

If you are required to develop applications that are cross-platformed, Xamarin is a great tool to use. It will help save time and effort from your development team to be able to build applications seamlessly for android, IOS, Windows, and web on a single platform instead of requiring multiple tools to get the job done.

Incentivized

• It uses WebDriver API so it makes it easy to use for former web test automation engineers.
• It can be managed via the command line via an extensive set of parameters.
• It handles implicit waits at the server side that is especially valuable in distributed infrastructure.

Incentivized

• Easy to setup and quick to get started: As Katalon comes with built-in features a new automation tester can automate easily. Script creation time is quick.
• Test Analytics: Katalon also provides analytics which tells you the pattern of your automated tests at a different instance of time.
• Third Party Integration: Katalon provides amazing integration with 3rd party tools like JIRA, ALM, GIT, JENKINS, etc.
• Image-based testing: Katalon also provides inbuilt image based test automation.

Incentivized

• Xamarin allows you to write cross platform code. This allows companies to build apps more quickly by writing less code. Having code abstracted and reused across multiple platforms allows for more testing and less issues overall.
• The ability to use Visual Studio is a huge plus. Visual Studio is one of the best IDE's available and being able to write cross platforms apps while in a great IDE makes everything less painful.
• Xamarin is now free with a large company backing. This means that bugs on the platform get fixed more quickly and there is a large community of developers.

Incentivized

• Element browser sometimes is unreliable and has sporadic fails.
• Appium running is a bit slow, compared to tests written with Appium and with Espresso or XCTest.

Incentivized

• Integration with Azure is painstakingly slow as it takes too much time to upload reports to Azure. When raised with the support team i was advised to contact Microsoft who promptly and correctly replied that its an issue with the katalon plugin.
• Non Responsive Customer Support. Several mails were replied late.
• The Desktop app uses too much memory when booting up and running tests even on a 16gb RAM.

• Forms - not 100% there. Still needs work but is production ready.
• iOS - sometimes errors can be hard to understand, if they even show up.
• Insights - Xamarin offers their own crash analytics software. However, it's not perfect and sometimes doesn't pick up crashes.

Incentivized

I am rating 10/10 because Appium can use with multiple programming languages with multiple Test engines. Also, it works for both Android and iOS.

Incentivized

Xamarin has been great for developing different projects efficiently and effectively. It's nice to reuse the core business logic across different platforms so that there are less to maintain and little replications are needed. The biggest benefit is that C# programmers do not have to learn a different language to do mobile development.

Incentivized

I would like to give 9/10 rating to Appium because of it can easily integrate with popular frameworks and CI/CD tools, as well as it is reliable, flexible and easy to use. The setup can bit complex in initial step, but once on configured it's very easy to use and enables stable and scalable mobile automation for real and cloud devices.

Incentivized

Due to the slowness, there is an intermediate timeout. There is no other problem.

Incentivized

If you are required to develop applications that are cross-platformed, Xamarin is a great tool to use. It will help save time and efforts from your development team to be able to build applications seamlessly for android, IOS, windows, and web on a single platform instead of requiring multiple tools to get the job done

Incentivized

I love the support. They have provided answers and solutions for most of our questions or doubts that could come up. They also have provided basic tutorials on how to get familiarized with the program.

Incentivized

I never had to contact support for any help. Most of the problems we ran into, we were able to identify and use peer support through blogs and other internet sources to resolve the problems. There are plenty of sources online which provide tutorials, discuss problems, etc. Example: StackOverflow

Incentivized

Just with any programming tasks, have a plan first. Design out the system, spend time to build it correctly the first time and have plenty of testing and user acceptance opportunities. Xamarin was easy to implement for a C# programmer. However, you need to do tutorials to realize the platform's capabilities.

Incentivized

If you're an Apple developer, you use Xcode. It's practically a forced necessity. For system testing though, it doesn't have to be. You can have your development team focus on unit and integration tests in their platform and another team automate acceptance tests with a language they are more familiar with.

Incentivized

Katalon Studio is above both the tools is due to its codeless test automation feature and having API, desktop, web, and mobile automation in one place but obviously there is room for improvement when desktop automation is of primary importance in the automation.

Incentivized

Xamarin runs natively on MacOS, and the debugger and other integration and auto-complete tools are far better than Eclipse for C# .NET. It also carries much of the plugin/add-on capabilities that are so desirable on Atom. Eclipse is a better for generalized software development, provided a developer is comfortable switching between the IDE the command line for certain parts of their workflow, like building, package management, or debugging. But for C# .NET development on MacOS specifically, Xamarin is the best product I've used for the job.

Incentivized

• Appium is open source, so it's free. That's budget friendly right there.
• The ability to write mobile automation tests has saved considerable time for our manual test team, but that is true with most automation tests.
• We use Sauce Labs with our other automation, but Appium works great with Sauce Labs, as well, if I needed to run on emulators and simulators.

Incentivized

• Support can be better: As mentioned before, katlon community is still growing but is currently a small community. If you are stuck, its hard to find help in the community and one has to reach out to the support for help. This takes a while before you can get solution to your problem.
• Result upload takes little longer time: It is been observed that uploading tests results takes little longer.
• Sometimes it gets hard to integrate: Since this is fairly a new tool, requires lot of learning curve to get the job done effectively.

Incentivized

• Saves development time and deliver fast.
• Allows inhouse developers build both Android and iOS application without switching languages.
• Allows use coding in C# in Visual studio IDE from which we can code in different languages. We don't need multiple IDEs installed

Incentivized