What users are saying about
175 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener'>trScore algorithm: Learn more.</a>
Score 8.3 out of 100
2 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener'>trScore algorithm: Learn more.</a>
Score 6.7 out of 100

Likelihood to Recommend

Azure DevOps Server (formerly TMS)

Well suited:
  • Large teams developing heterogeneous applications
  • Following SAFe process / Multiple Agile Release Trains (ARTs) / Portfolio management
  • Company undergoes regular external audits of their SDLC practices
  • Comprehensive reporting is required
  • Pipeline templating is required
  • Many git repositories are needed having to link to a unified backlog
  • Access to backlog work items and reporting is required for business stakeholders
Ross Borissov | TrustRadius Reviewer

TestRail

The integration with other tools, and their user friendly layout and design. It's very comfortable to use and is way better than other tools with the Automation Testing tools, thanks to the great API that is included. Sometimes the integration with Jira is a little faulty, but the links to that tool usually work well. It could be cool if it had a better following for the bug items that were registered on other tools.
Binoy Shah | TrustRadius Reviewer

Feature Rating Comparison

Test Management

Azure DevOps Server (formerly TMS)
TestRail
7.0
Centralized test management
Azure DevOps Server (formerly TMS)
TestRail
7.0
Manage test hosts and schedules
Azure DevOps Server (formerly TMS)
TestRail
6.0
Map tests to user stories
Azure DevOps Server (formerly TMS)
TestRail
6.0
Test execution reporting
Azure DevOps Server (formerly TMS)
TestRail
9.0
Defect management
Azure DevOps Server (formerly TMS)
TestRail
7.0

Pros

Azure DevOps Server (formerly TMS)

  • Vast range of Template for Automated Build and Release Pipeline with option to have conditional triggers
  • Code repository mechanism is aligned with GIT which is a well known public repository system
  • Bug Management is also an integral feature of this tool
  • Dashboard creation option of Build and Release pipeline is also available
Anonymous | TrustRadius Reviewer

TestRail

  • Beautiful and productive UI.
  • Nice filtering system allowing me to find what I need quickly.
Binoy Shah | TrustRadius Reviewer

Cons

Azure DevOps Server (formerly TMS)

  • Searching through code can be somewhat cumbersome. It would be nice if there was a way to do general searches in certain areas of the system.
  • Without proper training, the system can be confusing to navigate. This issue can be prevented with good training but it is something to be aware of.
  • Navigation can be clunky at times depending on where you are in the system. For power users, this is not a huge deal but it is a tad bit annoying.
Michael Martinez | TrustRadius Reviewer

TestRail

  • Integration with Jira seems [a] little faulty.
  • Increase the API call limit for different editions.
  • Better way to trigger automation suites from TestRail.
Binoy Shah | TrustRadius Reviewer

Likelihood to Renew

Azure DevOps Server (formerly TMS)

Azure DevOps Server (formerly TMS) 10.0
Based on 1 answer
Because we are a Microsoft Gold Partner we utilize most of their software and we have so much invested in Team Foundation Server now it would take a catastrophic amount of time and resources to switch to a different product.
Bill Starling | TrustRadius Reviewer

TestRail

No score
No answers yet
No answers on this topic

Usability

Azure DevOps Server (formerly TMS)

Azure DevOps Server (formerly TMS) 8.6
Based on 3 answers
Azure DevOps Server or TFS is a complete suite in itself. From Developer's machine where the code is developed to the production environment where the code is meant to run it take care of complete flow within itself. It acts as a code repository you can check-in check-out codes using GIT interface. It also acts as a Build and Automation Test tool which can help you to judge sanctity of your code. It further acts as a release manager to deploy your application to the production environment. And all these steps can also be performed without any manual intervention with the option to have approval processes. Hence its a perfect blend of all set of tools and capabilities required to bring code to production.
Anonymous | TrustRadius Reviewer

TestRail

No score
No answers yet
No answers on this topic

Support Rating

Azure DevOps Server (formerly TMS)

Azure DevOps Server (formerly TMS) 8.6
Based on 22 answers
I have not had to use the support for Azure DevOps Server. There have never been any issues where I was not able to figure it out or quickly resolve. Our Scrum Master has used support before though, and the service has always been prompt and clear with a customer-focus
Kellie Crawford | TrustRadius Reviewer

TestRail

No score
No answers yet
No answers on this topic

Implementation Rating

Azure DevOps Server (formerly TMS)

Azure DevOps Server (formerly TMS) 8.0
Based on 2 answers
Do research beforehand and, if possible, do a trial run before implementing into production environment.
Bill Starling | TrustRadius Reviewer

TestRail

No score
No answers yet
No answers on this topic

Alternatives Considered

Azure DevOps Server (formerly TMS)

We're with Microsoft since day one of starting the business. Since then, we've not encountered even a single reason not to be a part of the MS Family.The primary reason for our association is Innovation, Cost Effectiveness, Ease of Use and Great Support.Support and help materials are really awesome. Mostly within 30 minutes, any critical issues are addressed.
Ashish Tripathi | TrustRadius Reviewer

TestRail

TestRail definitely saves times. I work in a company that consists of several development teams, all of which have different processes. Some of the teams leverage test cases, some do not. I've noticed that the turn around time it takes for me to pick up a ticket, QA it, and then pass/fail/send feedback is much faster when there is a test case created as I'm not reading through ticket description/comments to figure out what needs to be QA'd.
Binoy Shah | TrustRadius Reviewer

Return on Investment

Azure DevOps Server (formerly TMS)

  • Automated testing saved 2 million a year over manual costs.
  • Automated testing ensured quality of code builds if the test did not pass the build did not go forward wasting both development and test time.
  • Burn down provide a visual of sprint status in real time.
Roxanna Aramjoo | TrustRadius Reviewer

TestRail

  • Improved test management process and collaboration inside team.
  • Increased transparency of QA work.
  • Team became more satisfied.
Binoy Shah | TrustRadius Reviewer

Pricing Details

Azure DevOps Server (formerly TMS)

General

Free Trial
Free/Freemium Version
Premium Consulting/Integration Services
Entry-level set up fee?
No

TestRail

General

Free Trial
Free/Freemium Version
Premium Consulting/Integration Services
Entry-level set up fee?
No

Rating Summary

Likelihood to Recommend

Azure DevOps Server (formerly TMS)
8.8
TestRail
7.0

Likelihood to Renew

Azure DevOps Server (formerly TMS)
10.0
TestRail

Usability

Azure DevOps Server (formerly TMS)
8.6
TestRail

Support Rating

Azure DevOps Server (formerly TMS)
8.6
TestRail

Implementation Rating

Azure DevOps Server (formerly TMS)
8.0
TestRail

Add comparison