<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener'>Customer Verified: Read more.</a>
258 Ratings
Top Rated
156 Ratings

Azure DevOps Server (formerly TMS)

<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener'>Customer Verified: Read more.</a>
258 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener'>trScore algorithm: Learn more.</a>
Score 8.3 out of 100
Top Rated
156 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener'>trScore algorithm: Learn more.</a>
Score 8.6 out of 100

Attribute Ratings

  • Azure DevOps Server is rated higher in 4 areas: Likelihood to Recommend, Likelihood to Renew, Support Rating, Implementation Rating
  • ReadyAPI is rated higher in 1 area: Usability

Likelihood to Recommend

8.6

Azure DevOps Server (formerly TMS)

86%
48 Ratings
8.5

ReadyAPI

85%
67 Ratings

Likelihood to Renew

10.0

Azure DevOps Server (formerly TMS)

100%
1 Rating
7.4

ReadyAPI

74%
18 Ratings

Usability

8.7

Azure DevOps Server (formerly TMS)

87%
3 Ratings
9.9

ReadyAPI

99%
3 Ratings

Availability

Azure DevOps Server (formerly TMS)

N/A
0 Ratings
9.0

ReadyAPI

90%
2 Ratings

Performance

Azure DevOps Server (formerly TMS)

N/A
0 Ratings
8.0

ReadyAPI

80%
2 Ratings

Support Rating

8.5

Azure DevOps Server (formerly TMS)

85%
22 Ratings
8.0

ReadyAPI

80%
11 Ratings

Implementation Rating

8.0

Azure DevOps Server (formerly TMS)

80%
2 Ratings
7.0

ReadyAPI

70%
3 Ratings

Ease of integration

9.0

Azure DevOps Server (formerly TMS)

90%
2 Ratings

ReadyAPI

N/A
0 Ratings

Product Scalability

Azure DevOps Server (formerly TMS)

N/A
0 Ratings
7.0

ReadyAPI

70%
1 Rating

Likelihood to Recommend

Microsoft

Azure DevOps is good to use if you are all-in on the Microsoft Azure stack. It's fully integrated across Azure so it is a point-and-click for most of what you will need to achieve. If you are new to Azure make sure you get some outside experience to help you otherwise it is very easy to overcomplicate things and go down the wrong track, or for you to manually create things that come out of the box.
Read full review

SmartBear

As stated, we do a LOT of API testing, the swaggerhub import makes it easy to add APIs. This is very well-suited, as well as easy management of the steps/cases/suites inside of ReadyAPI. The one thing I do wish ReadyAPI was better suited for is changes to data, we have a lot of test cases in ReadyAPI and if we make a change to how the backend data is structured, one-by-one adjustments need to be made to the steps. Less appropriate, UI testing.
Read full review

Pros

Microsoft

  • Reporting Integration- Azure boards provides Kanban and other dashboard, their templates for easy management of project.
  • Project Pipeline- easy integration and development of CI/CD pipelines, helped in testing, releasing project artifacts.
  • Version Control- Integration with Git and code IDE made it easy to share, review our code, fix bugs and do testing.
Read full review

SmartBear

  • Different Load testing techniques with variable loads and settings allowing to load test your service in different ways.
  • Assertions are much easier to use by inspecting responses, especially xpath assertions.
  • Exporting test scripts and linking them on jenkins.
  • Modern and user friendly GUI compared to free version.
  • Running test scripts in parallel.
Read full review

Cons

Microsoft

  • Can add more build templates for specific technology requirements
  • Can have more features in dashboards which can help dev teams stream line their tasks and priorities
  • Can have raise alarm feature in case of any sort of failure in devops pipeline execution
Read full review

SmartBear

  • Sometimes Ready API Test performs poorly or freezes when REST response contains a lot of data
  • Performance Tests documentation could be updated to describe different assertion points with examples of usage.
  • Ability to disable certain test steps only in the Performance Tests section, while leaving them on for Functional Tests could be added.
Read full review

Pricing Details

Azure DevOps Server (formerly TMS)

Starting Price

Editions & Modules

Azure DevOps Server (formerly TMS) editions and modules pricing
EditionModules

Footnotes

    Offerings

    Free Trial
    Free/Freemium Version
    Premium Consulting/Integration Services

    Entry-level set up fee?

    No setup fee

    Additional Details

    ReadyAPI

    Starting Price

    Editions & Modules

    ReadyAPI editions and modules pricing
    EditionModules

    Footnotes

      Offerings

      Free Trial
      Free/Freemium Version
      Premium Consulting/Integration Services

      Entry-level set up fee?

      No setup fee

      Additional Details

      Likelihood to Renew

      Microsoft

      Because we are a Microsoft Gold Partner we utilize most of their software and we have so much invested in Team Foundation Server now it would take a catastrophic amount of time and resources to switch to a different product.
      Read full review

      SmartBear

      The only reason this isn't a '10' is because of the cost. This product is definitely meant for organizations who are serious about making sure they invest in the full ecosystem of API design, development, maintenance. But there is a significant cost associated with this investment. and because of this cost (and the non-tangible output for executives), it is a difficult line-item to justify in this post-pandemic environment.
      Read full review

      Usability

      Microsoft

      Azure DevOps Server or TFS is a complete suite in itself. From Developer's machine where the code is developed to the production environment where the code is meant to run it take care of complete flow within itself. It acts as a code repository you can check-in check-out codes using GIT interface. It also acts as a Build and Automation Test tool which can help you to judge sanctity of your code. It further acts as a release manager to deploy your application to the production environment. And all these steps can also be performed without any manual intervention with the option to have approval processes. Hence its a perfect blend of all set of tools and capabilities required to bring code to production.
      Read full review

      SmartBear

      SoapUI allows us to combine multiple tests and adhere
      to the sequence that they need to run in order to complete successfully.
      It has an excellent GUI design and the reporting mechanism is also very
      good. It does consume a lot of memory though during concurrent testing
      Read full review

      Reliability and Availability

      Microsoft

      No answers on this topic

      SmartBear

      Soap UI has managed to continuously build on it's solid foundation and keep improving by each release. It is by far the most dependable and accurate testing tool out there of its kind. Available via connecting to VM's created as SoapUI test machines give access to it anytime, anywhere practically.
      Read full review

      Performance

      Microsoft

      No answers on this topic

      SmartBear

      It has an excellent GUI design and the reporting mechanism is also very
      good. It does consume a lot of memory though during concurrent testing.
      Read full review

      Support Rating

      Microsoft

      I have not had to use the support for Azure DevOps Server. There have never been any issues where I was not able to figure it out or quickly resolve. Our Scrum Master has used support before though, and the service has always been prompt and clear with a customer-focus
      Read full review

      SmartBear

      To be honest, we didnt had much issues with the support, as there is already plenty of online communities available for help. But if ever there were some minor issues with the membership or the certificates, the tech support was always quick and efficient enough to resolve the issue ASAP
      Read full review

      Implementation Rating

      Microsoft

      Do research beforehand and, if possible, do a trial run before implementing into production environment.
      Read full review

      SmartBear

      no very easy but lacks documentation
      Read full review

      Alternatives Considered

      Microsoft

      In my opinion, DevOps covers the development process end to end way better than Jira or GitHub. Both competitors are nice in their specific fields but DevOps provides a more comprehensive package in my opinion. It is still crazy to see that the whole suite can be used for free. The productivity increase we realized with DevOps is worth real money!
      Read full review

      SmartBear

      ReadyAPI provides intuitive GUI capabilities compared to their own open source product. When compared to Postman, ReadyAPI also supports SOAP based services, which is a saver especially when integrating with legacy or other third party systems.
      Read full review

      Scalability

      Microsoft

      No answers on this topic

      SmartBear

      It has an excellent GUI design and the reporting mechanism is also very
      good. It does consume a lot of memory though during concurrent testing. However, I have read that added monitoring tools have been added, which if so the 7 could possibly go to a 8 or 9.
      Read full review

      Return on Investment

      Microsoft

      • It has streamlined the pipeline and project management for our agile effort.
      • It has helped our agile team get organized since that is a new methodology being leveraged within the Enterprise.
      • The calendar has improved visibility into different OOOs across the project team since we all come from different departments across the larger organization.
      Read full review

      SmartBear

      • We were able to achieve our needs in API test automation with the tool
      • In terms ROI, I would say that we have gotten what is basically needed
      • Though the initial setup was tough, after that we were able to continue it with user friendly UI and reporting.
      • Overall, it had [a] neutral impact in terms [of] ROI.
      Read full review

      Add comparison