Top Rated
383 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener'>Customer Verified: Read more.</a>
Top Rated
136 Ratings
Top Rated
383 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener'>trScore algorithm: Learn more.</a>
Score 8.6 out of 100

TestComplete

<a href='https://www.trustradius.com/static/about-trustradius-scoring#question3' target='_blank' rel='nofollow noopener'>Customer Verified: Read more.</a>
Top Rated
136 Ratings
<a href='https://www.trustradius.com/static/about-trustradius-scoring' target='_blank' rel='nofollow noopener'>trScore algorithm: Learn more.</a>
Score 8.6 out of 100

Attribute Ratings

  • Azure DevOps Services is rated higher in 4 areas: Likelihood to Recommend, Likelihood to Renew, Support Rating, Implementation Rating
  • TestComplete is rated higher in 1 area: Usability

Likelihood to Recommend

8.9

Azure DevOps Services

89%
56 Ratings
8.6

TestComplete

86%
86 Ratings

Likelihood to Renew

10.0

Azure DevOps Services

100%
3 Ratings
6.5

TestComplete

65%
6 Ratings

Usability

7.0

Azure DevOps Services

70%
5 Ratings
7.6

TestComplete

76%
7 Ratings

Support Rating

8.2

Azure DevOps Services

82%
20 Ratings
6.3

TestComplete

63%
9 Ratings

Implementation Rating

10.0

Azure DevOps Services

100%
2 Ratings
6.7

TestComplete

67%
5 Ratings

Likelihood to Recommend

Microsoft

[Azure DevOps Services (formerly VSTS)] is well suited for agile teams. It is a digital kanban for teams to access anywhere at any time. Especially during covid when most companies are still remote, it allows teams to fully perform and still be functional in operating within an agile methodology. It also gives visibility to teams capacity.
Read full review

SmartBear

To really make the most of TestComplete, at least some scripting is necessary. TestComplete works really well with clearly identifiable objects but needs some tweaking for objects that vary in e.g. quantity. We have some elements that vary, but the vast majority of UI elements have unique identifies, and those iterative elements can also be mapped to iterations of a semi-unique element (so more of a mapping of the item type that the specific item). I doubt the usability would be near as good if more items were not clearly identifiable. In our case, most are, and we handle our tests in form of nested keyword tests that occasionally also implement scripting when needed. Tedious tests like Verification of the correct presence and status of UI elements in all possible scenarios, and iterative tests of e.g. input values and combinations of such, are made easy to set up, execute and evaluate.
Read full review

Pros

Microsoft

  • Flexible Requirements Hierarchy Management: AZDO makes it easy to track items such as features or epics as a flat list, or as a hierarchy in which you can track the parent-child relationship.
  • Fast Data Entry: AZDO was designed to facilitate quick data entry to capture work items quickly, while still enabling detailed capture of acceptance criteria and item properties.
  • Excel Integration: AZDO stands out for its integration with MS Excel, which enables quick updates for bulk items.
Read full review

SmartBear

  • TestComplete is great for working with our non-web applications.
  • TestComplete allows us to interface with our web application in a robust way.
  • Despite the age of our architecture, TestComplete handles the old stuff that's been around a while as well as the newer technology when we are able to implement it.
Read full review

Cons

Microsoft

  • Column sorting when in filtered states.
  • A way to show cross-team dependencies.
  • A customized "From" field for notifications. Sometimes when a mail comes from Azure DevOps the teams do not realize that I am sending it
  • A way to do online poker that doesn't require a plug-in.
Read full review

SmartBear

  • Would love to see TC be a plugin for Visual Studio. Instead of VBScript or JavaScript, to use C# or VB.NET
  • Easier to set up with iOS. Android is not bad but iOS seems slow and confusing to get up and running
  • Built-in PDF reading support, built in Excel API
Read full review

Pricing Details

Azure DevOps Services

Starting Price

$2 per GB (first 2GB free)

Editions & Modules

Azure DevOps Services editions and modules pricing
EditionModules
Basic Plan$61
Basic + Test Plan$522
Azure Artifacts$23
Azure Pipelines - Microsoft Hosted$404
Azure Pipelines - Self-Hosted$155

Footnotes

  1. per user per month (first 5 users free)
  2. per user per month
  3. per GB (first 2GB free)
  4. per parallel job (1,800 minutes free with 1 free parallel job)
  5. per extra parallel job (1 free parallel job with unlimited minutes)

Offerings

Free Trial
Free/Freemium Version
Premium Consulting/Integration Services

Entry-level set up fee?

No setup fee

Additional Details

TestComplete

Starting Price

$2,256 per license

Editions & Modules

TestComplete editions and modules pricing
EditionModules
Node-Locked Base2,2561
Node-Locked Pro3,9502
Float - Base5,0773
Float - Pro7,9014

Footnotes

  1. per license
  2. per license
  3. per license
  4. per license

Offerings

Free Trial
Free/Freemium Version
Premium Consulting/Integration Services

Entry-level set up fee?

No setup fee

Additional Details

Purchase TestComplete Base and pay for only the modules you need. TestComplete Pro includes all three modules: desktop, web, and mobile, at a bundled price point, as well as access to the parallel testing engine, TestExecute. TestComplete has additional add-ons, including TestExecute and the Intelligent Quality Add-On.

Likelihood to Renew

Microsoft

I don't think our organization will stray from using VSTS/TFS as we are now looking to upgrade to the 2012 version. Since our business is software development and we want to meet the requirements of CMMI to deliver consistent and high quality software, this SDLC management tool is here to stay. In addition, our company uses a lot of Microsoft products, such as Office 365, Asp.net, etc, and since VSTS/TFS has proved itself invaluable to our own processes and is within the Microsoft family of products, we will continue to use VSTS/TFS for a long, long time.
Read full review

SmartBear

We have bigger test automation pack using test complete at the same time we also think this is not good performing tool for large number of test automation scripts.
Read full review

Usability

Microsoft

Azure DevOps is a powerful, complex cloud application. As such there are a number of things it does great and something where there is room for improvement. One of those areas would be in usability. In my opinion it relies too much on search. There is no easy way to view all projects or to group them in a logical way. You need to search for everything.
Read full review

SmartBear

It is usable when you become accustomed to its quirks. Not using it for two months and then you need to re-learn the quirks for some features (but some quirks are so awful that they will never fade from your memory). So, when using it regularly, it is possible to be quite productive, if no big correction in name mapping is needed.
Read full review

Support Rating

Microsoft

When we've had issues, both Microsoft support and the user community have been very responsive. DevOps has an active developer community and frankly, you can find most of your questions already asked and answered there. Microsoft also does a better job than most software vendors I've worked with creating detailed and frequently updated documentation.
Read full review

SmartBear

Some bugs were quickly resolved, but most UX quirks of the tool are just marked "as designed". No follow up for enhancement request.
Read full review

Implementation Rating

Microsoft

Was not part of the process.
Read full review

SmartBear

If you develop a mobile application and your testing process goes in cloud, probably you will face a problem - how to implement a stable connection between your mobile devices and testing servers
Read full review

Alternatives Considered

Microsoft

Azure DevOps is much easier to implement and use on a daily basis than VersionOne. This is based on end-user feedback and my personal experience as well. ADO has more features and seems to get updated more frequently than VersionOne as well. VersionOne is less expensive overall than ADO, but our company prefers to use ADO because it has a lot more benefits.
Read full review

SmartBear

TestComplete stacks up against them in terms of GUI and seamless performance. It records each and every step and action been performed in the application and produces a detailed report in a well-structured manner. It can connect and access seamlessly among various databases directly to speed up the testing process.
Read full review

Return on Investment

Microsoft

  • It provides a great IDE to make things easier, clear, and compact. It has always been a positive ROI
  • It's worth the money. I don't think any other software could replace the VS. It has helped us a lot to making things ready on time
  • The only thing I dislike is that it takes a lot of memory space when in an idle state
Read full review

SmartBear

  • On one of the applications, I've reduced the testing time from a month to five days.
  • I don't always have time to create tests for new features, then I revert to manual testing.
  • The Zephyr reports have given management insight on what happens in the QA department.
Read full review

Add comparison