- Recognizing Delphi VCL controls, except TVirtualTreeview, but TC Folks once showed me how to automate this as well and it's ok now.
- Clean testcase structure etc. and helpful IDE
- Supports Delphi script for writing clean test Code and doesn't force you into recording everything.
- It rarely crashes and given that the Task at Hand really Needs intimate Knowledge of the inner workings of the platforms supported, this is surely no easy task!
- No support for testing Delphi Firemonkey apps on mobile platforms yet and no clear indication of whether this will be delivered in the future and if so, when.
- The record and play back works great for our less technical team members. Basic UI validations and some of the happy path functionality can easily be automated.
- The structure and set up of Project Suites. Easy to organize tests and group into functional tests that can be reused in Test cases.
- Multiple Types of tests projects can be created. Mobile, Web Page and Web Services. When setting up a new project the wizard guides the user through and allows for the selection of many different options as well as the coding language that is preferred.
- With some of the applications our company develops it is hard to re-use tests on multiple browsers. There is usually some items that need to be "tweaked" in order for it to run properly on multiple browsers.
- When adding Environments to the list in Environment Manager the values are not always retained. I try to add a Windows 7 - 64bit with IE 11 and it defaults the values back to Windows 10 or Chrome 50 when I select the alternate values.
- Our company uses JIRA for defect tracking and TestComplete does not have the option in Issue Tracking to connect to JIRA.
"TestComplete, medically approved* to relieve tedium. (*i'm not a Doctor so this may not be accurate)"
Using TestComplete has really made a difference on the efficiency of the product test cycle. Previously a manual process involving hundreds of repetitive tasks, now, i simply load the test suite within TestComplete, press 'go' and it does the boring tests for me while i can do the more interesting exploratory testing manually.
TestComplete has even helped us isolate bugs in our software that we didn't even know existed, due to the nature of how it interacts with our software.
- Intuitive - whether writing scripts, or keyword tests, the TestComplete object discovery and mapping engine really helps keep things accessible to the 'driver', for example, i use the object browser to find elements and inspect their members, once mapped, referencing them in scripts and keyword tests is simple.
- Powerful - day to day, the problems I've had to solve to create tests were aided by TestComplete not hindered. Even if a scripting module is not present by default, it can easily be added/referenced
- Support network - I've had reason to ask the Smartbear support team for help a few times, and while their answers don't come immediately, they have been absolutely bang on the money with the answers given. In addition to this, the forums answer almost any 'mainstream' question you may have, they are usually my start point if I'm stuck.
- Repeat ability - the ability to repeat processes as many times as you like is a godsend that keeps tedium and human error to one side.
- The scripting languages offered while solid, are less than convenient for me. I would love to be able to script tests in C# as i have a room full of C# experts here to assist if required
- Pressing F5 doesn't run the 'current' test, but the ENTIRE test suite, as my brain is tuned to Microsoft's F5 to 'go' this is a constant user error of mine
I have also scripted an entire run through our Ribbon menus in most of our products, doing the most basic execution tests - i.e. does the command work in isolation. This is another tedious and long test that is now quick and easy using TestComplete
- The organization of test suites and test projects is very intuitive. This made it easy to learn and fairly easy to manage. We had a conceptual framework for tests that TestComplete seems to support very well.
- The blog web site for technical support is very responsive and the people supporting it know the product quite well. This has been essential in helping us to apply the product across a wide variety of testing needs.
- TestComplete allows one to create very robust automated test procedures. When it comes to testing on a very large scale, the test procedures must themselves be bullet proof. There are quite a few tricks to learn to make this happen, but TestComplete allows us to do this.
- The reporting features are poor to non-existent. We need to develop our own presentation formats for results and do not want to buy an add-on product (such as QA Complete) to do so. We have had to get creative and embed report meta data generating subroutines into our test procedures so that we can capture the data that we need. This is time consuming and inefficient.
- There are a couple of files that are particularly vulnerable to corruption if you are not very careful with how you manage your source control. The name mapping file and individual project files can be corrupted easily when more than one person is making changes to those files. Merging capabilities with something like the Git version control tool can be tricky. Remember to back things up frequently and carefully!
- The distributed execution module is great in the sense that it helps us to cut our Test suite execution time by more than half.
- The TestExecute module is very useful. It's not costly and is light weight. The Command Line Interface makes it flexible for us to write schedule jobs.
- The Distributed module needs to provide the ability to connect to any host instead of a Physical 1 to 1 mapping. This would allow the automation architect to provide a pool of servers to be used for test execution.
- The tool should have Java support since a huge majority of applications use Java. The IDE's support for Java will allow Java developers to leverage the tool more.
- A very low entry point, with a very high skill ceiling - It doesn't take a lot to get started in Testcomplete and I feel like I'm always learning a new use or function that I hadn't thought of or knew before.
- Feedback - The live debugging in script creation is very useful and it's very rare that I have a test fail and Testcomplete does not give me a descriptive enough answer to correct the issue.
- A large tool set - There are a large number of features that can be used to aid in the automation process. Testcomplete has different wizards and object tools that allow me to tailor the creation process to how I want it to be.
- Occasional crashes - sometimes a recorded test can cause the application to crash if it is too large for the system it's running on.
- Missing sort/search functionality in some places - There are some places where you can only search one field when there are multiple being displayed. It makes it difficult to find script files when I'm adding them as a test item.
- High skill ceiling - while I listed this as a positive, it can sometimes be daunting moving into an area that has a ton of options or variables that you can change with very little description next to the options.
- Ease of use for non-programmers, ability to record and playback your test cases.
- Support for mutliple programming languges, so it can be used by users with programming backgrounds, which in turn provides more solid test coverage and troubleshooting.
- Very convenient feature for executing recorded or written scripts over multiple machines via TestExecute feature. You can schedule and run your tests simultaneously over a number test machines.
- Data-driven testing is made a lot easier with TestComplete's ability to pull the data from stored Excel files or even from DB queries that can be run in real-time during test executions by accessing.
- Ability to analyze objects your test cases are working with, with their full sets of properties, and to decide which property of an object to verify.
- Reusability of recorded test cases; you can call a recorded test case within another test case.
- Cross-browser testing; you can run your web scripts over multiple browsers and browser versions.
- I would like to have the ability to export test results in bulk, from the Project Workspace section; currently, you can only export one test case execution at a time in HTML format.
- From time to time, I've run into an issue with object recognition, where TestComplete was not able to find some of our custom coded controls due to its multiple layer structure (Infragistics combined with WinForms, etc.). Our application uses a lot of search result grids, which are Infragistics control grids, and by using the keyword test script approach, I am not able to easily read the values and properties of the individual cells of the grid; object spy treats the entire grid as one element, one control.
- Great object recognition. We can get an access to the most of properties and methods of GUI controls.
- Support of Python programming language expands native TC functionality with third party libraries.
- One testing platform for web, mobile and desktop applications.
- Works relatively slowly during test runtime. Especially with iOS applications.
- Poor options for log export. Only .mht is supported.
- Unable to use variables within Objects in Stores.
Also it can be useful for QA-personnel without programming skills to quickly automate not very complex scenarios.
- Automating different types of applications (web, desktop and mobile apps)
- Supports multiple scripting languages for coding
- Features like name mapping, object spy, object browser which are helpful in identifying elements easily and to identify the process running in Windows OS
- Online support covering the features of TestComplete/assistance from a vast number of users/technical persons
- Debugging the code by enabling breakpoint
- Support for multiple browsers
- Record and playback feature
- Still more better online assistance/guide (w.r.t TestComplete tool and to the scripting language along with code) from the technical team. Already online guide is available and its vast, but it can be still improved/updated to include all features of TestComplete and syntax/coding standards of scripting language as well
- Still more detailed explanation about the errors (compile/Run time) which is observed could be provided so that debugging can be easier and tool will be user-friendly
- Name Mapping feature/object Spy tool could be improved
Also, if more number of test cases / scenarios are there, we can execute some test cases in server machine and remaining test cases in other (client) machine using TestExecute.
Then TestComplete can be used for automating Windows desktop application through VB scripts and also we can perform mobile apps testing easily through emulators even though if the actual mobile device is not available.
- TestComplete is good at making changes in the QUI visible right at once
- TestComplete gives more decipline in the development department. The developers need to be aware that changes in object naming have an influence on the automated test
- We have a lot of manual testing, but we can save time by automating the tests that don't involve direct hardware interaction
- Namemapping. From time to time TestComplete fails due to object names. Making changes in namemapping is hard to do as well as risky.
- The help files. All examples are scripts. I would like to see more examples directed for the people that do not script testcases, but record them
- We are using TestComplete for windows desktop applications and, compared to other tools that we tried, it was most intuitive and easy to use
- Other than that, since our application is large and consists of lots of different UI controls, TestComplete worked well with most of them, since some other tools didn't work at all
- TestComplete has integration with Git so we can easily save our tests to a remote Git repository
- Sometimes is not easy to find the property you would like to use for checkpoint
- TestComplete for windows could run faster in some scenarios
- Git integration could be improved (to not only use Tortoise Git)
- TestComplete is easy to use. Code is easy to maintain and update (if developer respects some elementary coding rules, of course).
- Production of mht log for tests is easy to do, and readability is good (expandable folder, etc.)
- Execution of a scenario over several browsers.
- Object spy is efficient.
- TestComplete Editor for script needs some functionality like highlight mapping object from script, possibility to begin a run from a line in script, etc.
- Some basics functionalities that have existed for many years in editors like notepad++, eclipse, etc. don't exist in TestComplete.
- Debug is hard to use.
- Sometimes automatic mapping of object during recording is bad (for example, for a web application, idStr is not taken as criteria because there is only one object of this type in the page). I prefer doing mapping to control it before recording.
- Edition of a same shared project by 2 users is too hard. I gave up because we lost some TestComplete code too many times.
- TestComplete can be blocked by a child process of Application under test, and it's hard to find it, and mainly impossible to stop run without killing TestComplete.
- Confirmation of object disparition is hard to code. Most of the time, it slows the scenario.
- TestComplete is the most effective automation tool for interacting with Delphi based code.
- TestComplete allows the user to choose from various scripting languages, and provides support for each.
- TestComplete is continually being updated and improved.
- TestComplete provides good support, including an extensive forum, plus a short response time for additional issues.
- If the app under test is not responding, TestComplete cannot always be paused, but requires being shut down.
- It is not uncommon to have to re-vamp automation scripts that once ran flawlessly, when there have been no changes to the PC, etc.
- The licensing can be onerous particularly if there are any changes to the hosting PC.
- TestComplete allows access to the entire view model of WPF programs, which allows for complex tests to be scripted.
- TestComplete has a feature called Object Spy, which makes scripting of new tests fairly quick, as users can quickly see the names an attributes of applications elements simply by clicking on parts of the application user interface.
- TestComplete's text editor is extremely slow for debugging (especially for loading local variables, which can take minutes to load for every step in the debugging process).
- During our use of TestComplete, we found that it would regularly hang during test runs, requiring a manual restart of the test run.
- TestComplete seemed to have difficulty working with complex WPF programs as we found it necessary to script frequent breaks and refreshes into our automated tests in order to be sure that TestComplete had an accurate version of the view model.
- Basic scripted testing of login pages and basic web operation. Keyword / macro capability easy to program.
- More involved functional tests with time period testing between events. Good for performance evaluation.
- The key issues are that TestComplete often has problems running previously created scripts as a browser version update - identifying page objects is problematic.
- It easily integrates with other tools like Jira, Github, Jenkins and even other test automation frameworks like Selenium. With these features, it makes a perfect case for continuous testing and CI/CD. Also perfect for DevOps style of development.
- It supports multiple platforms like desktop applications, web applications, and mobile applications. This combination makes a great one as in today's world an app should be functioning in all environments.
- It has cloud on demand test labs which is excellent for running your test suite in multiple environments without any hassle.
- Got to say that object recognition is way superior to other tools which is very key for test automation. Also supports codeless automation which is cool.
- Even though object recognition is their strength, it can still be better. Some times in a dynamic web application, the object recognition fails. Of course there are work arounds but you shouldn't really be using them.
- Switching to debug mode when an exception pops in is missing. Would be a great add on.
- There are many open source free tools in the market and license cost is a factor with this tool.
TestComplete provides an option to perform actions with multiple AUT technologies under one project/project suite.
TestComplete has a wide range of options to overcome your issue during automation.
- Script Extensions - Scripts can be developed using VBScript and JScript where it can be utilized by other script languages
- Project(Suite) Variables - These variables can be accessed any where in the project/project-suite and it has different types of data type to perform the actions better
- Custom Object Mapping - User can map the AUT custom objects into accessible controls where other tool doesn't have much on it
- Extended Find - This can be used when user based roles and screens for identifying same object in different hierarchy
- Code Editor - this needs to improved a lot in-order to be great product
- Performance on Debug - this needs to be optimized when we use large project suite
- Recovery Scenario
- Handling dynamic objects
- Handling complex object hierarchy
- Language selection for script development
- Script Extension provides you with an ultimate option to create your common and utility functions since it will run as non-debug code, it always much faster than unit codes.
- Handling test execution on multiple machines using NetworkSuite. It has a Sync points where it used to sync between two system which will help us may ways
Mobile testing - Less appropriate.
- Reduced Manual testing effort
- Faster test cycles
- Continuous integration Support
- Kalios enhances and simplifies test development
- Test scenarios maintenance is simplified by Kalios
- Good performance while executing tests
- TestComplete needs nearly the last version of testcomplete (Our PLM is working on a very old firefox version...)
- Test complete is the most stable tool I have ever had to do this job. I use it to test Internal Fat, mainstream but also Saas and other Internet applications.
- Com Object Testing is more stable than image recognition (but good image recognition is paramount for Citrix, and so on)
- I need to test Citrix (deployed apps and desktop) and for now TestComplete cannot do the job because image recognition is not good enough
- A way to store the encrypted password of many users would be also appreciated
- Citrix, Host / Telnet, Notes testing is a real pain in the ass, when its not directly impossible
We have utilized TestComplete and TestExecute for user uploads, profile setup, test case creation, regression testing after a build, replacing repetitive manual tasks, environment validation, and many other areas. In many ways, we use it to address pain points of our business users for tasks that either have an unusually large manual component or are taking up too much time to execute. Utilizing TestExecute with a backend SQL feed to manage test cases has made the process of testing many applications relatively pain-free!
- TestComplete has a good variety of low-level functions that give us the flexibility to create our own functions that really meet our needs. We have built up a significant 'common' script library that includes functions for interacting with fields, checking data, and even a re-written object search that has made working with new environments easy!
- With floating user licenses, we are able to be really flexible in our development environments. We can easily switch servers and run scripts from different locations to test network timings and environment setup with relative ease.
- Using a backend of SQL (rather than Excel documents) to handle test cases, we are easily able to scale our tests. We can have any number of TestExecute licenses running the same project and the SQL will provide each script with the data it needs to run the specified job.
- Having never used anything like TestComplete before joining my company, it wasn't exactly the easiest application to pick up. The code examples online (particularly the SQL ones) were a little difficult to get the hang of. I was greatly helped by having a guy on my team who had worked with TestComplete for a few years, but the onboarding was a little rough.
- We've run into various errors over the years involving memory leaks (which required a hotfix patch) and even scripts written in TestComplete not running in TestExecute. While these have always been addressed by patches (or new major versions in the case of the TestExecute issues), it has occasionally required some clever work-around in order to get it to do what we need.
- We have encountered many issues running TestComplete in an RDP session to a virtual machine. The RDP window needs to be open at all times and cannot be minimized. This may be a technological limitation, but it would be really convenient if we could run 'headless' scripts as scheduled tasks.
Most of the issues we have run into (changing object references as applications develop and unreliable websites) have been solved with custom object identification and error handling functions.
- User interface testing
- Keeping up with browser developments
- Identifying behavior changes in the interface
- The licensing system is unwieldy, particularly when it comes to removing a license from a dead machine.
- The stand-alone test runner, TestExecute, really needs a function for setting local variables and parameters.
- Adding exception handling for scripts would be very useful.
- Adding a 'pause for human interaction' script element (other than inserting a breakpoint) would be highly useful for certain tests.
- Easy to use even by non-technical people.
- UI automation made easy.
- Object spy is really good for finding object's properties and methods.
- Region checkpoints don't have anymore the option to directly crop the area of interest.
- Only one global object repository.
- Sometimes (very rarely) crashes.
- No quantifiable impact yet, since we have not run a system test until now.
- Finding new bugs.
TestComplete Scorecard Summary
- Instant Access to Real Environments in the Cloud: Get real-time access to the latest devices, resolutions, browsers and operating systems in our cloud lab. Run tests in parallel across more than 1500+ remote environments to improve test speed and coverage.
- Robust Object Recognition for UI Testing: Automate anything you have seen on the user interface. With TestComplete's robust object identification engine, you can choose from over 50K object properties across more than 500 control types, including all major frameworks and third-party controls.
- Built for Continuous Testing: Easily integrate TestComplete with test automation framework tools, CI systems such as GIT or Mercurial, or Test Management platforms like Jira or QAComplete.
|TestComplete Pro||$3,500 / $7000||Node-locked / Floating User License|
|TestComplete Platform||$1,000 / $2,500||Node-locked / Floating User License|
|TestComplete Web||$999 / $1,998||Node-locked / Floating User License|
|TestComplete Mobile||$1,299 / $2,598||Node-locked / Floating User License|
|TestComplete Desktop||$999 / $1,998||Node-locked / Floating User License|
You can buy the TestComplete Pro Bundle or individual TestComplete Modules based on your needs. The TestComplete Pro Bundle comes with the TestComplete Platform, all three modules, (Desktop, Web, and Mobile) as well as the TestExecute add-on. TestComplete also has additional add-ons: -TestExecute -HP Quality Center Connector - Environment Manager
TestComplete Support Options
|Free Version||Paid Version|
|Video Tutorials / Webinar|
TestComplete Technical Details