Easy to use, stable tool for Performance Testing
Overall Satisfaction with Tricentis NeoLoad
We use Tricentis NeoLoad for our performance testing and engineering requirements. As a Product based company, we develop and maintain multiple products that are being used by millions of users worldwide. Therefore it is in our culture to give extreme importance to Application Performance. That being said, I work in a centralized team that is responsible for measuring, identifying, and tuning performance issues across all the products that go out of the Company, and also addressing issues faced by the customers. We need a solid tool for recording the real-world use cases, scripting them, and designing performance scenarios that are as close to how it happens in production and Tricentis NeoLoad addresses our needs on a daily basis. Though Tricentis NeoLoad can also be used for other purposes such as functional and security testing, we use it extensively only for various types of performance testing.
Pros
- Easy to learn
- No/Low code scripting
- Support for wide protocols
- Support for automated performance testing
Cons
- Integration with third-party tools. I cannot stream NL test data to a third-party DB and then plot graphs out of it.
- Keyboard shortcuts. It is tedious to use the mouse to click everywhere. I'd love to have some keyboard shortcuts
- Reports and Analytics. Lacks intelligence in identifying patterns of problems
- Time! Less coding work resulted in faster scripting
- Stability compared to our previous tool
- It does not demand any advanced skill, as there is almost no coding involved.
This is a problem that we are trying to address currently. We want to build a robust, automated performance testing framework that could be used by multiple people across our Organization, thereby reducing our effort in supporting simple tasks. The previous tool that we were using was very poor with automation, we had to go around the bush to automate simple tasks. Tricentis NeoLoad however, provides a strong CLI package with which we can easily create tests, schedule the run, and pull the reports. We wrapped the CLI with our in-house infrastructure building stage, together resulting in a completely automated performance testing framework that is helping us save hundreds of man-hours. This is truly testing at scale.
Although NeoLoad as a tool offers low code scripting and has an easily-understandable GUI, it lacks features when it comes to collaboration. It is tricky and at times cumbersome when two people would want to work on a single project in parallel. We did not find its inbuilt collaboration option optimal, thus leading us to import/export projects. That in turn created duplication of variables and bloating of the project size.
Apart from collaboration, maintanance of existing suits is much easy, there are extensive ways with which you can repair scripts, re-record existing scripts, or debug a script failure upon new build of the application. This significantly reduces the re-recording time.
Apart from collaboration, maintanance of existing suits is much easy, there are extensive ways with which you can repair scripts, re-record existing scripts, or debug a script failure upon new build of the application. This significantly reduces the re-recording time.
Do you think Tricentis NeoLoad delivers good value for the price?
Yes
Are you happy with Tricentis NeoLoad's feature set?
Yes
Did Tricentis NeoLoad live up to sales and marketing promises?
Yes
Did implementation of Tricentis NeoLoad go as expected?
Yes
Would you buy Tricentis NeoLoad again?
Yes
Using Tricentis NeoLoad
25 - Core performance engineering team, performance testers, developers, QAs, software architects etc;
6 - The core performance team at my Organization develops and maintains scripts and frameworks in and around NeoLoad.
- Low code performance testing requirement
- Integration with CI toolkits such as Git, Jenkins, AWS etc
- Integration with APM such as Dynatrace, AppDynamics, Datadog etc
- User friendly, understandable structure of scripts
- Used their CLI utility to automate entire testing effort, seamlessly integrated with AWS
- Used it for quick functional validations
- Developers found it intuitive to use
- Build a holistic performance automation framework
- Induce better visibility into performance metrics for SRE
Evaluating Tricentis NeoLoad and Competitors
- Cloud Solutions
- Integration with Other Systems
Most important priority for us was to automate the performance testing life cycle as much as possible. NeoLoad offers the automation capbility with their SaaS solution and a collaborative solution called NeoLoad Web, with which anybody across the Org can monitor, design the tests, view the results, generate the reports etc;
We should have researched more on their support capability and their SLO definitions on the support tickets raised.
Tricentis NeoLoad Implementation
- Implemented in-house
Yes - In the first phase, my team did POCs on migrating test scripts from existing tool to NeoLoad. Although it was not a 100% success, we figured out that it is easy to rather record new script in NL rather than finding a way to migrate the scripts that lead to a lot of errors.
Second phase involved in building the test scripts, we had to get used to NeoLoad and their various features, which helped us master the software.
Final phase was to explore their automation and integration capability to build a completely hands free performance test automation framework.
Second phase involved in building the test scripts, we had to get used to NeoLoad and their various features, which helped us master the software.
Final phase was to explore their automation and integration capability to build a completely hands free performance test automation framework.
Change management was minimal - NeoLoad creates random files in the file system each time you save a neoload project. This comes tedious when you check-in the scripts into your VCS like git.
Also, we tried to automate the repair process whenever a change is required to be made in an existing NeoLoad project, but the way the Project is stored in the file system is complex and it takes manual effort to import the project, alter it, and then export it back.
Also, we tried to automate the repair process whenever a change is required to be made in an existing NeoLoad project, but the way the Project is stored in the file system is complex and it takes manual effort to import the project, alter it, and then export it back.
- Lot of manual work
- Hard to use with VCS
- Poor support team
Tricentis NeoLoad Support
Pros | Cons |
---|---|
Kept well informed | Slow Resolution Less knowledgeable Problems left unsolved Difficult to get immediate help Need to explain problems multiple times Support doesn't seem to care Slow Initial Response |
While I am not 100% sure, I think the support was inclusive in the deal we made with Tricentis. So no, we did not buy any top-up support option.
Yes - Resolved to our satisfaction - mostly yes.
Resolved in a timely fashion - unfortunately no.
We feel that the initial response is very mechanical and just designed to buy more time by asking us the questions that have already been answered in the ticket description. We had to escalate a couple of times to get them back on track. It took over 6 hours for the initial response even when a critical bug was raised.
Resolved in a timely fashion - unfortunately no.
We feel that the initial response is very mechanical and just designed to buy more time by asking us the questions that have already been answered in the ticket description. We had to escalate a couple of times to get them back on track. It took over 6 hours for the initial response even when a critical bug was raised.
Sadly, I cannot think of any.
Using Tricentis NeoLoad
Pros | Cons |
---|---|
Like to use Relatively simple Easy to use Consistent Quick to learn Convenient Feel confident using Familiar | None |
- Script recording using browsers
- Automatic correlations using rules
- Comparing two test results
- Maintaining NL projects using version control systems like Github
- NeoLoad Web's test report is very basic and does not give full information about the test
- Tricentis' support team
Comments
Please log in to join the conversation