Overall Satisfaction with NeoLoad
NeoLoad is being used by the Performance Engineering Team to execute performance testing on new releases of applications before they are moved into production. The purpose of the testing is to reduce the risk of performance issues under load in the production environment. My Performance Engineering Team tests all applications in the division.
- Building performance testing scripts is easy. NeoLoad builds the scripts for you by recording your session as you navigate through the application under test.
- NeoLoad works with mobile applications using the same recording technique.
- NeoLoad has modules that permit gathering OS metrics from the target host and correlating the data with the load testing results. The modules are agentless, meaning that there no need to deploy NeoLoad code modules on the host to gather these metrics.
- There is no support for VT3270 terminal emulation to support mainframe-hosted applications.
- The ramp-up and ramp-down time data are not automatically excluded from the average response time calculations. Instead, you must exclude these times manually each time you run a test.
- There is not a summary graphic that shows the distribution of all of the response times on a single graph. We currently export the data as a coma separated variable (CSV) file and import into Excel to generate this summary graph.
- Reduced risk of performance issues in production.
- Increased employee efficiency.
- Increased customer satisfaction with online transactions.
- HP LoadRunner
NeolLoad is a less expensive alternative to HP LoadRunner that is also easier to learn and use. NeoLoad supports the same web-based protocols as HP LoadRunner, and, in addition, provides better support for testing mobile applications. In my organization, NeoLoad's main deficiency is lack of support for mainframe terminal protocols. Although NeoLoad does not include support for "deep-dive" diagnostics as is available for HP LoadRunner, it does provide interfaces for two leading diagnostic products, AppDynamics and DynaTrace. Neotys, the company that produces NeoLoad, also has an additional product, NeoSense, that executes NeoLoad scripts as an Application Performance Management (APM) tool. This product is a separate purchase, and I have not used it.
The most important question to ask in selecting a performance testing tool is - does it support the protocols that the applications to be tested use? NeoLoad supports all web-based protocols. It also supports Microsoft Silverlight and .net, and mobile applications. Unfortunately, NeoLoad does not support mainframe terminal protocols such as VT3270. Another key aspect is the ability to correlate host OS metrics with the results of the performance testing. NeoLoad supports this requirement through agentless monitoring. Finally, another key factor is ease of use. NeoLoad is easier to learn than a number of competitor products such as HP LoadRunner.