Item: Apache JMeter
Rating: 7
Author: Raman Ramanchuk

Use Cases and Deployment Scope

Hi, I use Jmeter in current company (Strevus) and used it in Oxagile to build load profile for sowtware applications and products that I was testing. Also I measured web applications response times when performing load and stress testing. Also I measure DB response times (MySQL) when loading DB with queries (selects, udpates etc.). We used Jmeter mostly in our Engineering (DEV and QA) departments

Pros and Cons

Easy way to create and configure http(s) requests being emulated by Jmeter. A lot of configurations of http requests
Easy way to configure and run DB load tests using connector-j library for MySQL
Jmeter Scripts debugging mode and real time debugging when scrips are being executed help to catch a lot of application issues and helps to fix issues in load scripts

Reporting tools and reports configuration. I would like to have configurable reports by metrics that Jmeter has
Integration with servers statistics gathering. AFAIK Jmeter can gather load logs only from TomCat server but I would like to have more advanced integration and gathering logs from other wide spread web servers (e.g. IIS etc)
Complex language (Java) to write your own assertions or advanced scripts that emulate non-predefined actions

Return on Investment

Jmeter helped to improve overall apps performance and revealed blocking performance issues before releasing product to wide audience
Jmeter helped to optimize server resources (configure servers), so less hardware was required to maintain the same load
Jmeter helped to get new testing projects (outsourcing) because of previous experience writing scripts with this tool

Alternatives Considered

I had experience mostly with Jmeter. I made quick review of PushToTest tool, but it didn't win competition with Jmeter as a performance testing tool

Likelihood to Renew

I renew Jmeter knowledge at each of last 3 working places (Sowtware companies, startups), as each web project\product requires performance testing and I use Jmeter to evaluate performance of web appliations\products. One more factor - Jmeter is free and easily configurable, so it takes me 1 day to configure testing env.

Likelihood to Recommend

Jmeter is free tool, so it's preferable in case if there is no budget for performance testing. Jmeter is most suitable to build simple load profile and quickly analyze web app or DB performance. It's cross-platform and can be configured to have cluster and create required load. Jmeter is less appropriate if you need to perform custom actions that are not listed among Jmeter handlers (it's required to write Java code and requires development skills). It would be great if Jmeter can support multiple scripting languages like TestComplete (automation tool)

Products Replaced

Key Differentiators

Price
Product Features
Product Reputation
Prior Experience with the Product

Price

Evaluation Lessons Learned

I would review new tools available on the market
I would add critical criteria and features that project require from performance tool
I would analyze my previous experience and reuse previous scripts if I'm short of time to create new scripts

Jmeter as performance testing tool for web applications/products

Software Version

Modules Used

Overall Satisfaction with JMeter