JMeter is a great all around load test tool with some caveats.
Updated September 30, 2015
JMeter is a great all around load test tool with some caveats.
Score 8 out of 10
Vetted Review
Verified User
Software Version
2.9
Modules Used
- HTTP Cookie Manager
- HTTP Request Defaults
- HTTP Header Manager
- CSV Data Set Config
- Counter
- User Defined Variables
- Simple Config Element
- Random Variable
- Thread Group
- BSF PostProcessor
- BSF PreProcessor
- Regular Expression Extractor
- XPath Extractor
- BSF Assertion
- Response Assertion
- HTTP Request Sampler
- Debug Sampler
- Aggregate Graph
- View Results Tree
- JMeter Plugins
- Statistical Aggregate Report
Overall Satisfaction with JMeter
We currently use JMeter at one of our divisions in the organization to assess site/system performance, fixing any issues that arise and looking for regressions in performance. We do this on a yearly basis before our holiday peak comes. Although strategically, our test framework/tools team is standardizing the entire company to use another open source tool going forward called Gatling. But we have yet to complete full migration to it, so in the meantime we still use JMeter.
JMeter was selected initially when we started formalizing load testing at my division and was evaluated along with another tool called WebLoad. We picked JMeter for its ease of use, usefulness, and being open source which saved us licensing costs. We primarily use JMeter to generate HTTP requests against our websites, either simulating browser activity or making back end REST API calls.
JMeter was selected initially when we started formalizing load testing at my division and was evaluated along with another tool called WebLoad. We picked JMeter for its ease of use, usefulness, and being open source which saved us licensing costs. We primarily use JMeter to generate HTTP requests against our websites, either simulating browser activity or making back end REST API calls.
- Easy of use - in generate load like HTTP requests, and processing/analyzing the responses. No coding is necessary at the basic level, just need to understand load testing and the infrastructure being tested.
- Automatic management of things like cookies to help with session state support - so you don't specifically have to worry about it or handle it
- Lots of testing/configuration options to suit your needs in making the right load generation (sampling requests), and analyzing the results, including any pre and post processing of the results first. Things like the Beanshell/BSF pre/post processors, response assertion, regular expression extractor, XPath extractor, CSV data set config
- There is a JMeter cloud service called BlazeMeter that I think would be useful for those that need to scale up high load without provisioning their own systems. I've not personally tried it though, but I recently attended a meetup presentation that highlighted nice useful features that BlazeMeter provides. One should evaluate the service if they are considering JMeter and need to expand beyond existing hardware resources.
- Programmatic customization - for more lower level detailed programming fine tune control of load test actions and processing of results, one has to write a JMeter plugin and load that plugin with JMeter to use it. Or encapsulate the logic into something like a Java class and access that through Beanshell, etc. in the pre/post processor or sampler, etc. It provides somewhat less control of everything than some other tools like Gatling. Or at least it requires more effort and a greater level of understanding of the workings of JMeter internals than a normal user has.
- Test plan reuse - by design JMeter doesn't work very well when you have frequent components that you reuse across tests or having different modular tests chained together to form a bigger load tests. It gets harder to do so as you scale up with more tests or more levels of testing. Though someone once pointed out to me you can deal with this to some extent using test fragments and encapsulate the reusable component configurations in a test fragment that you can import to other tests. The fragments would be like the test templates. I've personally not gotten around to trying that out though. The other approach is to abstract the common usage functionality/code into Java class (for use with Beanshell, etc.) or JMeter plugin and reuse that. And the lazy way most beginners would deal with reuse is copy & paste from one area to another or one file to another.
- JMeter might not generate load as well as other test tools not being able to scale up the load as high. I personally have not observed this yet comparing different load tools. But our test frameworks/tools division has found that Gatling scales higher and better than JMeter for high concurrency load against the system. So just wanted to point that out.
- Confirmed system performance to give us peace of mind that our systems are operating well for peak load
- Helped us flush out and find performance problems before holiday peak load arrives
- Gatling,WebLOAD
Besides JMeter, we've looked at WebLOAD and Gatling. Our organization used to use WebLOAD and we switched to JMeter at our division for its ease of use and configurability compared to WebLOAD. However, we will eventually switch to Gatling because Gatling provides greater programmatic control via Scala scripting language and can generate a higher concurrent load against the system than JMeter (as observed by our test frameworks/tools division). But personally, I do find it easier to get started and up and running with load testing using JMeter than with tools like Gatling and WebLOAD.