Apache JMeter vs. GitLab

The key questions I would pose are: how high of a concurrent load do you need to generate against the system? How technically skilled are your personnel that will perform the load testing? How much programmable/scripting/code based control do you desire from the tool? Do you prefer more of a GUI based tool or a programmatic code/scripting based tool? JMeter will be for those that want a GUI, moderate to low scripting/code based control/programmability, moderate to low load generation, and for users experienced in the novice to expert range.

It is well-suited for any project that needs VCS. It's an excellent choice for teams that might be remote or have to collaborate across teams. Plenty of features allow for async working. With its dashboards and reporting features, it is also suitable for nontechnical PMs or stakeholders. It allows for very bespoke customization and can most often do much more than you need it to.

Incentivized

• Easy to construct Webservices script - You just need to add a sampler with the endpoint, payload and header information. And boom you can run a load test on the service with varying load and capture the response time and throughput. In a commercial tool like Loadrunner or Silkperformer, this process would take at least couple of hours, while with Jmeter we can do this in 10 minutes or less.
• Modularizing the scripts - Jmeter has a very good way of organizing the scripts and the reusability of the existing script (transactions) is also as easy as copy pasting the script.
• Jmeter tests can be run from anywhere, so you can run it from the idle webserver itself if you have some firewall restrictions to test it from your local or other environment.
• Jmeter distributed tests can be spawn from the AWS and results collated to see the performance from different regions in the world.

• GitLab excels in managing code versions, allowing easy tracking of changes, branch management, and merging contributions.
• It helps maintain code stability and reliability, saving time and effort in the development or research workflow.
• Powerful code review features, enabling collaboration and feedback among team members.
• Robust project management features, including issue tracking, kanban boards, and milestones.

Incentivized

• Even though it has very easy installation, you can run into issues easily. I was ab;e to trigger a test at browser level but couldn't stop the tests. I had to kill it from task manager and then had to reinstall from begining.
• There are so many things in JMeter you can do especially with the sampler for selenium for UI tests. More documentation/webinars etc.. would help the community better.
• The jmx files should have an easy way to be integrated with ci/cd tools and cloud.

Incentivized

• CI variables management is sometimes hard to use, for example, with File type variables. The scope of each variable is also hard to guess.
• Access Token: there are too many types (Personal, Project, global..), and it is hard to identify the scope and where it comes from once created.
• Runners: auto-scaled runners are for the moment hard to put in place, and monitoring is not easy.

Incentivized

Price, Wiki and user sharing. Having access to the information provided by the developers and other open source providers is key for me. The ability to share information and get answers directly is very important to success in software testing. And the price of this product currently is amazing. Too many companies charge way too much money for products that are far behind in their value and pertinence

I really feel the platform has matured quite faster than others, and it is always at the top of its game compared to the different vendors like GitHub, Azure pipelines, CircleCI, Travis, Jenkins. Since it provides, agents, CI/CD, repository hosting, Secrets management, user management, and Single Sign on; among other features

Incentivized

The purpose related to performance and load testing through Apache JMeter works fine but the usability of the tool should be improved quite a lot. If someone starts with the Jmeter fresh without prior experience, they need to put more efforts in understanding the tool. The UI is not that great which is the main reason not to give high rating on usability.

Incentivized

I find it easy to use, I haven't had to do the integration work, so that's why it is a 9/10, cause I can't speak to how easy that part was or the initial set up, but day to day use is great!

Incentivized

I've never had experienced outages from GItlab itself, but regarding the code I have deployed to Gitlab, the history helps a lot to trace the cause of the issue or performing a rollback to go back to a working version

Incentivized

GItlab reponsiveness is amazing, has never left me IDLE. I've never had issues even with complex projects. I have not experienced any issues when integrating it with agents for example or SSO

Incentivized

I have been using JMeter for the last year. By using this tool, you can make sure the system will work under varied loads. It helps us to simulate real time scenarios by creating required virtual users and make sure the application will work under load. Perform load, stress, and stability testing using JMeter.

Incentivized

At this point, I do not have much experience with Gitlab support as I have never had to engage them. They have documentation that is helpful, not quite as extensive as other documentation, but helpful nonetheless. They also seem to be relatively responsive on social media platforms (twitter) and really thrived when GitHub was acquired by Microsoft

Incentivized

I have evaluated LOADUI (web free version) and it was a very unstable tool and I could not rely on those results completely as I was not sure how the tool was performing. It only generated the top 10 less transaction times and when Jmeter was in use, it was very effective in using plugins to accommodate more outputs.

Incentivized

GitHub is an inferior product from most points of view. We had to use it and the teams finds no positives about it. Everything is a downgrade from our previous GitLab solution. GitLab CI\CD is vastly superior to workflows, for example doing a manual node is just "when : manual" in GitLab while you have to do clickops in GitHub to achieve the same. No overview of code in branches is a minus when we tried to figure out what our colleagues are trying to merge as it looked off.

Incentivized

I think is very well designed, and like any VCS it works as intended

Incentivized

• It helped to estimate resources required for a particular client requirement. For example if the client wants the application to be used by 500 users for 1 hour, based on the load test we perform for 500 users and 1 hour we recommend resources and their capabilities.
• Jmeter was used to identify bottleneck. This would lead to better performance.
• Jmeter was used to identify failures as well.

• GitLab cut down our spent on container, package and infrastructure registry
• Best thing is we can now have everything in single platform which cost effective too
• Quality of support is really good and they do have emergency support team as well which is great

Incentivized