1. Home
  2. Testing Services
  3. Applause Reviews
  4. User Review of Applause
Applause - way to save time on regression testing
September 22, 2015

Applause - way to save time on regression testing

Martin Štefan | TrustRadius Reviewer
Score 9 out of 10
Vetted Review
Verified User

Software Version

Enterprise

Modules Used

  • Functional testing
  • Localization testing

Overall Satisfaction with Applause

We use Applause in the quality engineering department for functional in-the-wild testing and localization testing. We started utilizing the service back in 2011 and the relationship evolved during time. Currently we have couple of test leads handling everything within the Applause platform, the testers and bug triage that come from the crowd source testing. The test leads also transfer valid tickets to our internal bug tracking system, which makes the collaboration between our companies seamless. Applause is utilized for multiple, mostly mobile, apps testing.
  • In-the-wild testing - it cannot be done internally. It's impossible to cover the wide matrix of devices and OS versions.
  • Exploratory testing - more eyes see more. Every mind is unique and thinks of a workflow differently - provides better coverage of the product.
  • Care provided by test lead is essential - they help a lot with initial bug triage and bug transfer to our internal systems.
  • Retest of found tickets once they are fixed - saves time!
  • Applause people make a good effort to create a relationship, it's not just business.
  • The service is quite expensive.
  • The first year of using the service is about finding the right balance. It can initially be more time consuming then the previous model without Applause service.
  • The model motivates the testers to not go very deep during testing - they are paid by bugs found. Means there are cases when they go for only the low hanging fruit.
  • Saves time spent on regression - internal resources can focus on more important tasks.
  • Saves money on device / OS coverage.
  • Price for a bug fixed is lower than price for a bug hot fixed.
By the time we signed the contract, there was no competition to Applause (back in time the company was called uTest). Ever since, we didn't evaluate anyone else because we built a very close relationship, which works for both sides. I think Applause is the furthest in building the test community.
The key value of Applause lays in exploratory testing and in-the-wild testing. Besides these, it's impossible to cover all combinations of devices and OS versions internally. That's where Applause comes in really handy! Also localization testing is really easy since native speakers can be brought in. However - for more complex applications, training is necessary and building the test community as well.

Using Applause

20 - Internally Applause is utilized by individual contributors within quality engineering (different level of seniority), by quality engineering managers, by development managers and project managers. Test cycles are run cross product (we currently run test cycles for about 5 apps, which spreads through about 12 teams; these are mostly mobile apps, however we run cycles also for browser certification testing).
1 - Internally it's just me, who supports Applause. The others just request test cycles and evaluate incoming bugs.
It's just me who provides trainings, descriptions, scope, guidelines, etc. It took a while to figure out what level of detail Applause needs, which meant bigger time investment from my side. But now, it's significantly lower and Applause actually saves me plenty time.
Required skills: time management, planning
  • App release regression
  • New feature exploratory testing
  • Bug hunt
  • Bug hunt - when we are not able to find a bug reported from the field internally
  • New OS testing (for OS which were not released yet)
  • Using Applause's webinars and ebooks for internal trainings
  • Test automation
  • Security testing
  • Performance testing
The one missing point is for the price - it's quite expensive to maintain the service to the extent of how we use it (dozens of test cycles and hundreds of test case hours on monthly basis). However the benefits still weights the price, especially when thinking of the price of potential hot fixes. Still, the price can be a reason to take a look on how competition is doing.

Evaluating Applause and Competitors

  • Positive Sales Experience with the Vendor
During the time, the relationship never felt like vendor - customer only. I know that it's mostly about luck on the people you get to work with but maybe not. Maybe Applause really hire and train their employees in a way that they make you feel it matters. It's mostly the "friendly" relationship that makes Applause different from the other vendors.
It's still about spending money and getting the best value out of each dollar spent. By the time we were purchasing the service, there was no real competition to Applause (uTest at that time), therefore it was sort of a no brainer. If we would reconsider the whole contract, we would take on more competition.

Applause Implementation

I didn't need to be involved at all. It was seamless from our perspective. All products were inserted in by Vendor, only Test Cycles we needed to insert ourselves initially. Now even that is handled by Vendor. The only thing we need to pay attention now is - to request cycles at the right time and review the bugs found during the test cycles. Nothing else. Very good experience!
  • Vendor implemented
Change management was minimal - We didn't really requested any changes so - nothing to report here.
  • Integration of the plugin that makes the transfer between applause and internal bug tracking system happening. it took time to make it seamlessly running.

Applause Support

I know the score doesn't really correspond with my earlier answers, but I have really special relationship with the Applause people. I don't really go through official support. I rather use my internal connections to make my request handled as soon as possible - and it works really well! I don't need to go through the official channels. And it's known that the unofficial ones are much more effective. I can confirm!
ProsCons
Quick Resolution
Good followup
Problems get solved
No escalation required
Immediate help available
Support understands my problem
Quick Initial Response
None
Yep, it's when I wanted to change some default values for our integration plugin between the Applause platform and our internal bug tracking system. I didn't have to go through creating a ticket or reporting through Support at all. I just let our test lead know and he made the changes happen. Amazingly fast!

Using Applause

The UI and the whole app is updated on regular basis, quite often actually. There are some cool features, like integration with several other bug tracking tools, which makes the bug management really easy. However there are some key usability issues within some of the less used workflows, which brings the score down a bit. They need to work on better switching between products and better bug search, especially across purchased products.
ProsCons
Easy to use
Technical support not required
Consistent
Familiar
None
  • In the bug chat with the tester
  • Discussions
  • Browsing between bugs
  • Bug export
  • Bug search across projects
  • Switching products
  • Test cycle setup
Yes - I actually used it just couple times a while back, maybe in 2011 or 2012. It was such a horrible experience that I rather deleted the app and haven't tried it ever since. The bug browsing and approvals were really difficult, not even mentioning switching between products, which is not exactly easy within the web platform either. Forget the search.