Qualaroo: a reliable solution in a fast-changing market
Anonymous | TrustRadius Reviewer
Updated December 31, 2014

Qualaroo: a reliable solution in a fast-changing market

Score 7 out of 10
Vetted Review
Verified User
Review Source

Software Version

Professional

Overall Satisfaction with Qualaroo Insights

We use Qualaroo in combination with Google Analytics, Crazy Egg, Optimizely, and UserTesting.com to improve our overall site usability. We also use Qualaroo to learn more about our site visitors: what led them to our site, what kind of information they're looking for, and whether they're finding what they need. Though we do use Qualaroo to gather quantitative data (e.g. rate the usefulness of information on this page), the most insightful data we collect is often qualitative (tell us what else you'd like to see on this page). Though response rates are low (2-5%) and the volume of data we collect is small, we often discover new insights or are able to make better sense of our user data from the most thoughtful responses that we get.
  • Rapid roll-out of micro-surveys targeted to specific types of visitors (Qualaroo controls) and/or pages (via regular expressions) without having to use developer resources
  • Easy to brand (add a logo and change colors) to generally match or at least complement the look & feel of our site
  • With the exception of some hiccups we experienced early on with changes Qualaroo was making to the way regular expressions work for targeting "nudges" (micro-surveys), the platform has been very stable and reliable
  • Response notification and aggregate reporting (both online and via downloadable CSV files) are simple and intuitive
  • Easy to "clone" an existing nudge to rapidly modify and launch as a new micro-survey
  • We're able to pass session information directly into a Qualaroo submission from our site so that we can identify respondents without asking for an email address or other unique identifier (so long as they're logged in as returning/repeat site users).
  • The way regular expression targeting works can be quirky, depending on how your site works. A "nudge" that is sometimes supposed to display only on certain pages will sometimes carry over to a page where it should not display.
  • The interface for editing and sequencing steps within a "nudge" becomes cumbersome once there are more than just 3-4 questions/screens bundled into the "nudge" (think 3-4 single-question pages of a typical web survey). You can use branch logic to make a very sophisticated question sequence and you can preview/test thoroughly, but it's a challenging UI for advanced use cases.
  • The "exit survey" functionality does not work well on landing pages because the "exit" nudge displays when a user's mouse is in the tabs or navigation area of the browser. That's often where a user's mouse is located on page load for a landing page (especially direct navigation), so the "nudge" displays too quickly. There's no ability to set a delay before the exit "nudge" can be triggered.
  • Qualaroo's NPS question type is too constraining (no control over labels) and the presentation looks dated compared to other micro-survey tools that I've seen since we signed up for Qualaroo.
  • Qualaroo has allowed us to take questions that come up in weekly team meetings and translate them immediately (same day) into questions we pose to users, with actionable insights coming back to us within 3-5 days. For sites with larger traffic volumes (in the 100s of thousands of uniques per month, vs ours currently in the 10s of thousands per month), you can probably get actionable data even faster.
  • Qualaroo has worked well as an off-the-shelf micro-survey solution, allowing us to focus our limited development resources on other higher-priority projects (vs. trying to engineer our own micro-survey features into our site).
At the time that we chose Qualaroo (fall of 2013), it had features that we couldn't get with WebEngage (regular expression targeting and exit surveys). Qualaroo is also a more pro-active form of soliciting feedback than UserVoice. Qualaroo is more suited to an active marketing and sales mindset where UserVoice is more suited to a passive tech support mindset. Qualtrics, SurveyMonkey, and Wufoo are all more traditional, long-form, stand-alone web survey tools.
Qualaroo's exit survey feature is probably better-suited to eCommerce sites (which we are not). The two questions I would ask in considering Qualaroo are:

1) How much control do you need over targeting specific "nudges" to specific pages and how complex are the URLs on your site? If you want tight controls, Qualaroo can give you that, but you need to be able to write (potentially) complex regular expressions to get the targeting you need. If you don't need tight controls, there are likely simpler and cheaper options out there.

2) Do you want a solution that integrates seamlessly into existing systems? If your site has a registration wall or a pay wall or you otherwise have access to identifying information about your users, Qualaroo's API makes it easy to know exactly who is responding to your surveys. Again, there are less expensive options out there if you don't need this.

Using Qualaroo Insights

1 - Customer Service
Our ongoing use of Qualaroo requires minimal technical in-house support. This may change as we are planning to make more use of the Qualaroo API to control when and where specific nudges appear to specific users. Applying these more granular controls requires some developer effort on our end since we do not have a tag management system implemented.
  • Gathering on-page user feedback about specific content areas or pages within our site.
  • Quick polls (with follow-up survey questions) that contribute to the original research that we produce.
  • Engaging users who look like they're about to abandon either a landing page or a page in one of our interactive funnels to understand why they're abandoning.
  • We've used Qualaroo as a single call-to-action pop-up that redirects the user to a specific landing page or survey. This has allowed us to promote specific, timely offers and drive conversion.
  • We've also used Qualaroo pop-ups as purely informational banners. We're testing whether a Qualaroo nudge can increase our site registration rates by explaining to new visitors how our content wall works. This is a temporary testing step prior to committing developer resources to changing the way our registration wall works.
  • We'd like to be able to pass data we collect via Qualaroo directly into our CRM and marketing automation systems to trigger personalized follow-ups with our site visitors. This is currently only available with the much pricier Enterprise plan (which we don't have). If lower-cost competitors develop this capability, we will likely switch platforms.
The micro-survey landscape is evolving very quickly and it seems like there is a new entrant almost every month. I'm generally happy with Qualaroo (we got what we expected) and I like the folks on the Qualaroo team (support is fast and friendly and rarely needed), but there haven't been real improvements to the platform in the last eight months. We will be re-assessing Qualaroo's features and price versus other competitors when our current contract is up.

Qualaroo Implementation

Have your technical team set up the "Identify" feature right out of the gate - this is a "fire and forget" feature. Once it's set up, your data is much more valuable. Also, if there are very specific kinds of page targeting you might imagine doing, have a technical person set up and test at least 2-3 examples of using Qualaroo's regular expression features to achieve this targeting. With a couple good examples in place to work from, it becomes much easier to create your own regular expressions for custom targeting.
Change management was minimal - We had to come to terms with trading off some site registrations for user feedback where the Qualaroo nudges were competing for attention with other existing calls-to-action on our website. It took a few months for all of our stakeholders to embrace this trade-off. Once we started getting valuable insights from our Qualaroo nudges (with little or no impact on response rates to other calls-to-action on the site), we were able to ramp up our use of the product.

Qualaroo Training

Not available for the Pro edition.
Qualaroo doesn't offer full-blown online training, at least not that I've used or needed. The online help documentation is adequate and Qualaroo-sponsored webinars are informative.

Qualaroo Support

Qualaroo support is prompt, but seems ad hoc. From my limited experience, I would say that Qualaroo doesn't have a very rigorous process in place for tracking cases yet. It's all email-based with no customer portal where you can see your cases (current and historical). On the positive side, I haven't had to submit a support case in over 10 months, so it's entirely possible that their processes have evolved in the interim.
ProsCons
Quick Resolution
Good followup
Knowledgeable team
Problems get solved
No escalation required
Support understands my problem
Support cares about my success
Quick Initial Response
Not kept informed

Using Qualaroo

Qualaroo's feature set is narrow, keeping the admin interface relatively simple to navigate.

Qualaroo recently improved the way users can write and test regular expressions, improving the usability of that feature significantly (this was likely motivated by a high volume of support inquiries to Qualaroo on this feature).

The one area where Qualaroo could significantly improve usability is on the ability to design nudges with multiple questions. Adding a drag-and-drop feature to organize questions (even if this doesn't actually affect the branch logic that sequences the questions themselves) would be a big improvement.
ProsCons
Like to use
Relatively simple
Easy to use
Well integrated
Consistent
Quick to learn
Convenient
Feel confident using
None
  • Very easy to clone a "nudge" to create a new micro survey based on prior surveys.
  • Targeting of "nudges" using regular expressions has gotten better over time. You can now test your regular expression against specific URLs right in the Qualaroo interface without having to go to a 3rd-party tool for this kind of testing. This has made it much easier to work with regular expressions, especially for minor tweaks in targeting.
  • It's really easy to update the look and feel of "nudges" and to format the content within the nudge. This has gotten better over time with new support for HTML mark-down.
  • Qualaroo calculates response rates across all "nudges" and displays that in your dashboard. There's no comparable response rate calculation displayed for each individual "nudge". This is easy enough to calculate, but it would be nice not to have to do the mental math constantly or to be reaching for my calculator to figure out response rates for the nudges I care about most right now.
  • Having a single "dashboard" where all your "nudges" are visible in one long list and a related "Archived" tab is nice for it's simplicity, but a bit cumbersome once you're running multiple nudges across multiple domains.

Qualaroo Reliability

Qualaroo's simplicity has its down sides when it comes to scaling. Managing multiple nudges across multiple domains, at least in the Pro edition, is challenging. The active nudges all appear as a single list on the admin home page, automatically grouped by domains. Archived nudges are on a separate tab. It would be helpful to have a folder system to organize nudges.

Also, once you have multiple active nudges that are targeted to different sections of your site using regular expressions, it's difficult to keep track of what is appearing where. It's also difficult, if not impossible, to figure out where you may have competing nudges.

And there's no ability to assign reporting or configuration access to other members of your team. The Pro edition has a single login, which must be shared if you have multiple staff who want to use Qualaroo.


I've never experienced an outage where Qualaroo was unavailable. I don't think I've even gotten notices for scheduled down/maintenance time. The platform is always available.
Our developers complain that Qualaroo is just one of several similar site add-ons that slow down our site performance. I'm skeptical that Qualaroo has any measurable impact on our page load times, but don't have any hard evidence either way. The Qualaroo admin interface and reporting mechanisms are all quick and reliable.