1. Home
  2. Market Research Services
  3. Qualtrics Reviews
  4. User Review of Qualtrics
Qualtrics: Bang for Your Buck
February 25, 2014

Qualtrics: Bang for Your Buck

Benjamin Sider | TrustRadius Reviewer
Score 8 out of 10
Vetted Review
Verified User

Software Version

Enterprise

Modules Used

  • Research Suite

Overall Satisfaction

Qualtrics is used by a handful of people in our The New York Times Consumer Insight Group. We typically use it for:
  • Web intercept studies related to ad sales, consumer marketing and strategic planning
  • Qualitative research recruitment for our user experience team
While some studies are completed within a week or two of fieldwork, others are used for trending purposes over time.
  • Customer support - I like being able to talk to someone on the phone who can help me with technical issues related to complex programming issues. Although the support line is sometimes busy, the majority of the time I am able to talk to someone live who can fix my issue in a timely manner.
  • Intuitive programming - Coming from a basic background of survey programming, Qualtrics is the best I've used to date (compared to Survey Builder, Instant.ly, Key Survey and of course Survey Monkey). I like being able to seamlessly copy and paste text into Qualtrics very easily, use display logic appropriately and capture embedded data when needed
  • Top line reporting - I've had success working directly in the Results feature as well as exporting to Excel. Running filters is quite easy before exporting to Excel so that you can analyze various subgroups. I've even pulled in a vendor to run traditional tables using Qualtrics' SPSS file.
  • Cross Tabs - I like that Qualtrics offers the feature, but one of my biggest complaints is that it takes quite a long time to load responses when you need to merge columns or stubs together. Also, you have to merge two columns/stubs one at a time instead of doing all of your merging and running it at the very end. It is very time consumer and inefficient when doing simple things like age breaks, top 2 box, etc.
  • Email response times - While phone support is strong, I recall email support taking more than one business day to get a response for some simple/standard questions. I would like to see Qualtrics invest in online chatting windows similar to retailers to talk through issues.
  • Randomization - Currently you cannot nest response options together and randomize them.
  • Cost per complete - Sometimes we have niche targets that are more difficult to reach. My team has had to purchase additional batches of survey completions. Given that the sample is solely ours and not from a gen pop panel, I think Qualtrics' costs could be a little more client-friendly when it comes to those who terminate vs. actual qualified respondents.
  • Ability to recruit highly qualified individuals for qualitative research
  • Conducting research for ad sales projects that require very short turnaround times
  • Tracking attitudinal behavior over time
  • SurveyBuilder,SurveyMonkey,KeySurvey
I didn't choose Qualtrics per se - the contract was already in place when I started at NYT. But the level of sophistication and customer support is far better than other competitive products.
The simple fact is that I have a lot of historical data and questionnaires already tied into Qualtrics. Having the same product in place makes it easy to look back at old studies for learning purposes as well as design purposes for consistency.
It is well suited when you are under budget restraints and cannot afford a full service vendor. Studies where you are simply looking at total and a handful of easy to filter sub groups such as age/gender/HHI. It is far less appropriate when you need more advanced statistical needs such as segmentation, max diff, etc.

Product Usage

7 - Custom research
7 - Basic survey programming and writing
  • Web intercept
  • Qualitative recruitment
  • Panel
  • Being able to save money by programming ourselves but having a vendor do cross tabbing by exporting the data file to SPSS
  • Transitioning a proprietary panel to Qualtrics for more of a do it yourself approach

Evaluation and Selection

Not Sure
  • Product Features
  • Product Usability
  • Existing Relationship with the Vendor
I'm not the decision-maker, but did offer input in the 2014 contract renewal since I've been using Qualtrics extensively for 2.5 years.
N/A

Implementation

I like being able to easily integrate new colleagues by simply giving them a log in user name and password.
  • Implemented in-house

Support

One point docked since waiting times occur occasionally and another point docked for the long time it takes for an email response.
ProsCons
Quick Resolution
Good followup
Knowledgeable team
Problems get solved
No escalation required
Support understands my problem
Support cares about my success
Quick Initial Response
None
Yes - Some of our surveys are very complex. Using Survey Flow for pathing/display can be tricky, but I've learned a great deal by working with the support team.
Qualtrics is fantastic when it comes to including java script coding to make my surveys more presentable. I love it when they simply log in to my account and make a change on the spot. They are also really good for logic/piping/display complexities. Their support team is very patient.

Usability

I have trained one colleague who had no background in survey programming. She has picked it up quite nicely and can program things fairly easily without much assistance.
ProsCons
Like to use
Relatively simple
Easy to use
Technical support not required
Well integrated
Consistent
Quick to learn
Convenient
Feel confident using
Familiar
None
  • Survey look and feel options/design
  • Creating basic questions, but having the option to use grids, sliding scales, etc to change it up
  • Top line reporting
  • Cross tabulation
Yes - Pretty well, but many of my surveys have matrix style questions which don't work well for mobile.