A little underwhelming
Updated January 24, 2017
A little underwhelming
Score 2 out of 10
Vetted Review
Verified User
Overall Satisfaction with Qubit
Qubit is used for AB tests and wider MVT by the Ecommerce team, with the aim of improving customer experience on the website (where longer term fixes need longer timeframes / internal IT resource) and testing new ideas before these are added to the wider roadmap. It enables changes to be made at pace where, without an MVT solution, we would be unable to act as quickly.
- Self service solution so that tests can be built in house without support.
- Serviced offering for more complicated tests to be built.
- Difficult to use interface means reporting on tests is clunky. While improvements are being made with Livetap, this then relies heavily on clients having their own analysts capable of manipulating SQL databases to get access to the information they need, which is less than ideal.
- Reliability of data has proven problematic, with insufficient time spent validating the data within Qubit against other client sources before banding around results that later turn out to be incorrect.
- Problems with service levels, with a lot of the strategist team not close enough to the tests or how Qubit works, and more focused on big numbers they can promote than whether a test has worked successfully / whether the data is statistically robust.
- Difficulties using the serviced solution as often projects took longer than expected and communication on the progress of these was poor, making it difficult to work through a backlog of requests and manage expectations within the company in terms of delivery dates.
The decision to use Qubit was made based on the capacity to run a 3 month test before committing for a longer term and down to a recommendation from a sister company.