A/B tests for non-coders done right
August 09, 2019
A/B tests for non-coders done right
Score 6 out of 10
Vetted Review
Verified User
Overall Satisfaction with Optimizely
It is used by one team so far. It helps to address two problems: we do A/B testing, so we set up the tests and keep track of the results with Optimizely. We also wrap new developments into A/B tests so we can measure if we made a mistake towards the client's outcome, even if the expected result is to be flat (A being as good as B).
- The traffic reports don't seem to be far from our internal traffic reports.
- Non-coders appreciate the interface and graphs.
- It's slow at loading and it's blocking, so it damages your site performance.
- It uses a naive approach, where two tests running in the same page are still two A/B tests, not an A/B/C, damaging the results due to interference.
- Rarely do we save from doing a big development after a test runs. The most common is we fail when testing a big development.
- The increase in conversion is hard to measure. There are A/A tests (no changes) that report that one of the As is a winner and increases conversion.
- It saves time as a quick "techie" CMS where you can customise more than the traditional copy and images.
The reports and the segmentation are better (or at least feel more accurate).