GarrettCom Magnum Series Routers vs. Optimizely One

It is well suited for heavy-duty A/B testing where management would like to see and quantitatively determine the effect of a change. It is not so efficient to try for a single page simple form, except when the form is part of a larger workflow. The security model is not very well understood, including RBAC and protection against injection attacks.

Incentivized

• Provides quicker release cycle for the way the product cards should be displayed.
• Provides feedback for new product introduction and how it'll be perceived.
• Allows rapid blackout from user experience changes that alters traffic flow.

Incentivized

• Can use better interface for managing test, including scheduling and notification.
• Feels too heavy for simple projects, though this isn't a factor once past the PoC stage.
• Notification on various aspects can be made more powerful and granular if such a need arise.

Incentivized

There is a big difference between the two: Google Optimize uses Bayesian analysis while Optimizely uses Frequenting. There is a risk of counting multiple visits. Google Optimize data isn't available instantly (if I remember correctly). Optimizely's analyses dashboard is a lot richer and offers a better experience, though it may get intimidating.

Incentivized

• Almost immediate payoff in terms of the benefit.
• Easy to justify the TTM benefits to upper management.
• Still somewhat hard to justify ROI, though this is not my expertise.

Incentivized