Item: Statsig
Rating: 9
Author: Verified User

Overall Satisfaction with Statsig

Use Cases and Deployment Scope

I am a Product Manager and we use Statsig to test the effectiveness of new feature releases. We are mainly concerned with user success in properly evaluating the product while onboarding

Pros and Cons

Pros

Makes setting up experiments easy
Really responsive support
Advanced experimental config for detailed statistical analysis
Post experiment analysis tools

Cons

There is no way to test two different user segments in the same experiment
Adding metadata to events is non-intuitive, the available metadata isn't shown in the UI while you are in the configuration flow
I'm not sure the recent sidebar reorganization is better than the previous version
Since we primarily use experiments as gates, with just a boolean parameter, it wasn't obvious that there is a default parameter output when calling the API to register an exposure. Maybe the default parameter object should be a mandatory function argument

Return on Investment

We uncovered several feature releases that were causing a negative impact on our product activation rate by running exclusion experiments

For the most part it is pretty easy to use. - There are some quirks with the javascript SDK (getExperiment().getValue?). - The Events vs. Metrics design pattern is complex, and creating new Metrics from Events can be frustrating if you are trying to use event metadata - It's really frustrating not to be able to link Static IDs (before a user signs up) to User IDs, in order to follow users all the way through onboarding, or to log events that occur for signed in users when you are exposing the experiment to users before they've signed up

Alternatives Considered

Flagsmith and VWO

Statsig is easier to use and has more tools for analysis, though we were working with the open source version of Flagsmith, merely to handle feature flags, and using Heap and homegrown tools for manual analysis, so the Cloud version may be different

Key Insights

Do you think Statsig delivers good value for the price?

Yes

Are you happy with Statsig's feature set?

Yes

Did Statsig live up to sales and marketing promises?

Yes

Did implementation of Statsig go as expected?

Yes

Would you buy Statsig again?

Yes

Likelihood to Recommend

Statsig is well suited for running straightforward a/b tests on frontend deployments. We've actually used it to log events on our backend and used the HTTP API to integrate with tools like Hubspot to evaluate email campaigns

Comments

Please log in to join the conversation

Heyyy Statsig

Software Version