GitLab vs. Optimizely Feature Experimentation

It is well-suited for any project that needs VCS. It's an excellent choice for teams that might be remote or have to collaborate across teams. Plenty of features allow for async working. With its dashboards and reporting features, it is also suitable for nontechnical PMs or stakeholders. It allows for very bespoke customization and can most often do much more than you need it to.

Incentivized

Based on my experience with Optimizely Feature Experimentation, I can highlight several scenarios where it excels and a few where it may be less suitable. Well-suited scenarios: - Multi-Channel product launches - Complex A/B testing and feature flag management - Gradual rollout and risk mitigation Less suited scenarios: - Simple A/B tests (their Web Experimentation product is probably better for that) - Non-technical team usage -

Incentivized

• GitLab excels in managing code versions, allowing easy tracking of changes, branch management, and merging contributions.
• It helps maintain code stability and reliability, saving time and effort in the development or research workflow.
• Powerful code review features, enabling collaboration and feedback among team members.
• Robust project management features, including issue tracking, kanban boards, and milestones.

Incentivized

• Splitting traffic between variants and enabling you to scale up or down the amount of traffic in each one
• Giving a standardised report that you can share with a huge number of users
• Showing a large variety of results/metrics you can then dive into

Incentivized

• CI variables management is sometimes hard to use, for example, with File type variables. The scope of each variable is also hard to guess.
• Access Token: there are too many types (Personal, Project, global..), and it is hard to identify the scope and where it comes from once created.
• Runners: auto-scaled runners are for the moment hard to put in place, and monitoring is not easy.

Incentivized

• Difficult integration if your data is not front end
• Costly MAU model needs to be based on experiments not on site visits
• It's not easy to understand how to build an Experiment
• Onboarding team is more focused on punching through their slides and not focused on your needs or understanding.

Incentivized

I really feel the platform has matured quite faster than others, and it is always at the top of its game compared to the different vendors like GitHub, Azure pipelines, CircleCI, Travis, Jenkins. Since it provides, agents, CI/CD, repository hosting, Secrets management, user management, and Single Sign on; among other features

Incentivized

Competitive landscape

Incentivized

I find it easy to use, I haven't had to do the integration work, so that's why it is a 9/10, cause I can't speak to how easy that part was or the initial set up, but day to day use is great!

Incentivized

Easy to navigate the UI. Once you know how to use it, it is very easy to run experiments. And when the experiment is setup, the SDK code variables are generated and available for developers to use immediately so they can quickly build the experiment code

Incentivized

I've never had experienced outages from GItlab itself, but regarding the code I have deployed to Gitlab, the history helps a lot to trace the cause of the issue or performing a rollback to go back to a working version

Incentivized

GItlab reponsiveness is amazing, has never left me IDLE. I've never had issues even with complex projects. I have not experienced any issues when integrating it with agents for example or SSO

Incentivized

At this point, I do not have much experience with Gitlab support as I have never had to engage them. They have documentation that is helpful, not quite as extensive as other documentation, but helpful nonetheless. They also seem to be relatively responsive on social media platforms (twitter) and really thrived when GitHub was acquired by Microsoft

Incentivized

Support was there but it was pretty slow at most times. Only after escalation was support really given to our teams

Incentivized

It’s straightforward. Docs are well written and I believe there must be a support. But we haven’t used it

Incentivized

GitHub is an inferior product from most points of view. We had to use it and the teams finds no positives about it. Everything is a downgrade from our previous GitLab solution. GitLab CI\CD is vastly superior to workflows, for example doing a manual node is just "when : manual" in GitLab while you have to do clickops in GitHub to achieve the same. No overview of code in branches is a minus when we tried to figure out what our colleagues are trying to merge as it looked off.

Incentivized

In previous companies I've used Monetate which is a similar A/B testing kind of feature experimentation engine that is very similar from my memory, but again, back to the point of these new features of the analytics engine and Opal, it kind of cuts it above Monetate from my experience. Obviously Monetate may have improved since when I lost use it, but from what I can see, yeah.

Incentivized

I think is very well designed, and like any VCS it works as intended

Incentivized

had troubles with performance for SSR and the React SDK

Incentivized

• GitLab cut down our spent on container, package and infrastructure registry
• Best thing is we can now have everything in single platform which cost effective too
• Quality of support is really good and they do have emergency support team as well which is great

Incentivized

• We have improved various metrics throughout the course of our experimentation program with Optimizely and therefore sharing numbers is tricky. Essentially we only implement versions of the product that perform the best in terms of CVR, revenue/visitor, ATV, average order value, average basket size and so forth dependent on the north star we are trying to move with each release.

Incentivized