CircleCI vs. codeBeamer ALM vs. GoCD

Based on our experience, CircleCI is well-suited for automating mobile app release cycles. For example, to release an iOS app, you would need to build, sign, and upload it to TestFlight, which requires a dedicated Mac in the office. But with CircleCI, you can have macOS executors, so you don't have to manage a physical build machine. Another benefit is that CircleCI's certified AWS Orbs abstract away complex authentication and deployment logic, allowing us to build, push, and deploy Docker containers to Amazon ECS with minimal configuration and high reliability. CircleCI is less suited for smaller projects where the development and deployment are not that extensive, for example, a static site. Once you have built a static site, you probably won't make any further changes, so there's no point in paying for it.

Incentivized

In my opinion codeBeamer is well suited for any activity where detailed tracking is needed with teams involved. It can be less suited for small projects which do not last long, or are not continuous, etc

Previously, our team used Jenkins. However, since it's a shared deployment resource we don't have admin access. We tried GoCD as it's open source and we really like. We set up our deployment pipeline to run whenever codes are merged to master, run the unit test and revert back if it doesn't pass. Once it's deployed to the staging environment, we can simply do 1-click to deploy the appropriate version to production. We use this to deploy to an on-prem server and also AWS. Some deployment pipelines use custom Powershell script for.Net application, some others use Bash script to execute the docker push and cloud formation template to build elastic beanstalk.

Incentivized

• Automated builds! This is really why you get CircleCI, to automate the build process. This makes building your application far more reliable and repeatable. It can also run tests and verify your application is working as expected.
• Simple. Unlike Jenkins, Teamcity, or other platforms, CircleCI doesn't need a lot of setup. It's completely hosted, so there's no infrastructure to set up. The config file does take a bit to understand, but if you follow their example and start with something small and add to it, you can get it up and going quicker than it first looks.
• Scales easily. Again, since it's all cloud-based, you don't have to manage or scale infrastructure. Simply subscribe to the number of containers you want, and scaling up just means buying more containers.

Incentivized

• Tracing of requirements.
• Tracking of development.
• Tracking of QA.
• Tracking of bugs.
• Tracking of releases.
• Wiki.
• Users and teams.
• Works for teams scattered across the globe.
• Testing and coverage for releases.
• Integration with SVN.

• Pipeline-as-Code works really well. All our pipelines are defined in yml files, which are checked into SCM.
• The ability to link multiple pipelines together is really cool. Later pipelines can declare a dependency to pick up the build artifacts of earlier ones.
• Agents definition is really great. We can define multiple different kinds of environments to best suit our diverse build systems.

Incentivized

• While configuration is easy, the config files can get very very long.
• Price compared to some alternatives that are cheaper / free. Especially so if you are running multiple containers in parallel.
• Have experienced numerous outages (3-5) in the last few months where CircleCI has been down.
• Web documentation and tutorials haven't been as good as some of the competitors.

Incentivized

• Search.
• Metric calculation.
• Would be nice that cB is a source of truth, i.e., that say SVN commit is done only from cB; know this is difficult to achieve.

• UI can be improved
• Location for settings can be re-arranged
• API for setting up pipeline

Incentivized

The reliability & speed, it just works. The ability to spin up macOS runners and Docker containers on demand without managing hardware is a huge win. The Orbs system makes integrating with AWS and Slack incredibly easy, saving us weeks of custom scripting and providing real-time updates in our Slack channel. This makes it easy for us to track and ensures that everyone involved knows the status. Of course, it has drawbacks related to configuration complexity and, in some cases, cost transparency, but overall, it is an industry-standard, robust tool that solves our core infrastructure problems well.

Incentivized

Sometimes info in codeBeamer can be lost, in principle, if not well organized by the user, info can never be found, or can be forgotten about, etc

It's pretty snappy, even with using workflows with multiple steps and different docker images. I've seen builds take a long time if it's really involved, but from what I can tell, it's still at least on par if not faster than other build tools.

Incentivized

Unless you have a reasonably large account, you're going to be mainly stuck reading their documentation. Which has improved somewhat over the years but is still extremely limited compared to a platform like Digital Ocean who invested in the documentation and a community to ensure it's kept up to date. If you can't find your answer there, you can be stuck.

Incentivized

Excellent support professionally. Also, thanks for giving me for free access to SaaS cB for my hobbyist project.

Jenkins is usually self-hosted, Travis CI's infrastructure is largely unreliable (lots of tests time out for no discernable reason), and Semaphore encourages you to configure your CI/CD from a web UI. We like CircleCI because its hosted, our tests run largely as expected on their infrastructure, and we can configure it from a config file that we track in GitHub.

Incentivized

codeBeamer is by far better.

GoCD is easier to setup, but harder to customize at runtime. There's no way to trigger a pipeline with custom parameters.

Jenkins is more flexible at runtime. You can define multiple user-provided parameters so when user needs to trigger a build, there's a form for him/her to input the parameters.

Incentivized

• We pay over $5K/ month and we have high expectations for service. Sometimes I feel that we don't get the value, but only sometimes.
• We have had to build our own application to keep state and broker releases and deployments. We call our app deployer. I feel that CircleCI could do more to understand our needs and possibly build additional features that would enable us to invest less in build and deployment infrastructure and justify paying more for Circle.

Incentivized

• Customer tracking.
• Process tracking.
• Workflow framework.
• Can be overhead.

• ROI has been good since it's open source
• Settings.xml need to be backed up periodically. It contains all the settings for your pipelines! We accidentally deleted before and we have to restore and re-create several missing pipelines
• More straight forward use of API and allows filtering e.g., pull all pipelines triggered after this date

Incentivized