CircleCI vs. GitHub vs. Heroku Platform

Based on our experience, CircleCI is well-suited for automating mobile app release cycles. For example, to release an iOS app, you would need to build, sign, and upload it to TestFlight, which requires a dedicated Mac in the office. But with CircleCI, you can have macOS executors, so you don't have to manage a physical build machine. Another benefit is that CircleCI's certified AWS Orbs abstract away complex authentication and deployment logic, allowing us to build, push, and deploy Docker containers to Amazon ECS with minimal configuration and high reliability. CircleCI is less suited for smaller projects where the development and deployment are not that extensive, for example, a static site. Once you have built a static site, you probably won't make any further changes, so there's no point in paying for it.

Incentivized

GitHub is an easy to go tool when it comes to Version Controlling, CI/CD workflows, Integration with third party softwares. It's effective for any level of CI/CD implementation you would like to. Also the the cost of product is also very competitive and affordable. As of now GitHub lacks capabilities when it comes to detailed project management in comparison to tools like Jira, but overall its value for money.

Incentivized

Heroku is very well suited for startups looking to get a server stack up and running quickly. There is little to no overhead when managing your instances. However, you'll need a background in basic DevOps or system management to make sure everything is set up correctly. In addition, it's easy to accidentally go crazy on pricing. Make sure you're only creating the server instances you need to run the base application and set up an auto-scaler plugin to handle peaks.

Incentivized

• Automated builds! This is really why you get CircleCI, to automate the build process. This makes building your application far more reliable and repeatable. It can also run tests and verify your application is working as expected.
• Simple. Unlike Jenkins, Teamcity, or other platforms, CircleCI doesn't need a lot of setup. It's completely hosted, so there's no infrastructure to set up. The config file does take a bit to understand, but if you follow their example and start with something small and add to it, you can get it up and going quicker than it first looks.
• Scales easily. Again, since it's all cloud-based, you don't have to manage or scale infrastructure. Simply subscribe to the number of containers you want, and scaling up just means buying more containers.

Incentivized

• Version control: GitHub provides a powerful and flexible Git-based version control system that allows teams to track changes to their code over time, collaborate on code with others, and maintain a history of their work.
• Code review: GitHub's pull request system enables teams to review code changes, discuss suggestions and merge changes in a central location. This makes it easier to catch bugs and ensure that code quality remains high.
• Collaboration: GitHub provides a variety of collaboration tools to help teams work together effectively, including issue tracking, project management, and wikis.

• Heroku has a very simple deployment model, making it easy to get your application up-and-running with minimal effort. We can focus on our efforts the unique aspects of our application.
• The robust add-on marketplace makes it easy to try out new approaches with minimal effort and investment -- and when we settle on a solution, we can easily scale it.
• Heroku's support is quite good -- their staff is quite technical and willing to get into the weeds to diagnose even complicated problems.

Incentivized

• While configuration is easy, the config files can get very very long.
• Price compared to some alternatives that are cheaper / free. Especially so if you are running multiple containers in parallel.
• Have experienced numerous outages (3-5) in the last few months where CircleCI has been down.
• Web documentation and tutorials haven't been as good as some of the competitors.

Incentivized

• Not an easy tool for beginners. Prior command-line experience is expected to get started with GitHub efficiently.
• Unlike other source control platforms GitHub is a little confusing. With no proper GUI tool its hard to understand the source code version/history.
• Working with larger files can be tricky. For file sizes above 100MB, GitHub expects the developer to use different commands (lfs).
• While using the web version of GitHub, it has some restrictions on the number of files that can be uploaded at once. Recommended action is to use the command-line utility to add and push files into the repository.

• Large price jumps between certain resource tiers (2x Dyno for $50 per month versus Performance Dyno for $250). Free Postgres next jumps to $50 per month.
• Marketing/Branding to non-technical stakeholders. As the years pass, I've had to fight more to convince stakeholders on the value of Heroku over AWS.
• Improve Buildpack documentation. This is one area where Heroku's documentation is fairly confusing.

Incentivized

GitHub's ease of use and continued investment into the Developer Experience have made it the de facto tool for our engineers to manage software changes. With new features that continue to come out, we have been able to consolidate several other SaaS solutions and reduce the number of tools required for each engineer to perform their job responsibilities.

Heroku is easy to use, services a ton of functions for you out of the box, and provides a means to get a software product off the ground and managed quickly and easily. The tools provide allows a small to medium size org to move very quickly. The CLI tools provided make managing an entire technical infrastructure simple.

Incentivized

The reliability & speed, it just works. The ability to spin up macOS runners and Docker containers on demand without managing hardware is a huge win. The Orbs system makes integrating with AWS and Slack incredibly easy, saving us weeks of custom scripting and providing real-time updates in our Slack channel. This makes it easy for us to track and ensures that everyone involved knows the status. Of course, it has drawbacks related to configuration complexity and, in some cases, cost transparency, but overall, it is an industry-standard, robust tool that solves our core infrastructure problems well.

Incentivized

GitHub is a clean and modern interface. The underlying integrations make it smooth to couple tasks, projects, pull requests and other business functions together. The insights and reporting is really strong and is getting better with every release. GitHub's PR tooling is strong for being web based, i do believe a better code editor would rival having to pull merge conflicts into local IDE.

Incentivized

Easy to use web based console and easy to use command line tools; deployment is done directly from a GIT repository. What more could you ask for? The one thing that keeps me from giving it a 10 is that custom build packs are almost incomprehensible. We used one for a while because we needed cairo graphics processing. Fortunately, I was able to figure out a different way to do what we needed so that we could get off the custom build pack.

Heroku availability correlates pretty strongly to AWS US EAST availability. We had a couple of times where there was a Heroku-specific issue but not for the last 7-8 months.

It's pretty snappy, even with using workflows with multiple steps and different docker images. I've seen builds take a long time if it's really involved, but from what I can tell, it's still at least on par if not faster than other build tools.

Incentivized

The only issue that I ever have is that about 1 out of 20 deployments (git push) will hang and need to be cancelled and done again.

Unless you have a reasonably large account, you're going to be mainly stuck reading their documentation. Which has improved somewhat over the years but is still extremely limited compared to a platform like Digital Ocean who invested in the documentation and a community to ensure it's kept up to date. If you can't find your answer there, you can be stuck.

Incentivized

There are a ton of resources and tutorials for GitHub online. The sheer number of people who use GitHub ensures that someone has the exact answer you are looking for. The docs on GitHub itself are very thorough as well. You will often find an official doc along with the hundreds of independent tutorials that answers your question, which is unusual for most online services.

Incentivized

I've used it for many years without facing any major problem. It's not hard at all to get used to it, it's documentation is outstanding and simple. We are close to 2020 and I don't think most of the existing companies or startups should still face old problems such as wasting time deploying code and calculate computing resources.

Incentivized

Be ready to pay a bit more than expected in the beginning if you're migrating from a big server. The application is probably not ready for the change and you have to keep improving it with time.

It's also important to consider that you can't save anything to the disc as it will be lost when your application restarts, so you have to think about using something like S3.

Incentivized

Jenkins is usually self-hosted, Travis CI's infrastructure is largely unreliable (lots of tests time out for no discernable reason), and Semaphore encourages you to configure your CI/CD from a web UI. We like CircleCI because its hosted, our tests run largely as expected on their infrastructure, and we can configure it from a config file that we track in GitHub.

Incentivized

While I don't have very much experience with these 2 solutions, they're two of the most popular alternatives to GitHub. Bitbucket is from Atlassian, which may make sense for a team that is already using other Atlassian tools like Jira, Confluence, and Trello, as their integration will likely be much tighter. Gitlab on the other hand has a reputation as a very capable GitHub replacement with some features that are not available on GitHub like firewall tools.

Incentivized

Heroku is the more expensive option for hosting compared to some of the cloud platforms we investigated, but it's worth it for us because of the plug-and-play nature of Heroku deployment. We can be up and running in a few minutes and know with precision how much it will cost us each month to run the application, unlike Amazon Web Services where you have to go to great pains to configure it correctly or else you might end up with a shocking monthly bill. Overall, spending the time to configure Amazon Web Services or one of its competitors is likely the more affordable and powerful choice, because you have control over so many specifics of the configuration. But it also requires the burden of continuing to maintain and update your AWS instance, whereas with Heroku they take care of security fixes and platform upgrades. It's a great service and we are happy to pay the extra cost for the value-adds Heroku provides.

Incentivized

• We pay over $5K/ month and we have high expectations for service. Sometimes I feel that we don't get the value, but only sometimes.
• We have had to build our own application to keep state and broker releases and deployments. We call our app deployer. I feel that CircleCI could do more to understand our needs and possibly build additional features that would enable us to invest less in build and deployment infrastructure and justify paying more for Circle.

Incentivized

• Team collaboration significantly improved as everything is clearly logged and maintained.
• Maintaining a good overview of items will be delivered wrt the roadmap for example.
• Knowledge management and tracking. Over time a lot of tickets, issues and comments are logged. GitHub is a great asset to go back and review why x was y.

Incentivized

• It has been critical in seamlessly operating our platform with runs all of our programs.
• It has been impressive with its ability to scale quickly which results in the growth of our work.
• It allows for tracking of different features which allows for quick problem solving which saves us time.