Azure DevOps vs. Jenkins vs. TeamCity

Azure DevOps works well when you’ve got larger delivery efforts with multiple teams and a lot of moving parts, and you need one place to plan work, track it properly, and see how everything links together. It’s especially useful when delivery and development are closely tied and you want backlog items, code and releases connected rather than spread across tools. Where it’s less of a fit is for small teams or simple pieces of work, as it can feel like more setup and process than you really need, and non-technical users often struggle with the interface. It also isn’t great if you want instant, easy programme-level views or a very visual planning experience without putting time into configuration.

Incentivized

Jenkins is a highly customizable CI/CD tool with excellent community support. One can use Jenkins to build and deploy monolith services to microservices with ease. It can handle multiple "builds" per agent simultaneously, but the process can be resource hungry, and you need some impressive specs server for that. With Jenkins, you can automate almost any task. Also, as it is an open source, we can save a load of money by not spending on enterprise CI/CD tools.

Incentivized

TeamCity is very quick and straightforward to get up and running. A new server and a handful of agents could be brought online in easily under an hour. The professional tier is completely free, full-featured, and offers a huge amount of growth potential. TeamCity does exceptionally well in a small-scale business or enterprise setting.

Incentivized

• Utilize Git as a repository to share work between multiple users
• Ability to configure Pipelines to build containers to run virtual deployments and testing scripts.
• Split individual tasks and relate to master documents for quick navigation and ability to see overall picture of project.
• Track status of each task
• Integrate with Git to utilize branches, merging, approvals, history, etc.

Incentivized

• Automated Builds: Jenkins is configured to monitor the version control system for new pull requests. Once a pull request is created, Jenkins automatically triggers a build process. It checks out the code, compiles it, and performs any necessary build steps specified in the configuration.
• Unit Testing: Jenkins runs the suite of unit tests defined for the project. These tests verify the functionality of individual components and catch any regressions or errors. If any unit tests fail, Jenkins marks the build as unsuccessful, and the developer is notified to fix the issues.
• Code Analysis: Jenkins integrates with code analysis tools like SonarQube or Checkstyle. It analyzes the code for quality, adherence to coding standards, and potential bugs or vulnerabilities. The results are reported back to the developer and the product review team for further inspection.

Incentivized

• TeamCity provides a great integration with git, especially Bitbucket.
• When a new code release (build) fails TeamCity has a great tool for investigation and troubleshooting.
• TeamCity provides a user-friendly interface. While some technical knowledge is required to use TeamCity, the design helps simply things.

Incentivized

• I did mention it has good visibility in terms of linking, but sometimes items do get lost, so if there was a better way to manage that, that would be great.
• The wiki is not the prettiest thing to look at, so it could have refinements there.
• It could improve the search slightly better.

• The UI could be slightly better, it feels kind of like the 90s, but it works well.
• An easier way to filter jobs other than views on the dashboard.
• An easier way to read the console logs when tests do fail.

Incentivized

• The customization is still fairly complex and is best managed by a dev support team. There is great flexibility, but with flexibility comes responsibility. It isn't always obvious to a developer how to make simple customizations.
• Sometimes the process for dealing with errors in the process isn't obvious. Some paths to rerunning steps redo dependencies unnecessarily while other paths that don't are less obvious.

Incentivized

I don't think our organization will stray from using VSTS/TFS as we are now looking to upgrade to the 2012 version. Since our business is software development and we want to meet the requirements of CMMI to deliver consistent and high quality software, this SDLC management tool is here to stay. In addition, our company uses a lot of Microsoft products, such as Office 365, Asp.net, etc, and since VSTS/TFS has proved itself invaluable to our own processes and is within the Microsoft family of products, we will continue to use VSTS/TFS for a long, long time.

Incentivized

We have a certain buy-in as we have made a lot of integrations and useful tools around jenkins, so it would cost us quite some time to change to another tool. Besides that, it is very versatile, and once you have things set up, it feels unnecessary to change tool. It is also a plus that it is open source.

Incentivized

It's a great help to get more information about new feature release and stay updated on what the dev team is working on. I like how easy it is to just login and read through the work items. Each work item has basic details: Title, Description, Assigned to, State, Area (what it belongs to), and iteration (when it’s worked on). See image above.They move through different states (New → Discovery → Ready for Prod → etc.).

Incentivized

Jenkins streamlines development and provides end to end automated integration and deployment. It even supports Docker and Kubernetes using which container instances can be managed effectively. It is easy to add documentation and apply role based access to files and services using Jenkins giving full control to the users. Any deviation can be easily tracked using the audit logs.

Incentivized

No, when we integrated this with GitHub, it becomes more easy and smart to manage and control our workforce. Our distributed workforce is now streamlined to a single bucket. All of our codes and production outputs are now automatically synced with all the workers. There are many cases when our in-house team makes changes in the release, our remote workers make another release with other environment variables. So it is better to get all of the work in control.

Incentivized

TeamCity runs really well, even when sharing a small instance with other applications. The user interface adequately conveys important information without being overly bloated, and it is snappy. There isn't any significant overhead to build agents or unit test runners that we have measured.

Incentivized

When we've had issues, both Microsoft support and the user community have been very responsive. DevOps has an active developer community and frankly, you can find most of your questions already asked and answered there. Microsoft also does a better job than most software vendors I've worked with creating detailed and frequently updated documentation.

Incentivized

As with all open source solutions, the support can be minimal and the information that you can find online can at times be misleading. Support may be one of the only real downsides to the overall software package. The user community can be helpful and is needed as the product is not the most user-friendly thing we have used.

Incentivized

Was not part of the process.

Incentivized

It is worth well the time to setup Jenkins in a docker container. It is also well worth to take the time to move any "Jenkins configuration" into Jenkinsfiles and not take shortcuts.

Incentivized

Microsoft Planner is used by project managers and IT service managers across our organization for task tracking and running their team meetings. Azure DevOps works better than Planner for software development teams but might possibly be too complex for non-software teams or more business-focused projects. We also use ServiceNow for IT service management and this tool provides better analysis and tracking of IT incidents, as Azure DevOps is more suited to development and project work for dev teams.

Incentivized

Overall, Jenkins is the easiest platform for someone who has no experience to come in and use effectively. We can get a junior engineer into Jenkins, give them access, and point them in the right direction with minimal hand-holding. The competing products I have used (TravisCI/GitLab/Azure) provide other options but can obfuscate the process due to the lack of straightforward simplicity. In other areas (capability, power, customization), Jenkins keeps up with the competition and, in some areas, like customization, exceeds others.

Incentivized

TeamCity is a great on-premise Continuous Integration tool. Visual Studio Team Services (VSTS) is a hosted SAAS application in Microsoft's Cloud. VSTS is a Source Code Repository, Build and Release System, and Agile Project Management Platform - whereas TeamCity is a Build and Release System only. TeamCity's interface is easier to use than VSTS, and neither have a great deployment pipeline solution. But VSTS's natural integration with Microsoft products, Microsoft's Cloud, Integration with Azure Active Directory, and free, private, Source Code repository - offer additional features and capabilities not available with Team City alone.

Incentivized

• We have saved a ton of time not calculating metrics by hand.
• We no longer spend time writing out cards during planning, it goes straight to the board.
• We no longer track separate documents to track overall department goals. We were able to create customized icons at the department level that lets us track each team's progress against our dept goals.

Incentivized

• Faster Time-to-Market: Jenkins automate the build, testing, and deployment process, enabling faster feedback and continuous improvement.
• Improved Quality: Jenkins automatically run unit tests and integration tests, ensuring that code changes meet the necessary quality standards.
• Cost Savings: Jenkins is an open-source tool that is free to use

Incentivized

• TeamCity has greatly improved team efficiency by streamlining our production and pre-production pipelines. We moved to TeamCity after seeing other teams have more success with it than we had with other tools.
• TeamCity has helped the reliability of our product by easily allowing us to integrate unit testing, as well as full integration testing. This was not possible with other tools given our corporate firewall.
• TeamCity's ability to include Docker containers in the pipeline steps has been crucial in improving our efficiency and reliability.

Incentivized