Azure DevOps Services

Formerly VSTS

Source control integration.

Templates for multiple Agile types.

Document management (implement something similar to Sharepoint libraries).

Streamlined permissions structure.

Seamless integration into Microsoft development tools.

CI/CD built-in, very customizable.

Lots of marketplace items for integration into various non Microsoft tools.

It is not as customizable as JIRA. They are working to improve this. But this is the one area where it is lacking.

Ease of use/integration with Visual Studio.

Very responsive and easy to maintain a site.

Has full Git capabilities.

Capability to store unlimited numbers of projects.

The price/license per user could be a little less than many of the open-source type source code platforms on the market.

Ease of integration with other development IDE's than Visual Studio.

Agile Management.

Metrics.

Customized Reporting.

Scaleable Team Projects.

Column sorting when in filtered states.

A way to show cross-team dependencies.

A customized "From" field for notifications. Sometimes when a mail comes from Azure DevOps the teams do not realize that I am sending it

A way to do online poker that doesn't require a plug-in.

The backlogs and Kanban boards for planning and tracking work are second to none. Forecasting and capacity management are made easy with the Azure DevOps tools at our disposal.

The ability to customize work items and workflow is crucial to our business. We are in a specialized, highly regulated business with requirements, unlike most software businesses. We continue to strive to keep development and delivery as lean and agile as possible. We are slowly adopting more DevOps principles. This is especially challenging for our business due to government regulations. As a result, we've adopted an Agile/Waterfall hybrid methodology. Customizing the process, work items and workflow gives us the ability to meet our unique needs.

Git integration is a key feature and keeps our developers happy.

Some of the administrative tasks and management leaves much to be desired. Security and permissions are managed in different places instead of one central location. Alerts and notifications management could use improvement.

Due to the nature of our business, we are not able to move to the cloud and must use the on-prem version. While Microsoft officially supports the on-prem version, they are geared towards the cloud version of Azure DevOps. Microsoft support for many of our on-prem needs seem to be waning.

Work items should be able to be baselined along with code. While we can label the code that was built, there is no way to take a snapshot in time of the historical state of the work items at the time of the build. A feature like this would save our QA department lots of work.

Usability: The usability of Azure DevOps is great! Being a new user, it was easy to pick up and go with this tool with very little requirement to seek external documentation.

Integration: This tool integrates well with other systems (ie. GitHub, Chef, etc).

Built in activities: Azure DevOps has a ton of prebuilt activities that allow you to basically build whatever you need without writing any extensive code.

While usability is great, it did take me a few times to find "hidden areas" (like the visual designer link for creating pipelines). Having these in more defined noticeable areas will only improve on the already great usability.

As with other Microsoft tools, the Microsoft login get's a little crazy when you have multiple accounts. In my case, I have several accounts (personal, university, and work) and getting into Azure DevOps with the appropriate account could sometimes be an act of futility.

Once set up it makes deployments to various environments a breeze

YAML backend is a huge plus

Large groups can work on the same solution seamlessly

More streamlined set up of CI/CD

Better error messaging to explain why sometimes a build is successful and sometimes not

Easier set up of deployment tools

Easy representation of software requirements in user stories and tasks.

It allows the tracking of activities and team performance

It allows interaction between team members in a specific task

You can use only a few tools to complement your existent workflow

There is a free version for small teams (up to 5 users)

Some small improvements could be done in the user interface.

Sometimes the performance is low (some requests/queries are slow)

Easy to use and set up and extendable through marketplace.

Highly flexible in configuration right out of the box.

Agent allocation and provisioning is very easy.

Cost can be a little bit more transparent for agents.

More information on their YAML strategy would help us plan better.

More configuration options in agents would be good as well as on premise agents.

Unified environment for all DevOps tasks and procedures

Amazing User Experience within a beautiful UI

Great marketplace extensions collection

Code merging has room for improvement

Such a broad set of features can become confusing for the novice user

Flexible Requirements Hierarchy Management: AZDO makes it easy to track items such as features or epics as a flat list, or as a hierarchy in which you can track the parent-child relationship.

Fast Data Entry: AZDO was designed to facilitate quick data entry to capture work items quickly, while still enabling detailed capture of acceptance criteria and item properties.

Excel Integration: AZDO stands out for its integration with MS Excel, which enables quick updates for bulk items.

Central Dashboard of Development Metrics: AZDO nests its dashboards in workflow-specific tracks, which is useful. Still, I'd like to see a home page personalized for each user which provides relevant updates on the most recent work items (updates to features, etc.) and work progress.

Complex Queries: AZDO is great for simple queries, but complex queries and the display of results doesn't always produce intuitive results. For example, sorting and drag/drop can be unreliable in some views. I suspect the AZDO team will work out these issues over the next few releases.

Lack of Themes: AZDO allows for the tracking of Epics and Initiatives, but there doesn't seem to be a structured interface for tracking product investment themes.

It offers an easy relationship between product backlog items, development tasks, and the source code changes.

No overhead management tasks, people keep focused on product development and it reduces work time.

Dashboards show the summary of the most important indicators, and offer different views for distinct professional roles, keeping people in touch with entire progress.

Full support for Scrum artifacts and processes.

Great for documenting process flows.

Great for developing org charts.

Great for mapping out ideas and brainstorming.

I'd like to be able to set up the automatic spacing and organizing workflows to flow from top to bottom.

Board view of the work we're doing. Project management has found this to be especially helpful and has enjoyed how configurable it is to our needs.

Automation: From testing to the publishing process, there are plenty of options to automate workflows and processes. This has freed up resources and time for our teams to get more meaningful work done.

Collaboration: Since we came from TFS, it's the small things that have enabled greater collaboration within the team and with other departments/stakeholders like the ability to tag individuals in comments, following an item and receive email updates, etc.

It would be nice to have a master board to see everything going on across all projects. This is more of a management view, I would think. For me, I'm having to jump back and forth between our projects. I would like to be able to projects on the master board if this view were available.

Writing detailed use stories

Assigning people to tasks

Allowing for attachments

It has nice tools for gathering data for reporting purposes.

Having a more modern user interface which doesn't feel outdated.

VSTS is very development team-centric. Development teams are working very closely with product management, who sets their tasks and priorities. There is no way in VSTS for product managers to develop product roadmaps based on product strategies and trickle that down to user stories for developers. The link between product management and development is missing.

Tools like Estimably or something similar for pointing stories in an agile development setting would be very useful, as they are used daily by agile teams.

More built-in visual reporting functionality that could be customized would be a huge asset.

Visual Studio supports many languages like C#,HTML.C++ and many others. We can also make Visual Studio to support other different languages with the help of plugins, like for Java, Python, etc.

We are using TFS with the VS, so it makes it easy to track the code and project changes, and if required we can also track the code of every single line to see what it means, by whom this code is written, and for which problem.

A unique work item change order number is assigned to the changes that you are making, apart from your shelveset creation

We can create the build definition through which we can automate the deployment and upgrade processes.

It provides a lot of menus and options/templates through which we can make our computer application for enterprise or for non-enterprise

We can easily integrate our customized/required menu in the VS through registry files or through deployment of the code

Need to make the changes so that it doesn't occupy most of the CPU utilization and memory

Execution of Bulky SQl Queries leads to either the SQl being out of exception or the VS being unresponsive

Integration with Microsoft products is easy, but with non-Microsoft products it is more difficult, and you have to make a lot of configuration changes to integrate

With every upgrade of the Visual Studio, like from VS 2010 to VS 2013 , we need to upgrade our hardware/machine, as the VS hardware requirement also increases

If code is getting compiled in one visual studio, like in VS 2010, that the same code could possibly give an error when compiled in VS 2013, due to certain changes in keyword, data format, etc., with the VS upgrade

Version control of the files: each developer has access to the server and regularly gets the latest server changes in his local box. The local file is maintained and also remains updated with the server version. The program also provides a different copy of a file so that it can be recovered in case of a failure from any individual developer box.

Some features like shelveset creation, work item monitoring, and publishing the project database.

I can access any developer's local box and see what changes he is doing if he has shared the shelveset with me.

Applying any new changes and undoing them using the pending changes feature.

Once, I added a PDF file to my shelveset so that I could share my changes with my team, but they were unable to unshelve my changes. This is because PDFs cannot be opened in more than one box, and it was locked on my box. So, I had to undo it from my box, and even then it was only accessible to one person at a time. It was problematic. They should at least be able to open it in read-only mode.

Lots of settings are necessary. However, if you accidentally uninstall one, instead of having to install it again, there should be a backup kind of thing for my profile settings in Visual Studio.

The default compare and merge tool provided by Visual Studio does not fill our needs, and we need to install other products like Delta Walker or Araxis Merge.

It allows you to incorporate different languages, or written code in different encoding to the workflow without affecting the development to which it is oriented

By using the Entity Framework you can establish the database design of the system in a correctly standardized and highly efficient way

The debugging options allow you to have control of the code that is being made, in addition to having a large amount of plugins that allow to use this feature in different languages

There are few things that we can say in a negative way because a pending issue was the license and Visual Studio has a lite option that although it does not have all the features of the full version, allows you to fulfill the work planned

The download size of the full version is usually very long and can take a long time to get it

In case of applying the entity framework in non-standardized tables (already existing projects) it generates inconveniences in the creation of the entities

VSTS has a wonderful integration with Team Foundation Source control and Git. This is good because these were two source control systems that we used.

VSTS can be scheduled to run its builds and test at various times of the day. This means it can in the middle of the night and be ready for the developers when they get in in the morning.

VSTS handles Microsoft builds very easily. Building a .Net application can be set up with almost no work. You just have to use the Visual Studio solution that was used by the developers to create the application.

VSTS is very Microsoft centric. If the application you want to build is not based on Microsoft items such as C#, or Visual Basic it is very difficult to use.

If you are using the latest version of VSTS then you will find a documentation problem. It can be very hard to find methods and help trying to get something running.

Passing data between VSTS build steps is not easy. It can be done, but it is not a normal thing that you would Microsoft would have put into their product.

Work item management is fantastic and easy to use. It can be used as simply just tracking tasks, or as complex as tracing work items from multiple projects being assigned to a single developer. Work items can be customized easily to meet the needs of your organization as well, and it isn't difficult to manage the customization.

Source code management is excellent and tied directly into VSTS. You have a choice of TFVC or GIT management options - we've switched to git and have not looked back. It is fully featured and commits can be tied directly to a work item using # tags.

Metrics/dashboard - being able to write simple queries and move them to a dashboard in a few minutes is great. It enables team managers a quick board to review the status of a project and quickly act on any issues cropping up.

Build and release management - if you don't have this... you'll want it. The integration is magic, the interface is easy, and to setup an automated build using the hosted agent was a breeze. We did have to purchase an additional license because of the number of builds we had, but it was worth it.

VSTS has a log of flexibility... almost too much. It's hard to actually decide how best to use it until you just set it up and try it out.

Currently getting a list of work items on the main home page is messy. There's no hierarchy so it can sometimes be just a blast of work without any sorting or prioritization settings. There's a simple work-around to simply create a query for yourself and enable "search across multiple projects". That works alright, but it would be a better experience if the home page had this already covered.

Teams management could be better. It's a little confusing, and not easy to setup. Our specific use case of having some teams able to only see their backlog and nothing else was not achievable in the current implementation of VSTS. As a result, whenever we have to contract out work, we end up creating a new Team Project. It's not all bad, because team project contents can be easily migrated to another project, but it's still a bit annoying.

If the backlog is well maintained, it provides a great "at a glance" view of forecasting/projections.

It provides a nice visual for committed tasks that is sequestered from the larger group of "future" work.

It allows for linking among backlog items that are related or dependent on each other.

It can be tricky to figure out the best way to use Iteration paths.

It isn't easy to differentiate iterations for certain execution cadences (sprints 1 - 6 repeated every 6 weeks).

There is not a great way to denote "code ownership" within the product (we use an external system to figure out who should be assigned what bug based on where the bug is in the code - not the UI).

Code repository

Automatic Build & Release

User management

It's easy to see the work that you are assigned. VSTS also encourages the use of its additional details so features don't get confusing

You can track progress of a particular sprint so you can see how far ahead or behind you are during a certain sprint

Integration with git is excellent so viewing difference in files, previous commits, etc. are easy with VSTS

The Query functionality is hard to use. I understand that its supposed to let users get more control of what they want to see, but its not easy

It's hard to see total hours spent on a project. I mentioned in the pro that you can track progress, but it's hard to track how much effort was already spent

Build scripts can be confusing to use. Sometimes builds fail for random reasons, but to re-run a build can be confusing

Change management

Continuous integration/delivery, automate application deployment

Code reviews, conduct a design review

VSTS online code editor

Complete Azure integration

Distributed workflow

Cost

More robust API

More straightforward comparison tool

Global availability as opposed to having everything done in silos when teams are dispersed all over the world.

Configuration and administration that reaches out to more people and has overall less impact than having to do spotted administration.

Agile project management is easier to deal with in conjunction with some third-party SCRUM/Agile tools.

Test Automation endeavors can be quickly ramped up as opposed to using other third party solutions that require greater technical resource overhead.

Possibly some test management and ALM desires that are left out.

Some performance issues not related directly to internal network performance bottlenecks.

It may be pricey from an initial outlook (for larger distributed teams) and that may throw off some companies with challenged budgets.