Apache Kafka is an open-source stream processing platform developed by the Apache Software Foundation written in Scala and Java. The Kafka event streaming platform is used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
N/A
Atlassian Bamboo
Score6.7 out of 10
N/A
Australian company Atlassian offers Bamboo, a continuous integration server.
$1,200
1 remote agent
GoCD
Score8 out of 10
N/A
GoCD, from ThoughtWorks in Chicago, is an application lifecycle management and development tool.
Apache Kafka is well-suited for most data-streaming use cases. Amazon Kinesis and Azure EventHubs, unless you have a specific use case where using those cloud PaAS for your data lakes, once set up well, Apache Kafka will take care of everything else in the background. Azure EventHubs, is good for cross-cloud use cases, and Amazon Kinesis - I have no real-world experience. But I believe it is the same.
If you value integration over cost, Bamboo is clearly the way to go. It offers tight integration to the rest of the Atlassian suite, and when you need traceability from issue to build, Atlassian is the right way to go. However, if you find yourself needing to save on costs, you may consider taking an approach of rolling your own build system with open source alternatives, such as Jenkins, if you don't [mind] putting in a little extra elbow grease.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Previously, our team used Jenkins. However, since it's a shared deployment resource we don't have admin access. We tried GoCD as it's open source and we really like. We set up our deployment pipeline to run whenever codes are merged to master, run the unit test and revert back if it doesn't pass. Once it's deployed to the staging environment, we can simply do 1-click to deploy the appropriate version to production. We use this to deploy to an on-prem server and also AWS. Some deployment pipelines use custom Powershell script for.Net application, some others use Bash script to execute the docker push and cloud formation template to build elastic beanstalk.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Really easy to configure. I've used other message brokers such as RabbitMQ and compared to them, Kafka's configurations are very easy to understand and tweak.
Very scalable: easily configured to run on multiple nodes allowing for ease of parallelism (assuming your queues/topics don't have to be consumed in the exact same order the messages were delivered)
Not exactly a feature, but I trust Kafka will be around for at least another decade because active development has continued to be strong and there's a lot of financial backing from Confluent and LinkedIn, and probably many other companies who are using it (which, anecdotally, is many).
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Levels of granularity. Organization has many projects that have many build plans that have many jobs that have many tasks, etc. And branch builds allow source control branches to be built separately.
Versatility. I can use bamboo to manage my Java, node, or .NET build plans. I can use it to spin up Windows or Linux build agents, or install it on a Mac to build there as well.
Bamboo integrates with other Atlassian products like Bitbucket, Stash, JIRA, etc. If a company commits to the entire Atlassian stack then work can be tracked through the whole development lifecycle which is really useful.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Pipeline-as-Code works really well. All our pipelines are defined in yml files, which are checked into SCM.
The ability to link multiple pipelines together is really cool. Later pipelines can declare a dependency to pick up the build artifacts of earlier ones.
Agents definition is really great. We can define multiple different kinds of environments to best suit our diverse build systems.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Sometimes it becomes difficult to monitor our Kafka deployments. We've been able to overcome it largely using AWS MSK, a managed service for Apache Kafka, but a separate monitoring dashboard would have been great.
Simplify the process for local deployment of Kafka and provide a user interface to get visibility into the different topics and the messages being processed.
Learning curve around creation of broker and topics could be simplified
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Apache Kafka is highly recommended to develop loosely coupled, real-time processing applications. Also, Apache Kafka provides property based configuration. Producer, Consumer and broker contain their own separate property file
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Bamboo offers solid usability for teams looking for an integrated, scalable CI/CD solution, especially those using Atlassian tools. Its interface is intuitive for existing Atlassian users, and its focus on deployment automation makes it a strong option for continuous delivery. However, its complexity and cost may pose challenges for small teams or those new to CI/CD. Overall, Bamboo’s usability shines in environments where ease of integration and streamlined workflows are prioritized. Still, it may require more effort for teams unfamiliar with its setup or without dedicated resources.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Support for Apache Kafka (if willing to pay) is available from Confluent that includes the same time that created Kafka at Linkedin so they know this software in and out. Moreover, Apache Kafka is well known and best practices documents and deployment scenarios are easily available for download. For example, from eBay, Linkedin, Uber, and NYTimes.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Bamboo is a fairly small product but having said that it was fairly easy to get assistance. Especially for the small to easy things. Anything large or fairly complex was an issue finding detailed answers for. This caused a lot of trial and error on our part to try to find a solution.
I used other messaging/queue solutions that are a lot more basic than Confluent Kafka, as well as another solution that is no longer in the market called Xively, which was bought and "buried" by Google. In comparison, these solutions offer way fewer functionalities and respond to other needs.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
We selected Bamboo because its capabilities to integrate with other Atlassian products specially Jira Software, Bitbucket and in some useful scenarios with Confluence. Also, we found these pros important for us: great user interface, easily agent deployment, Docker compability, simply to maintain / manage, and straightforwardly integration with different notification platforms
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
GoCD is easier to setup, but harder to customize at runtime. There's no way to trigger a pipeline with custom parameters.
Jenkins is more flexible at runtime. You can define multiple user-provided parameters so when user needs to trigger a build, there's a form for him/her to input the parameters.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Positive: Get a quick and reliable pub/sub model implemented - data across components flows easily.
Positive: it's scalable so we can develop small and scale for real-world scenarios
Negative: it's easy to get into a confusing situation if you are not experienced yet or something strange has happened (rare, but it does). Troubleshooting such situations can take time and effort.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
It helped us achieve the Continuous Deployment and Continuous Integration goals for our applications, a huge milestone that saved a lot of time for developers in making the builds and deployments and saved time for QA in running the automated tests.
Helped with DevOps: we moved the formal approval from the email to the system and allowed the approver to actually push the button for the production deployments.
Biggest positive impact of using Bamboo is that it improved our response time to customers and increased the frequency of our deliveries to them.
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
Settings.xml need to be backed up periodically. It contains all the settings for your pipelines! We accidentally deleted before and we have to restore and re-create several missing pipelines
More straight forward use of API and allows filtering e.g., pull all pipelines triggered after this date
A de minimis incentive was given to thank the reviewer for their time. The incentive was not used to bias or drive a particular response, nor was the incentive contingent on a positive endorsement. More Info
