Testing FTW!
January 18, 2017
Testing FTW!
Score 9 out of 10
Vetted Review
Verified User
Overall Satisfaction with CircleCI
We're using CircleCI to run our suite of unit tests for a few projects (we handle end-to-end testing separately). We love that it auto-runs tests on git commits, runs our tests in parallel containers, and is configured with a simple YAML file. The UX is super clean and easy to debug when things go wrong. We're part of their beta program and they have some great features coming down the product pipeline that addresses our few headaches like their lack of support for "docker exec" (you have to use lxc-attach) and old Ubuntu containers.
We use this product every day and we're pretty happy with it.
We use this product every day and we're pretty happy with it.
- Parallel testing - run your unit tests in parallel containers shared across all branches.
- UX - When your tests fail CircleCI makes it easy to see what failed and why.
- Configuration - whether your project runs NodeJS, PHP, Docker, Java, or whatever else you can control just about every step of the provisioning process.
- Price - You pay by concurrent containers. We're currently using 10 to run unit test across dozens of branches for a completely reasonable price. No caches.
- Incomplete Docker support - currently CircleCi doesn't support docker exec (although they have a workaround), which is a little frustrating for our team.
- Old Ubuntu versions - the containers used to use Ubuntu Precise which made provisioning more difficult, but they are currently moving to Trusty Tahr.
- Quality - Our product quality increased dramatically as engineers started running tests on every commit instead of ad-hoc before merging code.
- Testing Speed - Our unit test framework was taking up to 30 minutes to run 1,000+ unit and integration tests. Moving to CircleCI allowed us to cut that down to around 10 minutes by running test in parallel.