Scaling our data layer without headaches
Updated April 13, 2020
Scaling our data layer without headaches
Overall Satisfaction with IBM Cloudant
Cloudant allows us to scale our data layer without worrying about the operations side of things to manage such a big cluster. Cloudant is the main database system for our company. It's business critical for our company and our many customers rely on its performance on a daily basis to perform their activities.
- Scale your data layer without issues.
- Great support team.
- Very few operational problems
- No detailed monitoring (which views are used or not, performance metrics) on top of the basic stuff provided by CouchDB
- No playground environment to test some very large map/reduce queries (and therefore any changes in the design have to reindex the whole DB).
- Pricing is quite steep
- No integrated backup features, as you'd expect from an enterprise product
- Ability to quickly scale and launch a new tenant on the system
Yes, Cloudant operations team manages the underlying systems on our behalf and the support team is very helpful and proactive. They monitoring when the system is reaching some critical metrics. Some operations like automated backups and exports for reporting are still up to our team, though.
This is the killer feature of CouchDB which we use extensively. This hasn't much improved recently as such (it's been working well since the beginning).
No, it's not really part of our setup with Cloudant. They were not really proactive about this.
Cloudant wouldn't really scale to Terabytes level (if you have to change any view, that would be really problematic). For a real life database with many views, 50GB to 100GB seems to be the sweet spot in our experience.
Cloudant wouldn't really scale to Terabytes level (if you have to change any view, that would be really problematic). For a real life database with many views, 50GB to 100GB seems to be the sweet spot in our experience.
- IBM Cloud Foundry
Well integrated in the IBM Cloud system.
Actually, none. Since we rely on Cloudant.
IBM Cloudant Feature Ratings
IBM Cloud Support and Implementation
Our team is quite experienced with the CouchDB/Cloudant technology so the original implementation went really smoothly. As usual with database implementation, the key is to design the data model and related view in the most future proof way so that you can avoid model and view changes.
Advanced - Very good support for time critical operations. This has been really worth it for our teams.
At some point, the system needed a further full rebalancing of the data on the nodes due to some issues with some highly conflicted database (which was a user error). Cloudant support helped our team to scale the reindexing by add free-of-charge 3 additional server nodes to the system for the during of the incident.
- Central Documentation and Learning Hub
- IBM Cloud Docs
Using IBM Cloudant
10 - - Developers
- Operations
- Operations
We don't have any engineering working specifically on the support. Of course some of the developers are involved with Cloudant on a day-to-day basis.
Skills required:
- JSON
- Javascript (views), we use nano for the Node.js parts of the application
- Map/reduce concepts
- Document design.
- Java (we use Ektorp for the Java parts of the application)
Skills required:
- JSON
- Javascript (views), we use nano for the Node.js parts of the application
- Map/reduce concepts
- Document design.
- Java (we use Ektorp for the Java parts of the application)
- Storing the business transactions performed offline
- Storing usage metrics
- Storing content configuration documents
- Reporting on all of the above.
- Distributed replications
- Waiting on some really cool use cases for DashDB.
- More reporting with DashDB.
- More ad-hoc reporting.
- Much more storage of data.
- Automatic archiving of the database (e.g. like Amazon Glacier) would be a killer feature.
Evaluating IBM Cloudant and Competitors
- Product Features
- Prior Experience with the Product
- Positive Sales Experience with the Vendor
Features, since we rely on CouchDB's interface. But also the gold support contract that gives us peace of mind and a dedicated support engineer in case of questions. We have been very happy with the support provided to our team.
No, we are happy with the service. The selection process was very simple since we were already using Cloudant through the Heroku connector (which was unexpectedly terminated, to be honest).
Configuring IBM Cloudant
- Avoid as much as possible to the Search
- Avoid large documents
- Use UUID as _id whenever possible to avoid conflicts
- Learn to write efficient views
- Avoid large documents
- Use UUID as _id whenever possible to avoid conflicts
- Learn to write efficient views
No - there is no facility to customize the interface
No - the product does not support adding custom code
Using IBM Cloudant
Pros | Cons |
---|---|
Like to use Relatively simple Easy to use Consistent Convenient Feel confident using | Requires technical support Lots to learn |
- Editing the docs in the dashboard
- Writing ad-hoc queries
- Permissions management
- The Dashboard is slow and takes time to load many databases.
- Writing a new map/reduce view indexes the entire DB for every change. In Couchbase, you have a feature to create a view for a subset of the data, this is very convenient.
IBM Cloudant Reliability
IBM Cloudant Support
Pros | Cons |
---|---|
Quick Resolution Good followup Knowledgeable team Problems get solved Kept well informed Immediate help available Support understands my problem Support cares about my success Quick Initial Response | Escalation required |
Yes - We have a gold support contract.
Yes - Yes, some deployments done a few months ago were related to some issues we raised.
Our account contact director Chris Snow is helping us to develop some extensions to Cloudant Sync during his free time.
IBM Cloudant Implementation
- Implemented in-house
Change management was a minor issue with the implementation - We designed the system around the CouchDB architecture, so this was always part of the implementation since day one. Cloudant actually provided the "service" around the CouchDB technology.
- Designing the documents to be scalable.
- (Manually) sharding of the document on many databases was a key design decision for us (we shard per "day"), so that we can more replicate back and forth.
- Designing performing views for large amounts of document (ad-hoc reporting) is very tricky.