Scaling our data layer without headaches
Updated April 28, 2021

Scaling our data layer without headaches

Nicolas Peeters | TrustRadius Reviewer
Score 9 out of 10
Vetted Review
Verified User

Overall Satisfaction with IBM Cloudant

Cloudant allows us to scale our data layer without worrying about the operations side of things to manage such a big cluster. Cloudant is the main database system for our company. It's business critical for our company and our many customers rely on its performance on a daily basis to perform their activities.
  • Scale your data layer without issues.
  • Great support team.
  • Very few operational problems
  • No detailed monitoring (which views are used or not, performance metrics) on top of the basic stuff provided by CouchDB
  • No playground environment to test some very large map/reduce queries (and therefore any changes in the design have to reindex the whole DB).
  • Pricing is quite steep
  • No integrated backup features, as you'd expect from an enterprise product
  • Ability to quickly scale and launch a new tenant on the system
Yes, Cloudant operations team manages the underlying systems on our behalf and the support team is very helpful and proactive. They monitoring when the system is reaching some critical metrics. Some operations like automated backups and exports for reporting are still up to our team, though.
This is the killer feature of CouchDB which we use extensively. This hasn't much improved recently as such (it's been working well since the beginning).
No, it's not really part of our setup with Cloudant. They were not really proactive about this.
Cloudant wouldn't really scale to Terabytes level (if you have to change any view, that would be really problematic). For a real life database with many views, 50GB to 100GB seems to be the sweet spot in our experience.
  • IBM Cloud Foundry
Well integrated in the IBM Cloud system.
Actually, none. Since we rely on Cloudant.
It is less suited for reporting. Reporting is an issue. This is a general issue with document-based NoSQL systems, but there's no real solution provided by IBM for this (apart from "dump to a SQL database"). It's very well suited for a document-based system and we of course rely very much on the replication capabilities.

IBM Cloudant Feature Ratings

Data model flexibility
Deployment model flexibility

IBM Cloudant Usability and Efficiency

Clearly, this is one of the most important features of a PaaS like IBM Cloud (where Cloudant is hosted as a platform). It literally saves one DevOps FTE to manage the clusters and the bare metal or Cloud instances. No more worries about disk space and networking.

One thing that is missing though is a good facility for Backups (on top of simple Node scripts). In the early days, this was present as a feature, but has been removed lately.
It's very easy to operate with limited staff. Cloudant has arranged default configurations that are managed by their team of experts, so users like ourselves don't have to worry about this. On IBM Cloud it's possible to change the capacity with a convenient slider (more throughput can easily be added or reduced, even programmatically).
- Applications that are eventually consistent and often offline.
- Replication is the killing feature of Cloudant/CouchDB. We use it extensively in our applications.

What you shouldn't use Cloudant for:
- Reporting, ad-hoc queries (indexing after the fact can be terribly slow)
- Data that frequently changes in a rolling fashion (like logging), especially if you need to keep your database small
- We typically recommend against using Cloudant Query/Search (I hear it's being deprecated (or deprioritized) in the Transaction Engine (CouchDB, foundationDB))

IBM Cloud Support and Implementation

Our team is quite experienced with the CouchDB/Cloudant technology so the original implementation went really smoothly. As usual with database implementation, the key is to design the data model and related view in the most future proof way so that you can avoid model and view changes.
  • Test the architecture on CouchDB helped us to address initial design flaws.
  • The migration to Cloudant as such was very painless.
  • We have migrate our replication system to Cloudant Android Sync for mobile devices.
  • We have regular informal contact with the Cloudant leadership to discuss our use cases and implementation strategies.
Advanced - Very good support for time critical operations. This has been really worth it for our teams.
At some point, the system needed a further full rebalancing of the data on the nodes due to some issues with some highly conflicted database (which was a user error). Cloudant support helped our team to scale the reindexing by add free-of-charge 3 additional server nodes to the system for the during of the incident.
  • Central Documentation and Learning Hub
  • IBM Cloud Docs
Very happy by the commitment given by the team which has been really good over the last 7 years of usage.

Using IBM Cloudant

10 - - Developers
- Operations
We don't have any engineering working specifically on the support. Of course some of the developers are involved with Cloudant on a day-to-day basis.
Skills required:
- Javascript (views), we use nano for the Node.js parts of the application
- Map/reduce concepts
- Document design.
- Java (we use Ektorp for the Java parts of the application)
  • Storing the business transactions performed offline
  • Storing usage metrics
  • Storing content configuration documents
  • Reporting on all of the above.
  • Distributed replications
  • Waiting on some really cool use cases for DashDB.
  • More reporting with DashDB.
  • More ad-hoc reporting.
  • Much more storage of data.
  • Automatic archiving of the database (e.g. like Amazon Glacier) would be a killer feature.

Evaluating IBM Cloudant and Competitors

  • Product Features
  • Prior Experience with the Product
  • Positive Sales Experience with the Vendor
Features, since we rely on CouchDB's interface. But also the gold support contract that gives us peace of mind and a dedicated support engineer in case of questions. We have been very happy with the support provided to our team.
No, we are happy with the service. The selection process was very simple since we were already using Cloudant through the Heroku connector (which was unexpectedly terminated, to be honest).

IBM Cloudant Training

Configuring IBM Cloudant

There are limited options available to the user (by design)
When you create a database, you can configure the number of internal shards.
Cloudant support can help with the advanced settings if needed.
- Avoid as much as possible the Cloudant Search
- Avoid large documents (prefer a lot of small documents).
- Use UUID as _id whenever possible to avoid naming conflicts (especially in distributed environments where you don't control this).
- Learn to write efficient views (instead of search)
- Learn about handling the 429 error code!
- You don't need to have replication of the data on 8 nodes by default (check out the Q parameter)
No - there is no facility to customize the interface
No - the product does not support adding custom code
Everything can be driven by API requests (even stuff like scheduling, creation of custom shards type), it's very rich and well documented.

Using IBM Cloudant

Because it's very intuitive, if you a familiar with the core CouchDB concepts.
Like to use
Relatively simple
Easy to use
Feel confident using
Requires technical support
Lots to learn
  • Editing the docs in the dashboard
  • Writing ad-hoc queries
  • Permissions management
  • The Dashboard is slow and takes time to load many databases.
  • Writing a new map/reduce view indexes the entire DB for every change. In Couchbase, you have a feature to create a view for a subset of the data, this is very convenient.

IBM Cloudant Reliability

In general, Cloudant feels very stable with large volumes of data (specifically with the multi-tenant setup). It allows us to scale our growth in the terms of transactions (summer months in our operation) without too many headaches.

- Pure response time speed is actually something to be improved.
- The performance of the Dashboard (especially with a lot of databases) needs improvement.

There are a few cases where we noticed issues: some replication jobs were not properly triggered (this is a long standing issue that we are dealing with with the support team).

Relationship with IBM

IBM Cloud Sales has been working with us to create an adequate pricing for the usage that we have which corresponds and scales with our growth. It's also not very aggressive and sales-y which is great.
CSM have also been helpful at times.
Also good support from the Sales team in terms of renewal and changing our subscription and support plans.
Monthly spending credits were slightly discounted compared to base list price.

IBM Cloudant Support

Quick Resolution
Good followup
Knowledgeable team
Problems get solved
Kept well informed
Immediate help available
Support understands my problem
Support cares about my success
Quick Initial Response
Escalation required
We have a gold support contract.
Yes - Yes, some deployments done a few months ago were related to some issues we raised.
Our account contact director Chris Snow is helping us to develop some extensions to Cloudant Sync during his free time.

IBM Cloudant Implementation

Change management was a minor issue with the implementation - We designed the system around the CouchDB architecture, so this was always part of the implementation since day one. Cloudant actually provided the "service" around the CouchDB technology.
  • Designing the documents to be scalable.
  • (Manually) sharding of the document on many databases was a key design decision for us (we shard per "day"), so that we can more replicate back and forth.
  • Designing performing views for large amounts of document (ad-hoc reporting) is very tricky.