DevOps Architect for Containerization Platforms and Microservices
Chose Longhorn Block Storage
GlusterFS was first Persistent Storage solution used in our Kubernetes-based clusters. It is file-based what in some usages led us to many data corruptions. CEPH is object-based persistent storage which can be used as file-based Persistent Storage in Kubernetes. It is also is …
Longhorn is performing well as storage for databases and in almost any solution that uses exclusive access to volumes (ReadWriteOnce in Kubernetes nomenclature). When write access is required from many clients (ReadWriteMany) Longhorn Block Storage covers its volumes with NFS (file-based) access. Longhorn Block Storage also is well fitted in every architecture where data security (snapshots, backups, multiple replicas) is more important than access speed (in terms on IOPS and MiB/s).
GFS is well suited for DEVOPS type environments where organizations prefer to invest in servers and DAS (direct attached storage) versus purchasing storage solutions/appliances. GFS allows organizations to scale their storage capacity at a fraction of the price using DAS HDDs versus committing to purchase licenses and hardware from a dedicated storage manufacturer (e.g. NetApp, Dell/EMC, HP, etc.).
Scales; bricks can be easily added to increase storage capacity
Performs; I/O is spread across multiple spindles (HDDs), thereby increasing read and write performance
Integrates well with RHEL/CentOS 7; if your organization is using RHEL 7, Gluster (GFS) integrates extremely well with that baseline, especially since it's come under the Red Hat portfolio of tools.
ReadWriteMany Longhorn volumes are still using NFS (file-based) protocol in the core.
Using iSCSI as main protocol instead of FC ties Longhorn to Ethernet-based LAN which is in most architectures much slower that FC-based SAN.
Longhorn could implement S3 as alternative access protocol to its volumes.
Backups, and snapshots configuration could be configured at each volume-level by administrators (maybe from additional CRD object?), because currently is configured at storage-class level which is not granular enough.
Documentation; using readthedocs demonstrates that the Gluster project isn't always kept up-to-date as far as documentation is concerned. Many of the guides are for previous versions of the product and can be cumbersome to follow at times.
Self-healing; our use of GFS required the administrator to trigger an auto-heal operation manually whenever bricks were added/removed from the pool. This would be a great feature to incorporate using autonomous self-healing whenever a brick is added/removed from the pool.
Performance metrics are scarce; our team received feedback that online RDBMS transactions did not perform well on distributed file systems (such as GFS), however this could not be substantiated via any online research or white papers.
Longhorn is mature software defined storage solution that is still developed and receive new functionalities. From the beginning every Longhorn volume have multiple (at least two) replicas, can leverage manual or automatic snapshots and backup to external S3 volume. Longhorn provides nice and clear GUI for administrators, but also can be managed from CLI.
GlusterFS was first Persistent Storage solution used in our Kubernetes-based clusters. It is file-based what in some usages led us to many data corruptions. CEPH is object-based persistent storage which can be used as file-based Persistent Storage in Kubernetes. It is also is much more resource-hungry than other solutions including Longhorn. Dell PowerScale (or Isilon) is a hardware-software solution, that provides volumes that can be accessed by file-based NFS and CIFS protocols. Recently was added access to its volumes with object-based S3 protocol. Longhorn is in the middle. It is block-based, it is build on industry standards like iSCSI, performs very well on 10Gbit or faster LAN and commodity hardware (or in virtual machines)
Gluster is a lot lower cost than the storage industry leaders. However, NetApp and Dell/EMC's product documentation is (IMHO) more mature and hardened against usage in operational scenarios and environments. Using Gluster avoids "vendor lock-in" from the perspective on now having to purchase dedicated hardware and licenses to run it. Albeit, should an organization choose to pay for support for Gluster, they would be paying licensing costs to Red Hat instead of NetApp, Dell, EMC, HP, or VMware. It could be assumed, however, that if an organization wanted to use Gluster, that they were already a Linux shop and potentially already paying Red Hat or Canonical (Debian) for product support, thereby the use of GFS would be a nominal cost adder from a maintenance/training perspective.
