Reliable data storage and access is a big challenge in AI applications and IBM ESS is a service you need to solve the problem. If you are building an AI service and you already use the IBM ecosystem for AI compute requirements, ESS can serve as an excellent software based storage solution. Integration with other platforms such as EMR/Databricks is still a challenge and should be improved.
For companies that have 4-5 or more servers, StarWind is a great solution to provide high availability for a low price. It can scale up very nicely for larger implementations to more like 15-20 servers with ease. Although we haven't used it for even larger environments, it's doable. What its not appropriate are really small locations that have 1-2 servers. Unfortunately no one seems to have a good HCC solution in this space.
the StarWind Virtual SAN allows us to use Starwind Management Console to confirm health of the sync
The High Available nature of the deployment means we can fail over VMs without end users noticing any downtime
Their support is proactive alerting when firmware updates are needed (including iDRAC firmware) or when there are any warnings in the event logs, and schedule a time to remediate the issue with you.
Complexity: StarWind Virtual SAN can be complex to set up and manage, especially for organizations with limited IT resources
Limited protocol support: StarWind Virtual SAN supports a limited number of protocols, primarily iSCSI and SMB3
Limited scalability for storage-only deployments: In storage-only deployments, StarWind Virtual SAN has a limit of 32 nodes per cluster
Lack of reporting and analytics: StarWind Virtual SAN has limited reporting and analytics capabilities, which can make it difficult to monitor and troubleshoot performance issues
Limited backup and recovery options: While StarWind Virtual SAN provides some basic backup and recovery options, it lacks advanced features such as snapshot management, backup scheduling, and offsite replication
StarWind Virtual SAN is a great solution and is now an integral part of our network of servers. The product is superb and the support has been amazing. It's perfect for our organisation and we won't be looking to come away from it any time soon!
Overall I like the usability of StarWind Virtual SAN because it is a "Set-up and forget" software. Once you correctly have set up the parameters, StarWind Virtual SAN pretty much rolls by itself. The biggest fact that one needs to keep in mind, though, is that the licensing for StarWind Management Console needs to be purchased separately, and while managing StarWind Virtual SAN through the paid Management Console is really easy and is well documented, going the free or - in other words - PowerShell Template route can be taxing if you are not that deep into the topic. You need to be especially careful with it if you switch from paid to free because using the templates incorrectly can cause issues.
Their support team is dedicated to providing top-notch customer service and is always available to help with any questions or issues that may arise. Their expertise and responsiveness have proven invaluable in ensuring the smooth operation of our virtualized environment. With such excellent support, we feel confident in our ability to utilize this product to its fullest potential, and we highly recommend it to others.
Overall the setup was easy, we did require some help from the technical support team but other than that, we followed all of StarWinds prerequisites and everything else just fell nicely into place with hardly any downtime. The downtime was only due to moving VMs from our previous cluster over to the new StarWind storage cluster.
IBM ESS is optimized for AI and Big Data usecases while S3 is a general purpose storage solutions. EMR and Databricks have lakehouse/data warehousing solutions for distributed computing but are more optimized for just the big data pipelining solutions and not essentially for AI usecases, especially for inference, when you need to load model artifacts really quickly.
We have found the solution surprisingly simple to use. The management console allows us to monitor the solution and we have configured email alerts to alert us about critical issues. These alerts have been proven to work in an actual failure scenario, for example, when we had a memory issue with one of our servers that caused the entire server to crash. The management console also allows us to monitor the solution performance and provides us with access to system logs.
