Clustering and Load Balancing Hell Ride

First and foremost, I don\’t like to mention company name(s) or person related to this resolved issue because it was already settled. This is the reason why I was not able to update my blog for the past three days. Now I\’m posting this so share my experience and I hope you\’ll be able to learn something from my mistakes.

Within my server clustering-and-load-balancing-hell-ride days I have been tormented with different problems involving 3 clustered and load balanced servers. I am supervising these servers and to my surprise, the solution that we bought didn\’t have the features we look after a clustered system. Initially, during our presales with our datacenter, these features were mentioned: (1) uploading files would be easy – upload once, distribute to many servers, (2) servers would require little modification to run and (3) detailed documentation would be supplied. We didn\’t have the enough information in hand that\’s why we only had a limited idea on how it should work.

We trusted that everything will be ready when the servers were released to us; however, it seems that the servers were not yet completed by the time they ware released to us. Disappointed, we waited until the admins fix the cluster and so, after 3 week of standing by, we got the servers up and running. Then again, the servers were plague by intermittent ups and downs. We were really frustrated and complained to the point that we thought of getting our money back. After all, the diagnostic shows that the rsync program causes Apache to reload again and again to synchronize both servers. That\’s really not my forte so… hell I know! The reason we bought this clustering solution is to free ourselves from the complexity of clustering and load balancing, and so this should not be our problem. We just want the solution to work and that\’s all we want to get our expectation matched.

After all the negotiation, one of our hardware was replaced with RAID solution to accommodate the growing database. Now the SQL server runs independently now. On previous cluster design SQL server was already synchronized with the static content, therefore mixing the dynamic data, resulting to a lot of inconsistencies. Now that I know it already it\’s really ridiculous to begin with. Right now the clusters are already fixed, but we are facing another phase of testing and so I hope that everything goes smoothly as planned.

To know more about load balancing and clustering, refer to these links
Load Balancing

By Marck V.

Filipino IT consultant on enterprise software. On his spare time he do web project management, photography and blogging. Web 2.0 enthusiast.