Clusterin’ clusterin’ Yeah!
by Nick on Oct.21, 2011, under Administration, Cloud, E-Mail, News, Software, Virtualization
So I did a little bit of load testing on my new web cluster.
Not bad for not having a real load balancer…
This is a four host (with one NFS/MySQL host) against a WordPress Page (this one, in fact) Lots of Caching is enabled, but it’s using fcgid for rendering speed, not capacity. My first host took the brunt of the memory stress for some reason. Perhaps internal connections caused it to be the caching whore.
I suspect some of the trouble is the “stickiness” persistence method of pfSense. If it were pure Round Robin load balancing, I think things would look a lot smoother.
The mail cluster is going well too. Mail is a lot more… finicky, especially when you’re doing spam filtering and greylisting on the same box. I don’t expect that to be as scalable, but it’s working pretty well. If I had a few more resources, I’d break my servers into at least another tier, my ideal setup is something like this: Frontend->Spam filtering-> SMTP out/Delivery with separate POP/IMAP… But that’s not going to happen with my little (well, medium sized) VDC. Unfortunately, with mail, you have to take a wait and see approach.
So yeah, why clusters?
Well, one, because they can handle more load than a single big server. No matter how beefy you make it, a single LAMP server is going to crap out at around 250 simultaneous connections. Granted, that’s a lot of load, but still. Two, if I take down a server for updates, I don’t take down my website. Three, clusters are a good way to take advantage of a virtualized environment. Each of my servers has the advantage of having a big NAS behind it, and can move from host to host, making my cluster a lot more nimble, and available, than a VPS slice, Container, or Jail residing in a bigger OS.
Clusters are the way to go. Sorry Calypso…
UPDATE:
Switched the cluster to apache2-mpm-worker. Here are the results!

