Our server room’s battery backup consists of a couple of off-the-shelf APC battery backup units, running an ever-expanding collection of servers (about six, depending on what you term a “server”). Not the optimal solution, but a cost-effective one that was good enough when the server count was lower.
It’s been so good, in fact, that a very long time has past since the runtime on the battery backups was tested. Today, the test was unintentional.
Fortunately, since server room cooling has become an issue with such an enclosed space being filled with more and more machines, we are finally installing a cooling unit specifically to keep the server room cool. A big improvement over walking into the server room and starting to sweat almost immediately, to be sure! However, installing the cooling unit required turning off the power to the server room for a little while. It was off for a few minutes before I headed up to our all-staff meeting this afternoon, but it was back on before I went to the meeting and the battery backups held just fine. I knew it would need to be off for a little longer during the meeting, so I hoped the batteries would hold out. They didn’t. When you can’t connect to the Exchange server, or even get a new IP via DHCP over wireless, something’s up. Or, down rather.
I still don’t know how long the battery backups lasted exactly, as everything was already back on when I made it downstairs. Reboot everything in the right order, and half-an-hour later you’d never know anything had happened. And with everyone in the staff meeting, I was able to warn them before the meeting broke that I would need to work on the servers for a little while and not to expect it to be operational when they went back downstairs.
It worked out all right in the end, but it’s something I need to address and haven’t had the time or resources available. Nothing like a little priority-setting all done up nice for you :-)
Anyone have recommendations about on how you go about battery backup selection? I took the new building opportunity when replacing the core network switches to purchase a Tripp Lite rack-mount UPS unit for each of our three network closets, which so far have worked admirably, were cheaper than comparable APC brand units, and held the network rack up even through this same power outage.
Does it make sense to buy a smaller off-the-shelf UPS for each server, or each pair of servers perhaps, or to purchase one larger unit that can handle everything, even with the sticker-shock price tag? (Granted, several smaller units do add up themselves.) I have a feeling I know, but I’d be interested in feedback.
One of the best things I did at CLC was install an APC Smart UPS (XL 3000, I think) with a network management card. It’s definitely not cheap – $1800 or so – but there are lower end models that might be fine for you. The network card in the UPS signals all connected servers to shutdown in the event of a power failure. Graceful, unattended shutdown – very important, especially if you’re running Exchange!
Do you have a natural gas generator for the building? If so, do you have generator circuits in your server room?
If only! No generators…I was lucky when they expanded the server room in the last year that they ran a dedicated circuit (I think) for the servers. And the battery backups are (currently) the $150 units from the local warehouse club across the street…two of them for all the servers.
Wow, that stinks. I can understand the cost issue for a church (or any SMB for that matter). However, I think it is crucial for any organization. Perhaps on the back of a building-wide generator. You would want your congregation to have a place to go in an emergency, right? That would be my selling point. The continuity of power and data connectivity are extremely important as a rule and as a ministry. :)
I do agree that a building-wide generator would be an excellent idea to have. I’m not sure it would actually be as helpful in an emergency to the congregation, which is actually spread out pretty widely (by distance, although there are a lot of close members as well), as it would be to the local community since we’re surrounded by literally thousands of apartments. But it’s not likely something we can stomach in the budget right now; we just came off a new building addition campaign and finances are very tight. We’re only purchasing necessities.
Right now, I’d call a generator a luxury. We do need to fix the battery backup issue with the servers, I’d all that a necessity. If we do that, and have it set up so the UPS is set to shut down the servers gracefully without risking data loss, that’s going to be good enough. Exchange will still be there; everyone’s still got an offline copy in their .ost, and our spam filtering company will queue email while our server is unreachable (and they’ve told me they can set up webmail for emergency access if our server is down for an extended period of time). We would have trouble functioning normally without the servers running, but unless there was a catastrophe beyond the power being out, we’d wait it out and move on.
I do see the benefits to off site backup in a recovery situation, which is why I’m investigating Mozy and MozyPro right now, on Jason Powell’s suggestion.
A generator is something that might be worth looking at again in a few years when we may have the resources to do it, and perhaps a larger infrastructure to support, assuming congregation and ministry growth over that time frame.
At AAfter Search we have real problem with the off-the-self battery backups. They are un-reliable.
Apparently, their voltage waveform is not good enough to satisfy the servers. Hence, if there is an interruption, the server restarts even though the UPS passes all self-tests. Any suggestion?