It’s amazing the havoc a brownout can cause. Just before noon on Monday our building had a brownout that lasted all of a second—the lights never made it completely off before they were back on again—but it was enough to stop all work since then. We have just over 100 workstations, but each workstation is literally composed of multiple computers such that the total number of machines is multiplied by five. The server room is of a similar composition. We managed to get all but a couple of the workstations back up and running in short order, but the massive database that coordinates everything on the servers was badly corrupted. They’ve been working on rebuilding it ever since. They sent the first shift home and have canceled every shift since then. I went in yesterday on the off-chance we’d be back up and running in time for afternoon shift, but it didn’t happen so they told me to plan on staying home today. Which I am. And possibly tomorrow too.
The thing that amazes me is that this could have been prevented with a decent UPS system in place, but they don’t have one. At the very least I’d expect them to have one in the server room, but the company doesn’t want to spend the money. We could have been back up and running within 20 minutes of the original brownout had we had a UPS in the server room as the database corruption has been the source of our downtime. Instead we’ve lost two and a half days worth of work.
Sometimes I just don’t understand how upper management in even the brightest companies think.