Since the repair folks didn't put the order in soon enough the part for the cooling unit didn't arrive today. So we decided to go to plan B.
Plan B entailed rolls the racks the most critical servers were in out into the hall. We then run a power cord to them from a electrical circuit that wasn't limited by a thermal cutoff switch (the only I think other than the cooling unit). We then hoped a breaker wouldn't blow and the servers wouldn't melt. This was around 11:30am when we go them back online.
We took turns watching them. While we doubted anyone would walk off with the servers (heck, it would take about 4 brawny folks to waddle off with them) we didn't want anyone yanking some hard drives. I love how I've become more cynical in the last year as a sysadmin. :-)
Those Sun E450's do throw the heat. They didn't raise the hallway temperature much past 80 degrees F. Leaving in the room would have hit the magic 80 degree mark for the cutoff.
We then rolled the big boys back inside around 4pm. No one even contemplated trying to do an around the clock thing. Of course, we'll repeat this process early tomorrow morning and hope that the part gets in early and actually fixes the problem.
Of course, now we need to figure out how to keep this from being a regular thing. Fun.
