Monday, February 04, 2008

Power for Algernon

Transformer rat outlineMaybe he was a rat with dreams...Maybe he came from the wilds of Arizona seeking a higher education. Maybe he was just cold...We don't really know.

What we do know is that a lot of people spent most of the weekend repairing the damage caused by one little rat that crawled his way into a transformer.


Now, the rat couldn't have known that this transformer powered many of ASU's computer systems and that, besides the small matter of frying his brains, he was going to cause a major service disruption. Fortunately, he sought shelter late on a Saturday night during the lowest period of computer use. We have a UPS but the battery ran out before our power problems were resolved. Technicians worked through the night to grab a generator, hook it up and then restore our equipment.


We were back to full power by 6:30 am Sunday, when the software and hardware people took over to restore services. Support services, such as Blackboard, Exchange and IDEAL, had to come up first so that they'd be available when the Web came back up. Most services were available by 11:30 am, though system functions needed to be reviewed throughout a very long day.


 I'd like to thank each and every person who helped get ASU's services back up and running.  Many thanks to:


Terry Hinton & his staff - Chris Beke, Robert Blackmon, Alexandro Bonos, Shawn Bryan, Mark Fornefeld, Kevin Lockart, Reyes Mori, James Prather & Brian Shaw; Jack Hsu’s staff - Chris Coffin, Brent Dunlock, Jim Durand, Jane Flores, Sean Garrett, Paul Harper, Bob Kaneshige, Joe McDonald, OJ Redhair, Jim Salverson, Jeff Scheib, & Greg Wilson; Ken Carl & his staff - James Muscara; Robin Manke-Cassidy & her staff - Gregory Alcorn, Zach Chamberlin, Erich van Sanford & Goska Zapolska; Dave McKee & his staff - Franco Lomonte, Tom Bauer & Duane Woerman; Kari Barlow & her staff - Nancy Lee, Nathan Gudmunson, Andy Beier & Minnie Fontes; John Rome & his staff - Sangeeta Agarwal, Ali Benmalek, Jack Davis, Daren Kahus, Dennis Monce & Bob Sookvong; Max Davis-Johnson & his staff - Walt Ellis, Neil Fritz, Jason Green, Ron Hill, Tasleema Lallmamode, Joe Nadeau & Lori Reents; Sarah Hughes & Samuel DiGangi & his staff - Ruvi Wijesuriya.


And what about Algernon? He may not have made it to grad school, but he did help us learn some things. We knew that we needed redundant power systems and Algernon helped make it a priority. Unfortunately, the more elaborate the infrastructure, the more expensive redundancy becomes. As we move toward a 24 by 7 world, the last couple of 9s are the most expensive. It's interesting to note that services supplied by our partners, Gmail from Google and PeopleSoft functions from CedarCrestone, were not affected by this power outage. Our little rat friend would have had to travel a long way to disrupt them, and due to their scales of operation, they may be a little more varmint-proof than we are.


Comments - even rat jokes - are welcome...

10 comments:

Nancy and Derwin,  February 4, 2008 at 12:16 PM  

Rat jokes? OK...

Q: How many rats does it take to change a transformer?
A: One, apparently.

:o)

Ron February 4, 2008 at 1:49 PM  

Rats by Dr Seuss
If your ratty gets all scatty 'cos his coat is getting matty
And he's acting kinda batty now he doesn't look so natty
If he starts to get all chatty and to voice pratty demands
Then beware! You've got a bratty tatty ratty on your hands.

If your doe has had a go with all her bro' bucks in a row
And you've seen her face aglow and her belly grow with woe
If she's getting glad and gloating and gargantuous in girth
Then at any time you guess that your grand girl is gonna birth

If your buck has had no luck getting his lazy butt unstuck
And you pluck him from his hammock to a chorus of "You suck!"
If his flaccid furry form is in a squishy slug disguise
Then it's right to get this roly rotund rat some exercise.

Jeromy McMahon February 5, 2008 at 2:09 AM  

WOW! And I thought this was a urban legend at first. So how much energy (watts) did the little guy take?

Alan Bradford February 5, 2008 at 3:28 AM  

I thought I smelled something down there on the Tempe campus...

If he truly was cold, at least we can be thankful that he warmed himself up with electricity and not by setting a fire.

Pat,  February 5, 2008 at 7:53 AM  

He was probably the one who got elected to create the sparks and fireworks for Chinese New Year (Feb. 6) which just happens to be welcoming in the "Year of the Rat"! He was just a few days early!

aomitche,  February 7, 2008 at 2:28 AM  

Today marks the lunar New Year. It's the Year of the Rat. Those born in the Year of the Rat are imaginative, charming and generous to those they love. Gung Hay Fat Choy! I guess Algernon did not like the way the year was starting out.

bentrussell February 7, 2008 at 8:10 AM  

And not to forget the distributed IT folks, who maintain their systems along side UTO systems in the data center, and who came out to bring their Colleges', Departments', etc 24x7 online services back to life this weekend. We are out here and work weekends too when duty (and the data center) calls us :-)

I'd like to thank the prompt action by Terry Hinton's staff on behalf of Herberger College IT with regard to notifying us in distributed IT of the situation and keeping us in the loop. Many thanks to everyone in ECA/ECB!

And of course a big thank you goes out to those in the Firewall and Network teams. Without them, we'd all be in the dark, no matter how many rats invade.

Guy Mullins February 7, 2008 at 2:58 PM  

One rebel Rat is not a King
Google us up to see a Phenomenal Thing

MR,  February 8, 2008 at 12:06 AM  

Fifty years ago we were afraid of bugs. Now, of rats. I'm afraid of what it may be in another 50 years...

Anne December 8, 2008 at 3:06 AM  

Wow, poor little rat. You should get your rats hammocks (no, we do not make them) then they would spend their free time snoozing instead of shorting our electrical systems.