Thursday Night Something-or-other

Thursday – 11 February 2016
Today was supposed to be #NoBadNewsThursday. It started that way. About a hour-and-a-half into it, however… not so much. One of the servers at work decided to try a new trick:

  • It was powered on.
  • It could see – and transmit to – the network.
  • NOTHING on the network could see it or any of the traffic it generated.

And, to make things even more interesting: This was tied to a production (as in “manufacturing”) system. So, it was “kind of” important.

To quote Dr. Sam Beckett: “Oh, boy…”

I – and a couple of network engineers – troubleshot it for about an hour before we came to the inescapable conclusion that it was not a network problem. Digging into things a little further, I saw that it was pending a reboot. That didn’t solve things. Next up: Check its updates queue. There were a few sitting there that needed to be run. So I let ’em fly. Reboot 2: Electric Boogaloo. Nothing. Well, that’s not entirely true. It started rolling back the updates. (That’s usually not a good sign.) I checked update error messages when it came back up and saw that they failed because the hard disk was full. As in “Zero. Bytes. Free.” There was a good bit of cursing on my part, to be honest. Then it was a dive into the storage array and VM systems to make space appear. Then present it to the server. That all worked. Then I tried the updates again. Lo and behold, they all went through with no problems. (There may or may not have been a few “Amens” and “Thank yous” that went up at this point.) And, almost automagically, networking traffic all worked again. I verified with one of the network guys and an end user that all was right with the world again… and it was. Selah.

Part of getting through the day included being taken to lunch by networking vendors. It was not only a good change of pace, but it brought my day back from the Precipice of Doom.

After work, I picked up Team DiVa from daycare. Home. Dinner and dessert. They even got in almost 20 minutes of Disney Infinity playtime before it was time to start bedtime prep. Selah.

Carve Away the Stone
I’ve made halting steps at getting back to the gym. Last night, I considered going… then I opted out, because it was damned cold outside. I did, however, still manage to get in a bit of a workout, courtesy of my dumbbells, Swiss ball, and a couple of suggestions from YouTube. Was it quite the same workout I would have gotten at the gym, no. But, I wasn’t just sitting on my duff doing nothing, either. I’ll call that a win.

Stray Toasters

Alright, that’s it for now. I think I’m going to catch up on a show or two before calling it a night.

Namaste.