Thursday, November 17, 2022

Be calm, be brave, it'll be okay

Sometimes, some days. I mean, just. Wow.

Some heavy shit comes down or bad things happen. This entire week has been a challenge. For me, for a lot of other friends, for my sister, for my bestie. For ... Twitter!? 

Like actual heartbreaking shit, financial shit, and workloads and struggles of an overwhelming perspective.

What the actual h*ck. 

Today at work we had a massive, and I am not even exaggerating, massive site outage. All week long things have been a mess here and there and everywhere but today was a collapse of epic proportions. 

At one point, there were 20 of us swarming to fix things, and discovering more things that were broken as we weeded through all the noise. 

We're still not out of the woods, and there are problems that can wait for tomorrow morning, but holy shit. This was so stressful. It's fun to work on a team when everyone can band together and just jump in to do All The Things, but you shouldn't have to do that. 

It should never have to happen. But hallelujah for the vast majority of my team.

This was human error. Someone did something (we do not know who) and well, it had direct and horrible repercussions everywhere on the live site.

I think of once upon a time, back in like 2016, when Jess worked somewhere and made an error that no one caught for six weeks. They got fired for it. Instead of "let's fix it and make sure this never happens again." No one noticed it for six WEEKS. 

What happened today was an immediate "noticed" event, and when I think about how they got fired for the thing they did vs. this? I'm kind of still stunned about the firing. 

Today was a full day of lost work for about 20 people on our staff and others on our vendor's team. As well as lost time for the clients who use our product. 

If you do the math on this, I'd love to know how much of a "cost" this was to the system. Staggering.

But. What do you do about something like this? Compared to when Jess made a mistake and got fired. 

I am not calling for anyone to be fired or anything wild like that, but honestly I feel like our vendor owes us something. Someone should take accountability. Somehow.

My old boss Stephanie at our office in Boston would say "did anyone die? Anyone bleeding? Nuclear war? No? Let's focus on fixing it and then never let it happen again." I channel her when I need it. I keep calm. I repeat the Guster line of "be calm, be brave, it'll be okay." 

We are not out of the woods. Like I said. At about 7am tomorrow they are running a script to fix another thing that isn't a show stopper for our clients but so help me God - if they break things more. I do not know what I will do.

I'm still fielding tickets. Answering emails. Ready for bed. I ate half my dinner (thank you Geoff for cooking) and I'm trying my best to just keep it together through to bedtime. I found a packet of cub 

Managed to get in some steps though, so that was good. 

What a day. 

Oh, and I will say, one of my coworkers (and friend!)  got some very incredible good news about the adoption they've been working on. So while a lot of things were breaking down all around us, I was so happy when he slacked me at about 5pm to let me know all they need now is a court date. 

So holy moly. Up and down and all over the place today. Oys and Joys! 

Effin Birds is one of my favorite things on Twitter, and I'll be sad if Twitter folds and dies. Here is one of today's posts, and oh my gosh if they didn't know exactly the right thing to say.

digits

exercise: Dedicated 10+7 

blood glucose
9:30am: 178
5pm: 177
10:30pm: 156

food:
coffee, water
9:30am - low carb fajita wrap (net carbs 5) with pb&j
2pm: Turkey and muenster on 647 bread with mayo
2:30pm: metformin 
5:30pm: 1/2 a chicken parm breast
7pm: 2 celery sticks and peanut butter
9pm: large bowl of Cub Scout popcorn
7pm-10pm, 3 vodka tonics, mostly ice.
10:30pm: vodka & diet/low sugar cranberry juice

2 comments:

  1. Eliz / Rockport10:41 AM

    [insert giant hug emoji here]

    ReplyDelete
  2. thank you. i needed a hug yesterday. and today. sigh.

    ReplyDelete