I work in what we call the “Commercial” Market, which has an amorphous and liberal definition of around 500-1500 employees thereabouts . . . ish . . . give or take? There may also be a certain dollar amount along with that, but that too, is amorphous, as it were.
Let’s just say that these companies don’t qualify as something one might consider an “Enterprise-sized” company. Many of these companies, like any other, want to “do technology right”. They are often not tech companies: maybe they are banks, or retailers, or manufacturers.
I have run into more than a few of these companies who want to, in their own words, “run their business like an Enterprise.” Whatever the case, they’re not FANG, so they usually need guidance on how to do this. And just so we’re covering all the bases here, this thing I call “running the business like an Enterprise” goes by many names: “Digital Transformation” or “IT Modernization”. There are at least a handful of buzzwords that overlap with this process, but hopefully you get the point by now.
For the rest of this post, I will be using the phrase “Digital Transformation” for the sake of consistency.
My goal in Part 1 here is more about attitudes and breaking perhaps long-held misconceptions you may have about Digital Transformation. It’s such a large and broad topic that if I just come out swinging, you’ll get bogged down with “Oh yeah? Well, what about . . . ?” questions, so take a deep breath and prepare to think differently about this. This is about something that costs you little to nothing: culture change.
I would be remiss if I didn’t at least make a small reference to my credentials here: I have spent the last 12 years at 2 different Enterprise companies as a Senior/Principal Automation Engineer with 1200+ ESXi Hosts and thousands of VMs. At the first I spent time maintaining a large IaaS Healthcare Cloud and the most recent was a large, nation-wide US retailer that, although wasn’t perfect, really did run their business like an Enterprise. And if you know any companies that are perfect, hit up my DMs. I have few rhetorical questions I would like to ask you.
My point is that I’ve been around the block a few times . . . it’s not my first rodeo . . . uh . . . I’m a bit of an “old dog” . . . I, um . . . wrote a few pages in that book . . .
Big, Puffy, Agnostic Clouds . . .
I would also be remiss if I didn’t address what Digital Transformation is not. It’s not “EHMERGERD! LETS JIST PUUT EVERTING IN TEH CLOWD!” I mean . . . it might be . . . for your company based on lengthy analyses. But, to think of the Public Cloud blindly as a panacea is soooo 2015.
We’ve learned a lot since then. And again, I am not saying that you aren’t a shop that can’t put everything in the Cloud and it fixes all the things. You have to figure that out for yourself.
Even amongst the most ardent fantheys of Clouds . . . the “Clouderatti” . . . the “Cloudigentsia” . . . hold a consensus that you should analyze your business needs and environment before doing anything so . . . transformative (get it?). I have some guidelines on how to do that in Part 2, but a deep dive on the lengthy process on how to do that would be another
blog post book altogether. And I make a living now helping people with the process . . .
Here, I am enticing you with what proper results look like, and I would never lie to you, dear reader: the process to get there is usually lengthy and ugly.
Anyway, the “real talk” take about Digital Transformation is that you will need to analyze your current state architecture and process needs and align them with your business needs to figure it all out.
There are a host of reasons why a company would not be able to go to a public cloud: data gravity, regulatory compliance, cost, legitimate security concerns, and so on.
I would implore you to start thinking agnostically about where things run. Try to overcome your biases and think about what’s best for your business and realize that if you have a datacenter now, to start, all options should be on the table: entertain the possibility that maybe, just maybe, your datacenter could be run and managed like a cloud.
If that sounds obvious to you, then kudos, you’re ahead of the game. The best attitude to have is that you don’t care where it runs; you care about what is best for the business. And if it’s best to run it on your own datacenter, then let the chips will fall where they may. If it’s best to run in a Public Cloud, the chips can fall there too.
Culture Change Requirements: The List
There are two must-reads I recommend to everyone:
- The Phoenix Project (I did a review of its sequel here) – This is standard reading for anyone trying to run a more efficient enterprise.
- Site Reliability Engineering (O’Reilly Press) – You don’t have to be an SRE shop to implement a lot of the principles and practices in that book. There are some technical pieces, but this can be read by you “higher ups”.
I am listing items here from my own experiences, but also from those two books, aforementioned. These approaches should be “baked into the cake,” meaning that there should be no debate or argumentation or “weighing of the costs.” Everyone moves to the same beat:
- Embrace a No-Blame Culture. If there is a problem, your first and only question should be, “How do we resolve it?” After that, the second question is, “How do we prevent this in the future?”
And you should never ask “OK, who’s fault is this?”
Good engineers in a no-blame culture are more than happy to admit fault if they realize that they won’t be punished for it. Also, logic would dictate that what goes around comes around. I’ve always given the side-eye to people who point the finger when it’s not their fault. Then sure-enough, they eventually screw up and get all defensive. What are you, 12? Adults solve problems, not blame or punish. Needless to say, Root Cause Analysis (RCA) calls should be devoid of finger-pointing.
- Embrace calculated risks. The key word here is “calculated”. Been afraid to upgrade that Cluster that’s been sitting there running and don’t want to disturb it? Let someone take the calculated risk of upgrading it with a plan. The same goes for new projects like automation or right-sizing, or using previously un-leveraged features. In fact, encourage them to do so.
- Declare Standards. Everywhere.
- Have a laser beam focus on removing barriers to execution. Whether by implementing technology or political maneuverings, anytime someone says, “We can’t do that because we’re waiting for . . .” it should be investigated and eliminated immediately and for good.
- Have a passionate focus on Documentation and on stamping out tribal knowledge. In fact, one’s job security depends on disseminating information about what one does, not hoarding it. Handle your Brents.
- Where possible, releases should be iterative and cadenced. Even with IaC, encourage changes to be incremental to keep things moving.
- On call should be efficient and devoid of pesky and nervous leaders (the aforementioned SRE book helps here). Are you the micro-managing type who has to call in and say stupid shit like, “You guys need to get things back online ASAP!” Right. We’re very well aware of that, douchebag; it’s a Sev 1 incident and you aren’t helping. Just keep your mouth shut and you can help by ordering food for everyone. You’ll be the first to know once everything is back up and running, dork.
- Business Continuity (however you define it) should be baked into the cake. Pro-tip – every person in the IT organization should know how much it costs the company per hour if services are down. You should not be balking at getting two (or more) of something if it means a single point of failure will, like, you know, cost the company money and stuff. Think of it as insurance: you hope to never have to use it, but you will sleep better at night knowing it’s there.
This is not just a technological thing, it’s a human thing. If the one person who knows how to fix the Firewall is on vacation and no one else knows how to fix it, you are, and forgive my French . . . totally fucked. You shouldn’t be calling people on vacation; you should have multiple people who know a thing.
- Hire good people and pay them well. I will be bring this up later, but good organizations find a way to root out fear and doubt and anxiety about where one works.
- Ensure you are planning architecture using best-of-breed and using recommended implementations. No matter whether you build or buy, ensure that your people are getting the help they need to design everything properly. Things need to be designed for scale everywhere. Get professional services and get everything peer reviewed to ensure that you are going by a recommended implementation.
- The real purpose of automation is to automate away toil and allow your engineers to spend time on innovating the infrastructure. A
wordrant about that follows here:
Word Rant About the Purpose of Automation
Those of you who are long time readers of my blog are very well aware that automation is my jam. And far too often I run into customers who view automation as a way to, “decrease head count”.
This approach is synonymous with the whole, “automate your job away” approach, which I find . . . to put it lightly . . . problematic.
First of all, it’s definitely a morale-killer. You are literally disincentivising (it’s a word) your engineers from making, well, anything more efficient. Many times you’re hiring hands-on Engineer Bobs who are there to automate people’s jobs away and it turns things hostile in a hurry. The numbers may look good on paper in the short term, but as it turns out, human beings really hate being treated, you know, like a number.
Second of all, it’s not a great way to attract and retain good people. Word gets around. And anyone worth hiring is also interviewing you. High turnover is a red flag. Talking about “getting rid of head count” also qualifies. You remember that old adage (and what I said earlier!) about hiring good people and treating them well? Yeah. Do that.
Third of all, and I am not saying this one’s easy, but it’s far better to create a culture of automation. I’ve already blogged about this.
You should start thinking of automation as, insert you’ve-already-said-it-before eyeroll here . . . “baked into the cake”. It should be assumed that everything will be automated, when it makes sense, from Day 0, rather than it being an afterthought.
See you next time.