Will 2021 be the year of governance?
Perspective on 2021 cloud trends
Year-end is a great time to connect with customers and get a feel for what their priorities are moving into the next calendar year. The common trends emerging from those conversations clearly paints a picture of large enterprises stepping-up to the next level of the cloud maturity model.
I see big shifts into optimization (e.g. focus on improving cost, visibility and manageability) of the cloud ecosystem, specialization of cloud roles, and an emerging challenge between the “best-of-breed” and “standardization” religions.
Don’t lament that our green fields have turned brown…
As the cutting-edge technology adopters have moved on to the next hype cycle it is now the governance professionals time to shine. There is a giant sea of usage: multiple clouds, dozens of services and hundreds of accounts ready for optimization! One customer told me that they committed to 10% cost savings for their 2021 cost-savings goal, but actually plan on delivering 10% each quarter (> 35% for the year) because it is such a target rich environment.
Even AWS is on the bandwagon. The headliners of this years’ re:Invent announcements were solid improvements to cost, manageability and usability of existing services; maybe that is not as exciting as someone driving a semi-trailer on stage, but you can’t argue that getting more polished services at a lower cost will put a smile on customer faces. Before leaving on holiday, I suggest spending a day or two to analyze the changes that have the potential for biggest impact on your cloud ecosystem and putting a plan in place to test and implement them in Q1. For example, I found three key areas of cost improvement for Turbot (both SaaS and Enterprise across v3 to v5).
New gp3 volume types are 10-20% savers for every gp2 workload and many high IOPS use-cases too.
New Amazon S3 Bucket Keys will end up saving us thousands in KMS charges.
The change from 100ms to 1ms billing granularity for AWS Lambda is saving us so much more than we expected based on our numbers:
Turbot uses a ton of AWS Lambda to run governance guardrails at massive scale across AWS, GCP & Azure. In just one of our dev environments, we have 2,000+ event driven Lambda functions that are invoked >42M times/month.
Our Lambda functions have an average duration of 600ms with a floor of 0.35ms and long tail out to several seconds, because the wide distribution we initially assumed an average savings of ~50ms per invocation or about 10% of our monthly bill. However, our actual billed usage is more interesting.
We see that the change was implemented on Dec 2 and our Lambda billed usage went from an average of $32.33/day all the way down to $6.73/day (savings of 79%). Clearly, a large number of very short function invocations (<50ms) were spiking our costs.
It is gems like these that will have your management team and finance team finally understanding the true cost savings and optimization opportunities you cloud strategy promised when it was fresh.
The rise of specialization…
It is abundantly clear to our customers that the days of being the “Cloud Guy” are over. The service landscape from the cloud providers is too broad for one person (or team) to be expert across everything. Amazon RDS has much more in common with Azure SQL and GCP Cloud SQL than it does with EC2 or S3, so does it makes sense to have specialists in cloud database technologies? Our customers think so, and the same holds true for other specialties as well. We see teams organizing around core service capabilities in 2021 and beyond. Key areas of emerging specialization:
Identity (separate from security): specialists in IAM focus on automation and integration with enterprise identity. Building processes to enforce segregation of duties, least privilege and periodic review of access.
Cloud finance/cost optimization: focused on billing dashboards, cost allocation and coordinating organization efforts to optimize spend.
Storage: Data retention, resiliency tiers, backups, archive, with multiple services across multiple clouds… this is a really deep area.
Database: It’s clear from Snowflake’s IPO that enterprise customers have a never-ending appetite for database technologies. In the past you might have had technology specific DB teams, but we see the specialization here across cloud-based database technologies (helping teams pick the right tools for the application pattern.)
Networking: I think 2021 will be the year of NetDevOps… We are seeing the simultaneous elimination of data center networks and the change in focus here from inbound network segregation/security to how to I protect from data leaving my network.
…and the coming war of standardization
As we specialize, we need more people in those roles and headcount is difficult to justify. This will lead some to try and make the case that they need to support fewer technologies. The basic thinking here is “if I standardize on a fewer number of technologies than I need fewer people to manage them”. It is a logical argument, easy to rally people around and easy explain to management, but ultimately it reduces IT agility. This is a mistake; reducing IT agility will have a direct and measurable impact to business agility as well.
We see leading organizations using the following strategies to avoid the pain of the standardization pendulum:
Justify headcount with cost savings. A huge part of the “experts” job should be in assessing what is being used to optimize spend. A good rubric to know if an area is worthy of justifying a specialist head “is the spending in this space at least 4x the cost of the specialist?”
Generalize the specialist. Having a cloud technology specialist that works across products in the same family makes more sense than specializing on specific service offerings (e.g. I would rather have a Cloud-Database Architect than an RDS Manager).
Proactively communicate your wins. Cloud is moving fast; you need to highlight that pace of innovation and what your team is delivering on. Show you are in control (and accelerating) by ensuring that you have a steady stream of technology and business wins to share every two weeks. By keeping initiatives small (two-week sprints) you will have 20+ wins in 2021 across every service team, while keeping morale and productivity high.
What are your focus areas for 2021? We would love to hear from you and include some of your insights in an upcoming case-study. If you are new to the Turbot newsletter and like this type of content please consider subscribing and sharing with colleagues.