We have been doing a ton of operational work lately in order to achieve our SOC2 certification. One of the things our team automated was compliance scans for various cloud service configurations. The Zoom CIS benchmark was especially challenging. We decided to open source it for others to leverage and build upon: https://hub.steampipe.io/mods/turbot/zoom_compliance

You can run the scan from your local desktop using a recently released feature of Steampipe called mods. There are currently compliance mods for AWS, Azure, GCP and Zoom built by the open source Steampipe community. The output of the mod looks like this:

The compliance mod includes the full CIS benchmark for Zoom (100+ automated checks). I thought it might be helpful to channel David Letterman and boil it down to the 10 most important controls that should be monitored, burt first a shameless plug:

If you’re intrigued the idea of building automated governance controls using SQL, I will be doing a talk at the AWS Fest 2021 on June 22nd.

I will be discussing the pros and cons of industry benchmarks vs custom controls, and doing a demo and some live coding to show how easy it is to get started.

Top 10 Zoom Governance Controls

One of the most important things to understand with Zoom is that meeting hosts have tremendous autonomy in managing meeting settings. Many governance controls in Zoom simply change default settings, but allow host to override them. For the settings in this list we recommend that you ensure the settings are ‘locked’ in your organizations configuration so only administrators can change them.

1. Require use of passcodes and passwords

Ensure the organization's account settings require passcodes.

• Require a passcode for personal meetings — A common exploit across conferencing platforms is the reuse of an individual's personal meeting. Requiring a passcode to join a personal meeting can reduce the likelihood of 3rd party reuse.
  

• Require password for participants joining by phone — To prevent unauthorized third parties from joining scheduled meetings via phone, this setting will require meeting attendees that join by phone to authenticate with a personal passcode.
  

• Require passcode to access shared cloud recordings — If you allow hosts to share cloud recordings of their meetings, use this setting to require a password to view/download the recording.

2. Ensure adequate passcode & password complexity

There are multiple use cases within Zoom for usage of passcode/password secrets. Each has independent settings for ensuring appropriate complexity rules.

• Meeting passcodes: — Meeting passcodes are distinct from user passwords, they are a shared secret that indicates the user has been invited to a specific meeting. This key control prevents unwanted attendees from guessing meeting IDs and entering meetings they weren't invited to (a.k.a. Zoom bombing).

  Given that passcodes frequently show up in meeting invites, and that calendars are often visible/shared within an organization, we recommend that complexity factors be set high to encourage use of random meeting passcodes and prevent use of a meaningful pass phrases that might be a used in a password reuse attack.
  

• Recording download passcodes — If your organization allows sharing of cloud hosted meeting recordings, those downloads should be secured by a complex random password.
  

• User passwords — Most organizations will want to enforce single sign-on for Zoom users. With single sign-on enabled, the only users with passwords should be break-glass users and administrators with elevated access. Ensure that your user password complexity factors meets your organizations requirement for administrative secrets.

3. Ensure only authenticated users can join meetings

Ensure meeting hosts are not able to create publicly accessible meetings.

4. Enforce use of waiting rooms

Waiting rooms give hosts the ability to screen users before allowing them into a meeting. This setting can prevent unintended sharing of sensitive information (e.g. an unexpected user entering the meeting while sensitive content is being shared).

5. Prevent content sharing

Prevent exfiltration of sensitive information by limiting how hosts and users can share content.

• Ensure screen capture of messaging is set to disabled — Check that settings prevent users from taking screenshots of direct messages or group conversations.
  

• Display "Zoom Meeting" in place of meeting subject — Hosts can inadvertently disclose private information in a meeting title (e.g. A patient's name in a health care setting or an acquisition target as part of a M&A). This setting replaces the meeting title with "Zoom Meeting" so the actual meeting title isn't visble to anyone looking over the shoulder of an attendee (think of a crowded Starbucks).
  

• Closed captioning: Ensure save captions is set to disabled — Zoom has a closed captioning feature which is likely enabled (and important) for users with hearing impairment. This setting prevents the resulting "transcript" from being saved which could be both a way to exfiltrate data and a potential legal discovery issue for the organization.

6. Enable watermarking features

Zoom’s watermarking features add "invisible" digital data to images, video and audio within Zoom that can be used in an investigation to identify a user who inappropriately shared private or sensitive information outside your organization.

7. Add timestamps to recordings

Ensures that timestamps are added to recordings to help with forensic investigation of any security/data loss incidents.

8. Ensure only local regions are enabled

Many organizations have data sovereignty requirements that require that company information and PII of customers and employees do not leave specific regions. Check these settings to ensure that only specific local regions are used when hosting your Zoom meetings.

9. Notify host when a meeting is scheduled

If someone were to compromise a host account, this setting would at least ensure that the host will be notified if an unauthorized 3rd party scheduled a meeting using their account.

10. Ensure only approved integrations are enabled

Ensure that the organization specifically reviews and approves any available 3rd party integrations before allowing them to be activated by Zoom users.

References:

• Steampipe Zoom Compliance Mod — Turbot

• CIS Benchmark for Zoom — CIS

• Best Practices for Securing Your Zoom Meetings — Zoom

• Zoom Security Features — Cornell IT

Steampipe, a new open-source tool from Turbot allows you to ask interesting questions of your cloud, one query at time.

Which users have MFA enabled right now?

 
 select 
   user_id, 
   name,
   mfa_enabled
 from 
   github_repo_user;
                  

What security groups are open to the world?

 select 
   group_name,
   group_id
 from
   aws_vpc_security_group_rule 
 where 
   type = 'ingress'
   and cidr_ip = '0.0.0.0/0';
  

Which resources aren't tagged correctly?

 select 
   id,
   name
 from
   azure_compute_image
 where
   tags -> 'owner' is null or
   tags -> 'app_id' is null;
 

What storage volumes are not in use?

 select 
   volume_id,
   volume_type
 from 
   aws_ebs_volume
 where
   attachments is null;
  

What IAM Users have inline policies attached?

  select
    user_id,
    name
  from
    aws_iam_user
  where 
    inline_policies is not null;
 

Built for Cloud Professionals

At Turbot many of our day-to-day tasks require us to work with cloud resource metadata, sometimes as a basis for a search or filter, but also to combine with or enrich other data sources.

The existing tools we used to answer these questions were a hodgepodge of web-based consoles, CLIs, APIs, and SDKs, cobbled together with custom python or bash scripts. None of which has any level of consistency across clouds or even in a single cloud.

We needed better tooling to get our own cloud work done, so we built something to scratch that itch… and from the very first query it blew our minds. There is something very satisfying about seeing your cloud resources act like the database we always knew they were; it feels like a magic trick. Once we had it in our hands we were sure that we wanted to share it with the world, and the Steampipe open source project was born.

How does it work?

One of the goals of Steampipe was to make the first experience with it as simple as possible; we wanted zero barriers for people to download and run their first query. A lot of engineering went into how to elegantly install the dependencies and provide a first class interface to current and future plugins.

The simplicity of the user experience masks the complexity and depth of the architecture. Underlying Steampipe are three distinct software components: a CLI executable, plugin modules and a lightweight PostgreSQL background service. Data is not persisted in Postgres, instead Steampipe uses the PostgreSQL Foreign Data Wrapper to present external data as database tables. The foreign data wrapper does not directly interface with external systems, but instead relies on plugins to return data in a standard format. This approach simplifies the work needed to extend Steampipe, because PostgreSQL specific logic is encapsulated in the foreign data wrapper, while API and service-specific code resides only in the plugin.

Where do we go from here?

One of the fun things that happens when you use Steampipe every day, is that you start to wish that everything was SQL enabled.

Memes aside, we are finding that we do love using Steampipe to answer real world questions outside of the public cloud space, and those ideas are making their way into the product too. For example, our customer success group was struggling to produce simple aging reports from our Zendesk platform using the built-in reporting tools.

Today, we have an automated job that runs a query against Zendesk using Steampipe and posts it to Slack every morning; It sets the focus for the day for the entire team.

  select
    date_part('day', now() - t.created_at) as age,
    t.id,
    t.status,
    u.name as agent,
    o.name as organization,
  from
    zendesk_ticket as t,
    zendesk_user as u,
    zendesk_organization as o
  where
    t.assignee_id = u.id
  and
    t.organization_id = o.id
  and
    t.status in ('open', 'pending', 'hold');

  +-----+-----+---------+----------+------------------+
  | AGE | ID  | STATUS  | AGENT    | ORGANIZATION     |
  +-----+-----+---------+----------+------------------+
  | 43  | 373 | hold    | KKapoor  | Haymont Tires    | 
  | 41  | 374 | hold    | JHalpert | Tract Industries | 
  | 40  | 375 | hold    | KKapoor  | Dunmore HS       |
  | 22  | 376 | hold    | KKapoor  | East PA Seminary |
  | 14  | 418 | pending | JHalpert | Capital One      |
  | 5   | 435 | pending | DSchrute | Mr. Rammel       |
  | 2   | 437 | pending | JHalpert | Steamtown Mall   |
  | 0   | 439 | open    | SHudson  | Stone & Son Suit |
  | 0   | 440 | hold    | JHalpert | Larry Meyers     |
  | 0   | 441 | pending | Sudson   | Blue Cross       |
  +-----+-----+---------+----------+------------------+
 

As I write this, only 2 weeks after launch, there are 11 plugins with ~200 tables published to the Steampipe Hub: https://hub.steampipe.io, including AWS, Azure, GCP, Slack, Zendesk, Github and most recently Digital Ocean. We fully intend to keep expanding the coverage, scratching any itches we find, but we are even more excited to see where the open source community takes the project and what APIs they connect Steampipe to.

Bonus: Get started today and get some free swag!

That’s right, take 2 minutes out of your day to install Steampipe and when you run your first query let us know about it! Tweet your first query to @turbothq, or head over to the Steampipe message boards, and post some feedback there on your “First Query” (Can you tell we are excited about this?) So excited in fact, that we will handsomely reward everyone that posts #firstquery in February with Turbot and Steampipe laptop sticker swag!

Getting Started Resources

• Steampipe Homepage: https://steampipe.io

• Download/Install Guide: https://steampipe.io/downloads

• Documentation: https://steampipe.io/docs

• Open Source Repo: https://github.com/turbot/steampipe

• Discussion Board: https://github.com/turbot/steampipe/discussions

• Plugin Registry: https://hub.steampipe.io

I see big shifts into optimization (e.g. focus on improving cost, visibility and manageability) of the cloud ecosystem, specialization of cloud roles, and an emerging challenge between the “best-of-breed” and “standardization” religions.

Don’t lament that our green fields have turned brown…

As the cutting-edge technology adopters have moved on to the next hype cycle it is now the governance professionals time to shine.  There is a giant sea of usage: multiple clouds, dozens of services and hundreds of accounts ready for optimization! One customer told me that they committed to 10% cost savings for their 2021 cost-savings goal, but actually plan on delivering 10% each quarter (> 35% for the year) because it is such a target rich environment.

Even AWS is on the bandwagon. The headliners of this years’ re:Invent announcements were solid improvements to cost, manageability and usability of existing services; maybe that is not as exciting as someone driving a semi-trailer on stage, but you can’t argue that getting more polished services at a lower cost will put a smile on customer faces.  Before leaving on holiday, I suggest spending a day or two to analyze the changes that have the potential for biggest impact on your cloud ecosystem and putting a plan in place to test and implement them in Q1.  For example, I found three key areas of cost improvement for Turbot (both SaaS and Enterprise across v3 to v5).

1. New gp3 volume types are 10-20% savers for every gp2 workload and many high IOPS use-cases too.

2. New Amazon S3 Bucket Keys will end up saving us thousands in KMS charges.

3. The change from 100ms to 1ms billing granularity for AWS Lambda is saving us so much more than we expected based on our numbers:

   Turbot uses a ton of AWS Lambda to run governance guardrails at massive scale across AWS, GCP & Azure. In just one of our dev environments, we have 2,000+ event driven Lambda functions that are invoked >42M times/month.

   Our Lambda functions have an average duration of 600ms with a floor of 0.35ms and long tail out to several seconds, because the wide distribution we initially assumed an average savings of ~50ms per invocation or about 10% of our monthly bill. However, our actual billed usage is more interesting.

   We see that the change was implemented on Dec 2 and our Lambda billed usage went from an average of $32.33/day all the way down to $6.73/day (savings of 79%). Clearly, a large number of very short function invocations (<50ms) were spiking our costs.

It is gems like these that will have your management team and finance team finally understanding the true cost savings and optimization opportunities you cloud strategy promised when it was fresh.

The rise of specialization…

It is abundantly clear to our customers that the days of being the “Cloud Guy” are over.  The service landscape from the cloud providers is too broad for one person (or team) to be expert across everything.  Amazon RDS has much more in common with Azure SQL and GCP Cloud SQL than it does with EC2 or S3, so does it makes sense to have specialists in cloud database technologies?  Our customers think so, and the same holds true for other specialties as well.  We see teams organizing around core service capabilities in 2021 and beyond. Key areas of emerging specialization:

• Identity (separate from security): specialists in IAM focus on automation and integration with enterprise identity. Building processes to enforce segregation of duties, least privilege and periodic review of access.

• Cloud finance/cost optimization: focused on billing dashboards, cost allocation and coordinating organization efforts to optimize spend.

• Storage: Data retention, resiliency tiers, backups, archive, with multiple services across multiple clouds… this is a really deep area.

• Database: It’s clear from Snowflake’s IPO that enterprise customers have a never-ending appetite for database technologies.  In the past you might have had technology specific DB teams, but we see the specialization here across cloud-based database technologies (helping teams pick the right tools for the application pattern.)

• Networking:  I think 2021 will be the year of NetDevOps…  We are seeing the simultaneous elimination of data center networks and the change in focus here from inbound network segregation/security to how to I protect from data leaving my network.

…and the coming war of standardization

As we specialize, we need more people in those roles and headcount is difficult to justify. This will lead some to try and make the case that they need to support fewer technologies. The basic thinking here is “if I standardize on a fewer number of technologies than I need fewer people to manage them”.  It is a logical argument, easy to rally people around and easy explain to management, but ultimately it reduces IT agility.  This is a mistake; reducing IT agility will have a direct and measurable impact to business agility as well.

We see leading organizations using the following strategies to avoid the pain of the standardization pendulum:

1. Justify headcount with cost savings.  A huge part of the “experts” job should be in assessing what is being used to optimize spend. A good rubric to know if an area is worthy of justifying a specialist head “is the spending in this space at least 4x the cost of the specialist?”

2. Generalize the specialist.  Having a cloud technology specialist that works across products in the same family makes more sense than specializing on specific service offerings (e.g. I would rather have a Cloud-Database Architect than an RDS Manager).

3. Proactively communicate your wins.  Cloud is moving fast; you need to highlight that pace of innovation and what your team is delivering on. Show you are in control (and accelerating) by ensuring that you have a steady stream of technology and business wins to share every two weeks. By keeping initiatives small (two-week sprints) you will have 20+ wins in 2021 across every service team, while keeping morale and productivity high.

What are your focus areas for 2021?  We would love to hear from you and include some of your insights in an upcoming case-study.  If you are new to the Turbot newsletter and like this type of content please consider subscribing and sharing with colleagues.

In American football, the prevent defense is a type of defensive play called when a team is leading in the game with very little time remaining. The core idea is to allow the opponent to execute short plays and focus all energy on preventing a very long scoring play. John Madden famously chided teams that used the strategy with this zinger: “All a prevent defense does is prevent you from winning."

Similarly, I believe that the current trend of cloud operations & security teams relying heavily on preventative controls for cloud governance will ultimately undermine the organizations business objectives and create new risks for the organization; but before I get started defending that statement, lets quickly get people up to speed with a couple of definitions.

Business Objective: A business objective is a (hopefully measurable) positive result that an organization hopes to achieve through execution of a business strategy.

To achieve those business objectives every organization needs to manage risk, and as it pertains to public cloud governance the risks typically fall into one of a few key categories:

• Reliability and availability of systems

• Effectiveness and efficiency of operations

• Compliance with internal processes, laws and regulations.

• Protection of data and systems from internal and external threats.

Control objective: A control objective is a (hopefully measurable) statement that describes a desired outcome for the organization as it pertains to managing or mitigating risk, and we achieve that objective by implementing one or more controls. Types of controls:

Preventative controls: Mechanisms that are designed to keep undesirable events from occurring.

Detective controls: Mechanisms that are designed to find and raise awareness of undesirable events.

Corrective controls: Mechanisms designed to compensate for and reduce risk when an undesirable event occurs. Side note: By definition, this implies that a detective control was in place and effective as well.

With the advent of new shiny tools in the last few years, it is becoming increasing common for me to see an organization’s entire governance control framework designed around preventative controls. Frequently to the extent that it makes it difficult to actually apply detective and corrective controls to their environment. This often leads to slower cloud adoption, developer dissatisfaction with IT and an overall reduction in the business benefits organizations were seeking when executing a cloud strategy. Cloud advocates inside organizations need to raise the alarm early on the pitfalls of this approach, hopefully I can give you some talking points to create awareness within your organization.

Green field vs. Brown field

By the time that compliance, security and IT organizations get around to implementing governance controls there will already be a significant footprint of applications in your public cloud. Preventative controls cause headaches because they will often be closing the barn door after the horses have got out. The early adopters of cloud in your organization are critical allies to have in building out your governance strategy, so implementing a framework that ignores them completely, or blocks them from executing is not viable.

Exception Management

A hard-fought lesson for many enterprise IT groups is that one size does not fit all. Your marketing organization’s public websites, your R&D organization’s data lake and legacy data center migration workloads need completely different governance controls. Teams that start with a single “best-practice” preventative controls for their whole organization/tenant structure will spend 10x as long tweaking and relaxing those policies as they come in contact with the real-world use cases. The time spent doing that and the back and forth between the app teams and cloud team is non-value added work, that slows organization velocity.

Who’s watching the watchers

Someone in your organization (or contracted to your org) is writing and implementing those preventative controls. Too often I see a very small group or even a single individual that has visibility, technical understanding, and access to change those controls. What happens if they are targeted and compromised by bad actors or just make a mistake?

Even if you are not worried about insider threats, your auditors will be. In our experience organizations completely underestimate the time and resources required to implement elevated access controls and to make changes to your control framework when your organization relies primarily on preventative controls for cloud governance.

Friction with your business teams

Preventative controls (by their very nature) cause friction in someone’s workflow. Many implementations of preventative controls just blatantly deny users from taking an action with no explanation given.

This can often lead to an unexpected consequence that undermines the concept of “least privilege”. When confronted with an error message developers and data scientists will grant higher and higher levels of privilege to their users and services accounts to work around the access denied block. When it is finally discovered that it was a preventative control causing the blockage, how many teams will have the discipline to go back and remove the expanded privileges?

This friction impacts your development teams and data scientists, with upstream impacts to the projects they are working on. Your organization is competing to attract and retain these highly paid professionals, if their productivity is impacted you will see measurable reductions in the organizations business agility. A recent McKinsey study found that organizations that were leading with regard to enterprise agility also had better business outcomes:

What is the leading practice?

One approach does not fit all. I will avoid falling into the trap of saying there is one way to solve these problems for all organizations. The approach will need to change based on your industry, your organizations culture and where you are starting from, but I think these key tenants of implementing a cloud governance strategy are broadly applicable and can serve as a template to start from:

1. Visibility and observability are mandatory. Every cloud resource that has the potential to be misconfigured needs to be cataloged in a cloud-scale CMDB and changes to that resource tracked over time. This should be solved before you put a single control in place.

2. Define a list of services that are acceptable for use in your cloud(s), and a process for evaluating and approving new services as they become available.

3. Define a data classification strategy. Many controls need to behave differently, based on the classification of data they interact with (e.g. PII, Confidential, etc.). If you don’t already have a robust data classification strategy, go build one.

4. Document control objectives for each service. You should have a short list of control objectives for each approved service. (e.g. public access, encryption, tagging, etc.).

5. Implement detective controls for existing and new environments. Your detective controls should be effective in identifying any resource in the CMDB that is not meeting your control objectives. Query your environment and test that your control logic can correctly identify non-adherence and misconfiguration scenarios. Automating this testing now will pay dividends for years to come.

6. Notify and teach. Implement a process for detective controls to trigger a notification to application teams, with the details of the control violation and documentation on how to resolve the issue (or request an exception).

7. Exception process. Implement a process to assess and approve exceptions to controls. Resources with approved exceptions should be flagged in the CMDB to prevent additional alarms from being created on that resource during the time period of the approved exception.

8. Green means go. Governance approval for production release should be tied to proof that the pre-production environment (e.g. dev, test, qa, validation, etc.) is 100% green across all controls. This means that they are either meeting the control or have an approved exception in place.

9. Implement automated corrective controls for your most critical control objectives in production. Relying on detect -> alert -> fix manual workflows in production environments is not good enough for critical controls like public access.

10. Create governance accelerators for new projects. Implement automation and corrective controls that ensure that key foundational capabilities (especially ones that are commonly misconfigured) are created in advance for application teams, this helps them move faster while complying with organizational controls.

11. Strategically implement preventative controls in production for unapproved services and for capabilities where you have created governance accelerators. At this point, we are in great shape, we have all our application teams seeing green from a governance standpoint and we have built accelerators where compliance to organization standards is difficult or time-consuming. Implementing preventative controls at this point becomes a way to increase velocity and add value to your business teams, instead of becoming an invisible barrier to their work.

TL;DR

Preventative controls are necessary but not sufficient for cloud governance. Similar to the “prevent defense” in football, overuse of the approach can cause unexpected and deleterious consequences. When starting to implement a cloud governance strategy you should focus first on getting visibility and observability, communication and detective controls in place before tackling preventative lockdown.

If you enjoy cloud governance topics like this, be sure to subscribe to this newsletter and share it with others! Are you a control freak too? Join the conversation by sharing your comments below or contacting us at cto@turbot.com.

Subscribe now

1. Cost Center: Typically, free form but sometimes from a list of approved values.

2. Environment: e.g. Development, Staging, Test, QA, Production…

3. Owner: Typically email or employee id, sometimes a DL or other Group Identifier.

4. Data Classification: e.g. PII, PCI, Highly Restricted, Restricted, Private, Public…

5. Business Unit: e.g. Sales, R&D, Operations…

Strict Approach: Delete any (taggable) resource that does not have required tags/values.
This approach ensures that you always have some tag value (if not always a correct tag value), but it is difficult to retrofit into existing environments if you have large numbers of resources that need to be remediated  …continue reading on the Turbot blog

Our tagging automation was initially released in 2017 and the current v5 tagging capabilities can be taken to new heights via calculated policies. Let us know what your biggest tagging challenge is, we will select one to feature as a case study (with solution) in our open source developer library: https://github.com/turbot/tdk and reward the submitter with a $100 Amazon gift card!  Send your tagging challenge to cto@turbot.com or post an issue on the TDK repo.

Release Digest

Current Recommended Versions

• Turbot Enterprise (TE) 5.28.4

• Turbot EnterpriseDatabase (TED) 1.11.0

• Turbot Enterprise Foundation (TEF) 1.23.0

• Turbot Terraform Provider 1.6.2

• Turbot CLI 1.20.1

Recent New Features and Highlights:

• Support for Terraform 0.13 Beta.

• Improved Performance

• New Performance Dashboards for Enterprise Customers.

• Support for AWS China Regions.

• Full text search on policy type names.

• Support for change windows (prevent Turbot from taking actions except when in a change window).

Thanks for subscribing, and if you enjoyed this feel free to share with the link below.

All the best,
David Boeke, CTO & VP Services
Turbot

Subscribe now