De-mystifying Cloud Operations

So, customers have a lot to consider in ensuring security and resiliency for their cloud infrastructure.

Many customers assume that a public cloud providing Infrastructure-as-a-Service (IaaS) is equivalent to classic outsourcing². They expect that public cloud providers are full-service providers and should be managed like a traditional outsourcing provider³. They underestimate the implications of the shared responsibility model. Understanding the support responsibilities and deploying the right tools and controls to manage your piece of cloud is very critical. Many organizations struggle in the “Day 2” operations of their cloud.

In general, born-in-the-cloud startups and small IT organizations prioritize velocity and agility but operate their cloud environment without many written rules. That may work for a handful of cloud-hosted applications. However, for most organizations, there are regulations and policies to be met, and these organizations must carefully consider how they’re going to manage their cloud operations as they move forward. Clear governance is worth the time it takes to ensure a secure, high-quality cloud ops platform.

On the other hand, large corporations that have many decades of IT operations experience now need to evolve their people, process, and tools to handle the DevOps mode of operations. They typically have the governance process in place but lack the agility and velocity to be successful with cloud ops. If they fail to evolve and try to operate the cloud using their traditional processes, they will not realize the benefits of cloud ops and may end up repatriating from the cloud.

Cloud migrations only a means to an end – business growth. So, cloud Infrastructure Operations frameworks and methodologies must be devised to meet the business needs. We must have an outside-in approach to design our cloud operational framework and to adequately meet the business requirements. Business owners don’t care what technology or tools the IT team uses to deliver business applications for their customers. Every enterprise is now striving to accelerate their digital transformation and increase their customers’ delight in using their business apps.

Net promoter score (NPS) and Service level agreement (SLA) performance are some of the key metrics to measure customer delight and affinity. We must arrive at the service level objectives (SLO) to achieve the desired NPS and SLAs. Once we identify the SLO’s, we should calculate the Error Budget, which is a concept coined by Google as part of site reliability engineering (SRE). It is a measure of risk, or the amount of headroom IT Operations have above their agreed SLA. As the number of 9’s in the SLA goes higher, the window of acceptable downtime (Error Budget) becomes minutes or even seconds.

Higher levels of application resiliency need significant investments. At the same time, IT organizations should not over-engineer or over-invest in operational tools and processes. Too many tools will result in tool-fatigue and may impact the overall return of investment.

We need clear, measurable goals while designing a cloud operational framework aligned with business objectives:

It is a common theme to quickly stand-up a DevOps (sometimes called site reliability engineering – SRE) team with a few full-stack developers and expects them to build and operate a large enterprise IT environment in the cloud. However, in reality, it is rarely effective in action. Although the developer-heavy DevOps approach has many advantages, in large hybrid cloud operations, we need specific infrastructure operation skills like network management, element management for on-premise gears, and enterprise-level centralized log management implementation. Infrastructure automation is another area where a developer may find it difficult.

One way to address this challenge is to build a highly specialized CloudOps Ground-Crew team by cherry-picking the best talents from different teams: Datacenter IT, Full-stack Dev, and Automation SMEs.

Here are a few recommendations to build an effective Cloud Ops team:

In traditional IT environments, changes and releases are performed manually. Organizations establish a change approval board (CAB) to review and assess the impacts of each change and approve it for implementation in the next scheduled change window.

Unlike traditional environments, the cloud enables infrastructure as code which significantly affects operational processes. The age-old manual steps can now be automated and integrated into the continuous delivery pipeline.

In a cloud environment, elasticity, automated provisioning, and decommissioning are the norm. In some scenarios, we may need to perform automated remediation of any anomalies by intelligent automation (AIOps). Those actions must be pre-approved by the CAB and be triggered as business needs. For new software releases, end-to-end automated, continuous, deployment pipelines are used. With this approach, a change is automatically promoted through the various stages. Appropriate guardrails like blue-green deployments and canary deployments5 are implemented to ensure a seamless application upgrade.

Cloud environments need a different approach for instrumentation because the observability of cloud apps needs enhanced tooling. Cloud providers offer a host of native tools for monitoring, log management, security, and governance. At the same time, the market is very crowded with ISV tools for cloud management.

For service management and collaboration, most large enterprises tend to extend their existing legacy tools to the cloud. This approach often faces challenges because of feature limitations, licensing complexities, and integration issues.

If we adopt a 100% cloud-native approach for tooling and management, many CSP native tools are not mature in features when compared with established third-party IT management tools4. Each enterprise is unique and there are no “best practices,” which could be applied to every organization. We should consider business needs, cost implications, integration requirements, and vendor lock-in aspects before instrumenting the cloud. In the Interop ITX 2018 State of the Cloud report4, 37 percent of respondents mentioned that they use cloud-native tools. Unfortunately, relying on these native tools restricts enterprises from having a single pane of glass management dashboard, especially in a hybrid or multi-cloud scenario.

For a hybrid cloud enterprise environment, we believe that the ideal approach is to continue to rely on your on-premise third-party tools investments and extend them to cloud (with integrations with cloud services), if possible, because eventually most of the ISVs will be enabling their platforms with cloud management features. On the other hand, CSP proprietary tools will focus on a specific cloud, without much support for on-premise workloads.

We help enterprise customers to achieve the ideal interim state by marrying both worlds – integrating on-premise management platforms with Cloud-native tools and services (Cloud watch, Cloud watch Logs, Azure Monitor, Azure App Insights, and others). Most of the tools allow this integration with API hooks or connectors. This way, the cloud service anomalies are monitored, and the customers can continue to use the familiar dashboards and consoles to manage their hybrid environment.

However, in the case of new microservice-based applications, organizations may need modern tools to manage these environments. We will cover this in detail in the Tools Approach section below.

Observability is an important pillar of cloud ops because organizations are now moving towards containerized workloads and scalable microservices architectures in the cloud. Traditional agent-based tools may not scale or be adequate to monitor and trace the modern, distributed cloud-native systems.

There is a notion in the IT community that observability is nothing but monitoring-on-steroids; however, it’s much more than just monitoring. By definition, “Observability is a measure of how well internal states of a system can be inferred from knowledge of its external outputs”.

Monitoring is an action we do on systems. Observability is enabling or instrumenting the system to help us to detect and track anomalies in the system. Some observability practitioners call this a “digital exhaust”. When you re-architect your legacy apps to microservice-based distributed systems or build green-field apps in the cloud, it is critical to instrument your application with a high level of logging and then enable the operations with modern tools which can monitor the logs and trace the events.

Observability should start from the customer experience, and its instrumentation approach should encompass synthetic transaction monitoring, APM, Log monitoring, Tracing, and Dashboards. Popular tools for observability and monitoring for cloud-native, distributed business applications are included in the eBook.

Enterprises may choose any of these tools or combination of tools depending on the business needs, budget, and long-term sustainability. Every organization is unique. So, it is difficult to put any best practice here. Myridius helps customers make the appropriate decision with our extensive expertise in building cloud-native apps and applying various tool features.

While using cloud resources, it can be frustrating to ensure everyone has the right level of access to the services they need. Things get even more complex when administrators need to make sure everything is properly updated and compliant with security standards. Cloud Service Catalogs help us to organize, govern and distribute application stacks as templates or blueprints, and we can group sets of products and folders called portfolios. By applying Identity and Access Management (IAM) permissions and constraints, administrators can then provide users the ability to self-service, discover, launch, and manage those products or services without needing direct access to the underlying cloud services.

One of the challenges customers face in managing cloud is to manage accounts. Many enterprise customers implement Account Vending Machines (automated account creation) to support self-service and also for better security and governance.

One of the popular AWS solutions to automate account creation is the AWS service catalog. Using the AWS Service catalog, we can quickly generate accounts for the AWS organization, configure baseline templates to employ during the creation process and provision the new accounts using those templates. The account baseline cloud formation template will be deployed into the new account that is created and provision an AWS service catalog portfolio with a set of predefined products that can be deployed on AWS, the lambda function will create a user with least privilege access to the service catalog products.

Highly regulated enterprises prefer using hardened, purpose-built server and Docker images in their cloud environments. AWS Image builder is widely being used to generate an automated pipeline to produce customized Linux and Windows Server images. When software updates become available, Image Builder automatically produces a new image and distributes it to AWS regions after running tests on it.

Best practices include:

Tagging of cloud resources is essential for effective governance of multi-cloud environments. Tagging helps you better categorize cloud resources for budget tracking, reporting, chargeback, cost optimization, compliance, and security. Enterprises should approach tagging as follows:

Automation is key to implementing tags. If you use cloud management or provisioning orchestration tools, you must make sure that the workflows or pipelines scripts are equipped with tagging enforcement while provisioning new cloud resources.

Cloud users underestimate their wasted cloud spend12. In a recent survey, responders estimated 27% waste. Managing cloud spend is the highest priority for enterprise customers.

For effective cost management in the cloud, we must follow the below hygiene habits.

There are many tools available from cloud service providers as well as third-party ISVs.

Cloud governance is an extension of Enterprise IT governance. There’s no need to re-invent the wheel. There are many IT governance models like COBIT, ITIL, and CMMI, and they all take a different approach. COBIT is a more risk-focused approach in terms of the governances. ITIL is a very process-oriented framework and CMMI is a maturity focused framework.

For cloud security controls, you can use CSA, NIST or CIS controls. There is no need to adopt one framework to its entirety. As per your business and regulatory needs, you can mix and match suitable controls from these frameworks.

This concept is somewhat similar to the age-old active directory group policy. We can define guardrail policies at the organization level and at the OU level to prevent policy drifts.

Cloud IAM authenticates and authorizes all cloud platform users according to the policies and roles. Traditional identity providers like active directory can be integrated into cloud IAM to enable seamless federated access control with a single gate to manage. We can use multi-factor authentication methods to further control privilege account access to cloud platforms. Enterprises must implement MFAs before open up the cloud environment for everyone.

Here are some of the common patterns popular in the Enterprises: