Storage
Mobility
Protection
Analysis and control
TOPICShare
In part 1 of this article, I reviewed how your monitoring strategy can (and must!) go hand-in-hand with you compliance strategy. However, like most aspects of modern IT, there are plenty of challenges that can get in the way of achieving your goals. Three challenges stand out as the most significant:
Challenge 1: Volume of Events and Alert Overload
Alert overload is often a result of the sheer number of servers, devices, systems, and applications that live within our environments. Many organizations don’t calculate in advance the impact that proliferating cloud, virtualization, and microservices resources can have on monitoring requirements—and when speaking of the dynamic and ephemeral nature of containers in particular, filtering the expected from the unexpected can be tough.
The ease with which application owners can scale resources can lead to unexpected surges of event activities that overwhelm traditional monitoring systems or max out the resource allocation of event logs. To compound the challenge of a proliferation of tooling, many traditional monitoring systems don’t understand containerized environments, just as many modern tools don’t understand legacy environments.
Challenge 2: Complexity of Monitoring Rules
Tools expected to alert users to complex environmental changes can make monitoring particularly challenging for operations. A common problem occurs when operations teams assume false alerts and ignore potentially real events. This often happens when, due to high volume, inexact rules generate too many alerts for a typical operations team to monitor. The use of tools such as Cloud Insights provides AI and machine learning (ML) concepts to help make analytics actionable and valuable. These tools allow an organization to deploy monitoring in complex and difficult-to-manage environments without having to create static thresholds. Cloud Insights’ advanced technique of alerting on anomalous conditions provides advantages to teams working with large data sets who need to produce rapid and statistically valid responses.
Challenge 3: Architecture of Complex Systems
Systems and networks with different levels of trust introduce challenges for monitoring systems that must communicate between and across those trust boundaries. A possible solution to these challenges may be a unique trusted configuration for the monitoring system or multiple monitoring systems with some form of portal to compare information between environments. A company's level of data sensitivity and the availability of engineers to build an appropriately controlled monitoring environment may determine how the monitoring system must be installed. With intelligent tools such as Cloud Insights, your teams can use architecture-aware AI and ML features to start monitoring architectures that are not completely documented, but that have to be in scope for compliance monitoring.
Important Issues to Consider
For many organizations, implementing a monitoring system is often a decision based on product/tool and installation resources. If the systems to be monitored proliferate in the cloud or have to address multiple trust barriers (as discussed above), extra effort is often required to understand changes to the infrastructure and deploy monitoring assets appropriately. To avoid rework, it’s important to identify and prioritize assets that are typical elements of risk assessment in those environments. Tools like Cloud Insights should be used to keep track of all assets in the environment to ensure a complete and protected monitoring system.
Also, it’s important to select monitoring tools that address the technology and complexity of the environment that users have at present, as well the technology and complexity they may introduce in the near term. Cloud services, virtualization, and microservices can often introduce additional monitoring requirements outside of normal change control and planning. Unplanned expansion presents a tremendous compliance problem. As mentioned above, the introduction of microservices, virtualization, and multiple cloud environments often results in rapidly changing systems with monitoring gaps, simply because the infrastructure is too dynamic to keep pace with. To overcome this difficulty, users need to design solutions that are automatically or easily integrated into development and production operations. That way, virtual systems are automatically monitored as they are deployed.
Tool selection is particularly important in environments where high volume and high virtualization regularly occur. When tools can provide AI functionality, they help to process high volume correlations rapidly by judging what good and bad pattern behaviors look like. Modern monitoring tools often provide alerting and compliance reporting for regulated environments out of the box.
In Conclusion…
Monitoring is not only an operational imperative; it has become part of most compliance programs. Operations management must consider the value of monitoring requirements within the ever-increasing complexity of the computing environment, especially with virtualization, the proliferation of assets, and rising security risks. To accommodate these challenges, consider researching your industry’s architecture and compliance requirements before instituting a full monitoring solution. Your selection of automation tools should include features that will help your team find and rapidly implement the monitoring of new assets, new services, and network devices. Consider using tools like Cloud Insights that apply AI and ML to improve the intelligence of your monitoring. Avoid increasing the risks that come from compliance (for example, fines and mitigations) with a tool that is designed to work in complex, hybrid, and virtualized cloud environments.
Learn More About Cloud Insights
Visit Cloud Insights on Cloud Central to learn more about how this comprehensive monitoring tool can work for you.
