New views into cloud application performance

You can’t monitor what you can’t see. That seems overly simplistic, but it is the heart of the problem facing IT departments as their apps and other data resources are more widely dispersed among on-premises systems, and across public and hybrid clouds.

Universal visibility will become more important in the future: according to the Cisco Global Cloud Index, 92 percent of all workloads will be processed in public and private clouds by 2020 (more info on the Cisco site).

To address this “visibility gap,” companies have to find a way to identify and extract only the most-relevant monitoring data to avoid swamping your monitoring, analytics, and security tools. One problem is that in virtual data centers, 80 percent of all traffic is internal and horizontal, or “east-west,” according to Cisco. By contrast, most cloud traffic travels vertically, or “north-south.” They’re not designed to scale up and down fluidly as virtual machines are created, run their course, and disappear.

Adopting a virtualized infrastructure flips network traffic monitoring from predominantly east-west to predominantly north-south. Source: NeuVector

Four components of monitoring modern virtual environments

Realizing all the efficiency, scalability, and agility benefits of today’s virtualized infrastructures requires monitoring four distinct aspects of application traffic:

Horizontal scaling: as an app scales up to accommodate exponentially more users, the tool you use to monitor the app has to be capable of scaling right along with it.
Viewing virtual-network segments: What may appear as a single virtualized environment is in fact partitioned to secure sensitive applications and data. Monitoring traffic between the virtual segments requires peering through the virtual firewalls without compromising the security of the protected assets.
Accommodating containers: Analysts anticipate a ten-fold increase in the use of containers as more companies convert their apps into multiple virtualized containers to improve app performance. Your monitoring system has to access these containers as quickly as they are generated and redeployed.
Running DevOps at light speed: The lifespans of virtual machines, containers, and the apps they comprise are shorter than ever, which means the DevOps team has to keep pace as it deploys new builds and updates existing ones. This includes the ability to archive and retrieve monitored data from a container that no longer exists.

Integrate network security with app monitoring

It’s not just the people charged with app-performance monitoring who are racing to keep pace with the speed of virtualization. Security pros are struggling to find tools capable of identifying and reporting potential breaches of widely dispersed virtual networks. Network security managers often don’t have access to monitoring dashboards and are unaware of on-the-fly network reconfigurations.

This has led to a new approach network modeling that extends to virtual networks to security and compliance in addition to performance monitoring. The key is to normalize data from diverse sources – whether physical, virtual, or purely cloud-based – and applying a single set of network policies that encompasses security and performance monitoring.

Perhaps the biggest change in mindset is that all data traffic has to be treated the same: the traditional north-south pathways into and out of the network that have always posed security and monitoring challenges, as well as the east-west transmissions inside the network that in the past were considered trusted and 100-percent contained. There is no longer a “trusted zone” within the organization’s secured perimeter.

New views of end-to-end network modeling

The approach that many companies are adopting successfully is to model north-south and east-west network traffic using a single set of policies. Doing so offers what Skybox Security VP of Products Ravid Circus calls “a more realistic view of applied policy at the host level rather than verifying access only at ‘chokepoints’ or gateways to the virtual network.” Relying on one unified policy also breaks down the traditional barriers separating physical, virtual, and cloud networks, according to Circus.

How important is it to keep virtualized environments running at peak performance? In the words of OpsDataStore founder Bernd Harzog, “slow is the new down.” Harzog cites an Amazon study that found an added 100 milliseconds of latency translated to a 1 percent decrease in online sales revenue for the year. The monitoring challenge is compounded by the confluence of four trends:

More software being deployed and updated more frequently
A more-diverse software stack comprised of new languages such as NodeJS and new runtime environments such as Pivotal Cloud Foundry
Applications are increasingly abstracted from hardware with the rise of network, compute, and storage virtualization, as well as JVM and Docker
The rise of the virtualized, dynamic, and automated infrastructure

The monitoring challenge: faster, more-frequent app deployments on increasingly abstracted, dynamic, and automated virtual infrastructures. Source: Network World

Constructing your best-of-breed monitoring toolkit

If you’re relying on a single vendor for your network-monitoring operations, you’re missing out. Multiple tools from multiple vendors is the new reality, which makes selecting the optimum set of tools for your environment critical. The pit many companies fall into is what Harzog calls the “Franken-Monitor:” Each team relies on its own favorite tool at the exclusion of all others, so you spend all your time trying to move performance data between these virtual silos.

To avoid creating a monitoring monster, place the metric streams from all the tools in a common, low-latency, high-performance back end that lets the performance data drive IT the same way business data drives business decision making. That’s where a service such as Happy Apps proves its true value.

Happy Apps provides SSH and agent-based connectivity to databases, app servers, and messaging queues as a single application. You are shown an overall status on the unified dashboard, as well as the status of each group member. Dependency maps show users the relations between distinct IT systems, as well as the systems’ effect on applications. Best of all, Happy Apps keeps IT moving at cloud speed without launching your IT budget into the stratosphere.