Metrics: Valuable security indicator or noise?

Many organizations believe that IT, and by association IT security, is a wasted expense. They recognize that the financial investment needs to be made but few truly understand what they’re rubber-stamping, and even less believe they’ll see a return on this outlay. But it doesn’t have to be this way. How can security support revenue growth and profitability? The secret is metrics.

Metrics: More than numbers
By definition, metrics are “parameters or measures of quantitative assessment used for measurement, comparison or to track performance or production.’

When it comes to an organization’s network infrastructure, and even its security, metrics are a powerful indicator of how well, or badly, the enterprise is at responding to a given situation.

The reason many organizations fall short when using metrics is miss-communication. IT will often deliver reports detailing user access, permission structures and patch management timetables when justifying additional budgets.

A request for extra storage is made under the guise of gigabytes, terabytes and petabytes. Even orders for desktop computers, tablets and smartphones are complicated by a myriad of confusing acronyms and abbreviations.

While on the surface it all sounds plausible, and perhaps vitally important, what does it actually mean? The reality is, not a lot. Instead, of blindly accepting the proposal, CEOs need to demand comprehensible reports from the IT team, framed against the mission of the organization.

False flags
Let’s start by looking at one metric that is often tracked, but has little relevance as a management metric – the cost of the security program. In reality there is little correlation between cost and security. For example, if I halved the security budget would I be half as secure? Or equally if I doubled the budget would I be twice as secure? Of course not – infosecurity doesn’t work like that, unfortunately.

Security is not just the remit of the CISO but is a team effort. Any decisions need to be made with a cross-functional view – senior management, business units, sales, marketing, legal, customer support etc. so that everyone knows the part they play, and IT understand how to weave all the disparate elements together.

Mission-based geek
As an illustration, a large US retailer defines its CISO’s mission as:

• Insure our site is available to our customers when they want to shop
• Insure that our customers feel safe and secure as they shop with us
• Insure that our customers’ information is safe with us at all times
• Insure that we satisfy the necessary legal, regulatory or internal requirements so that we remain a viable business.

With clarity, the executive team are able to ask the CISO for metrics framed against these objectives. With each passing month IT’s results offer intelligence as to how well the infrastructure and other areas of the business are performing to support the mission, highlighting inadequacies and allowing adjustments to be made.

The unrealistic goal
Many executive teams set their CISO up for failure – setting the “mission’ as zero breaches. In the real world, things will happen, vulnerabilities will be exploited, and the organization may suffer a breach in spite of their best efforts. With “zero breaches’ as the target, your CISO will either fail or resign first.

Instead, ask for metrics and indicators that demonstrate success measured against achievable goals and continual incremental improvement. Good examples are:

• Percentage of breaches that have resulted in loss
• Mean time to detect and remediate breaches
• Reduction in the risk of injury incidents detected
• How often is the infrastructure offline: for how long, what caused the outage, what could be changed to reduce outages
• Are processes being adhered to
• Are security practices being circumvented: which ones, by whom, what alternatives could be introduced, what actions were taken to deter future infractions.

Metric top tips
Sxactly which metrics will be useful to any organisation is personal as it’s determined by the business’ goals. However, the principles remain the same:

Set the priority framework: From the outset, everyone within the business should understand what needs to be done to meet the organisation’s objectives. The metrics collected are to verify how well this is being met – or not! This ultimately helps focus efforts on what few things can be done today, to make the most progress towards the end goal. There will always be too much to do – priorities enable staff to make good decisions that align with the priorities of the business.

Perfection takes time: While the end goal may be perfection, a few mini targets of continuous improvement along the road will help build confidence.

Wheat or chaff: Rather than getting into granular detail, IT should be able to quickly and easily abstract the salient stats based on the mission. Solutions exist that automate the process to collect metrics which are then measured against rules. Results are then flagged – with red, yellow or green indicators, so performance can be determined at a glance showing where the organisation is on track and highlighting what requires immediate attention.

Use them or lose them: If IT are regularly producing a report that is not being used, or deemed to offer little value, then why let them continue? Either the statistics need to be presented in another useful format or not collected at all.

Ultimately, when it comes to metrics, don’t allow your IT team to hide behind complex pseudo-science or bamboozle you with stats. Show you’ve confidence in your IT team by asking what the end result is, what the requested investment will deliver, how it helps with the end goal of driving revenue growth or profitability and even how it will make customers happy. If you don’t understand the response make them explain it in a language everyone recognizes. They can speak English – although sometimes it has to be forced!

IT has the intelligence available at their fingertips – you just have to ask them the right questions to get it.