Structural integrity: Quantifying risk with security measurement

In my previous post, we set up the foundation for a risk quantification program. Many organizations have begun this part of their security strategy and are learning how to approach this challenge, which has plagued the security industry for years.

In this part, we talk about how a winning security metrics strategy aligns with the business’ goals and objectives and lay out the framework to develop the metrics strategy.

Security metrics are business metrics

A winning security metrics strategy will always align with the business’ goals and objectives. Only by considering these things can we arrive on a security metrics strategy that accurately assesses the risk.

Let’s start with what we CAN measure. Data can be divided into two categories: empirical-based quantitative data and its more experiential sister, qualitative data.

There is often confusion between the two approaches. Each has its benefits, and neither is better nor worse than the other. They often answer different questions, so it’s vital to choose the correct one for each query. Both may play an important role in the execution of your security metrics strategy.

A method I like to use for measurement is Goal-Question-Metric (GQM), an approach created by Victor Basili. Start with a Goal for the security program. Next, determine what Questions need to be answered to achieve the stated goal. Finally, determine what Metrics would answer those questions. Here’s an example of how the GQM strategy might work:

Imagine that a user browses to mywebmail.wolf-in-sheeps-clothing.com and inadvertently clicks on a malicious link embedded in the page. A malicious file is downloaded and installed on the user’s computer. The infected computer utilizes the network for internal data exchange and begins to exfiltrate company PII to a command and control server at mywebmail.wolf-has-your-data.com on port 1234.

How can we use GQM to ascertain the vulnerability associated with this scenario?

Goal statement: Understand the risks of sensitive data loss from “crimeware” by analyzing the implementation of the security control set from the perspective of the end-user.

This is a well written goal statement in that the objective is clear. That said, we can minimize subjective interpretation by defining a few terms.

Sensitive data:

• Company data labeled as “internal” or “confidential”
• Any company PII

Crimeware:

• Opportunistic in nature and have financial motivation behind them
• Frequently affects consumers and is where “typical” malware infections will land

Next, break down the scenario to determine what questions must be answered to achieve our goal. As a rule of thumb, we’ll want to think about and utilize industry recognized data where possible. We’ll also want to identify defensive security controls related to our scenario: OS patching, egress firewall rules, anti-virus, and administer rights all would make sense here.

Appropriate simplification of the analysis enables us to move forward swiftly and not get lost in the minutiae that could make measurement unnecessarily complex.

Document assumptions:

• Security controls are effective/efficient
• Security controls are weighted equally
• Value of corporate data is weighted equally
• Time period over a year (frequency)

Once we have defined a goal and clarified the assumptions and potential components that relate to our scenario, we need to determine the questions that need to be answered. Our outcomes may include:

1. What type of devices are on the network?
2. Where does the sensitive data reside?
3. Who has access to the sensitive data?
4. How many devices are utilizing the current security control.
5. How many crimeware help-desk tickets per year?

Now continuing down the framework, what can we measure to answer the defined questions?

1. Number and type of devices on the network
2. OS and distribution of devices on the network
3. Number and type of approved applications on workstations
4. Number and type of devices up-to-date on OS patches
5. Number of devices up-to-date on application patches
6. Number of active users and user accounts
7. Number of help-desk tickets associated with “crimeware” event

Data sources & analytics: Quantifying risk

Regardless of the desired metrics, when it comes to locating data to measure, there’s no shortage of potential sources. You might use an active directory, DHCP, DNS, vulnerability scanners, OS/application patching servers, network/security devices, identity and access manager/authentication stores, etc.

Utilizing the data gathered across all of your program goals, consider vulnerable asset value and loss probabilities to create a picture of overall risk. A book topic in its own right, there are a few key guidelines to remember:

• Make your risk assessment scenario-based. This not only helps the focus remain on probable events, but also that the right goals, corresponding questions, and appropriate metrics were considered.
• Ensure program-to-business alignment.
• Execute on a methodology for determining the nature and impact of the actual risk.
• Quantify risk with associated probable financial impact. This gives context to the risk assessment and can help guide any remediation strategy decisions.
• Prioritize security decisions and spend on empirical data. No longer are decisions made on gut feelings, but grounded in the findings of a more objective and repeatable process.

Building a winning security metrics strategy

Understanding an organization’s risk is critical. Only with a well thought-out security metrics strategy will you avoid collecting irrelevant data and drawing questionable conclusions. Whether assessing risk on your own or with the help of external security specialists, utilizing a measurement framework ensures your data answers the critical questions about your security, and leads to meaningful analysis that can guide your security program.

Make the most of your measurements: Always check assumptions. Consider gauging implementation before sweating program efficiency. Use measurement principles and calibrate for the human element for metrics you can trust.

And if all of this feels overwhelming, start simple. This might seem like a lot, but understanding your company’s risk exposure is critical, and you must start somewhere.