Two concepts of the Index require getting used to:
The Index is a difficult exam because adversaries are sophisticated. It has to change annually because adversaries evolve their techniques. The Threat Resilience Metric is a threat-driven metric unlike any previous public industry effort. Adopters of the Index need to learn its place in their metrics and program maturity assessments at the onset. Because we have established has good coverage across the sector and early adopters, we feel it is an industry leading benchmark. The Index is not intended to take the place of NIST CSF, FFIEC CAT, ISO27k, or any of the static frameworks that we’ve traditionally used to assess program maturity. Financial institutions are used to being assessed relatively high on a CMMI-like scale across NIST CSF functions and they can still do that. Adopting the Index complements those program-wide frameworks by validating maturity ratings through evidence-based testing rather than relying only on interviews and documentation.
The FS Index isn’t a cake walk and most organizations score below the benchmark their first time. An initial Threat Resilience Metric of 45%-55% is not unusual. That’s the bad news. The good news is that a low initial score drives a focused action plan for improvement that can gain CISO visibility and support for significant improvement. The Index, especially when paired with VECTR’s tracking and visualizations, helps the organization run down its weaknesses. For organizations who would be sensitive to discuss an initial low score, Security Risk Advisors recommends doing two Index exercises before widely sharing the results, because a compelling improvement story will already be on display and help get support for further adoption.
Another caution for the Index: be brutally honest. It is best used in an open book, broad stakeholder setting. If a fusion center or blue team runs it alone without the red team, or vice-versa, there is a risk that a partially successful outcome will be reported as successful, and a false sense of security. For example, a credential access test on a server that is “blocked” but not “alerted”. This should not be considered successful – not yet – until a high-fidelity alert is also sent to SIEM/SOAR. The attacker would otherwise just move on to another technique and remain undetected. The Index should be used in a collaborative workshop settings with red, blue, intel, hunt and GRC at the table.
Fusion Centers and SOCs can be crushed by the feeling they need to be perfect to thwart advanced adversaries. The Index is a carefully curated set of 60 test cases mapped to top threat actors. An organization that practices those 60 and succeeds at more than half is well on their way to flip the burden of perfection to the adversary. The adversary can no longer make mistakes and remain undetected. An organization using the Index has done its homework and practiced for the real test. It has many reliable traps and has increases the likelihood of, and time to detect. With the Threat Resilience Metric, the CISO office has a clear story to tell.
By adopting the Index and Threat Resilience Metrics, the CISO office can learn and elevate a clear story with the following concepts to the Board. Early adopters already have.
© 2023 FS-ISAC, Inc. All rights reserved.
Tim has been a speaker at RSA, Gartner, FS-ISAC, H-ISAC and (ISC)2 National Congress. Tim helped found Security Risk Advisors in 2010. Tim advises CISO Offices on modernizing cybersecurity strategy to improve...Read More
governance, communication, team culture and growth, detection and response capabilities. Tim is a thought leader in the area of purple teams and attack simulation and metrics to describe quantified threat resilience. Tim has a background in penetration testing, security assessment, and frameworks.