Metrics SIG

Computer security incident response and incident management has moved towards more mature phases of development. Although there are still new teams forming, many existing teams are focusing on increasing their responsiveness and improving effectiveness.

Like other communities (such as business, finance and government) that look for quantitative and qualitative methods for benchmarking operations and measuring success, there is an emerging need for similar mechanisms in the incident management community. Such mechanisms will need to identify specific baselines for effective performance and provide methods for measuring operational capabilities against such baselines.

The baselines help identify requirements, components, services, and processes for successful incident response or incident management. The measurements will help identify an organization's capability, product and service gaps, along with strengths and weaknesses that can be compared to the baseline or model.

Mechanisms will also be needed to help plan a path of improvement so that teams can not only identify and understand the current state, but define the desired state and a path to reach that state.

Knowing this information can help to identify risks to the team's mission success, determine a strategy for change and improvement, and ultimately improve the overall security posture of the organization.

The scope of this Metrics SIG will be to bring together interested members of the FIRST community to discuss and identify approaches for internally evaluating CSIRT and incident management practices within an organization. The work of this SIG will focus on determining further refinements for best practices for CSIRTs (e.g., building off existing metrics work, FIRST materials, ISO 17799, ITIL, NIST, etc.), defining key performance indicators for a team, determining measures for effectiveness, identifying appropriate performance metrics, and determining appropriate approaches for evaluating systems. There are areas that are beyond the scope of the SIG, namely:

  1. The Metrics-SIG will not, at the current time, focus on maturity models for CSIRTs
  2. The Metrics-SIG is not an accrediting or certifying body
  3. The Metrics-SIG will not evaluate other CSIRTs

During the initial meetings, the scope and charter was identified as well as our initial set of goals. One ultimate goal of this work is to identify or develop products that any organization with a CSIRT or incident management capability can use to evaluate and assess their capability.

Metrics SIG

Expected/Targeted members

  • FIRST members who are seeking approaches for benchmarking and/or improving their CSIRT processes and metrics to provide effective incident management quantification.
  • FIRST members who are interested and willing to help refine, align, and test metrics, as well as to suggest additional improvements for standardizing CSIRT practices within the community.
  • Any CSIRT seeking to improve its incident management capabilities.
  • Other external parties who might provide subject matter expertise in the area of metrics and standards or those who are interesting in building metrics tools and evaluation mechanisms.
  • Other SIGs as appropriate.

Chair

TBD

Co-chairs

Mike Murray (CERT/CC)
Robin Ruefle (CERT/CC)


All requests to join should be sent to first-sec@first.org