Day 1
Day 2
Day 1 | |
---|---|
09:00 – 10:00 | Opening Remarks |
10:00 – 11:00 | US GB Benjamin Edwards (US); Éireann Leverett (Concinnity Risks, GB) |
11:00 – 12:00 | The Impact of Internet-wide Scanning on Known-vulnerability Exploitation Jamie O'Hare |
12:00 – 13:00 | Lunch |
13:00 – 14:00 | An Open-Source System for Customizable and Interpretable Vulnerability Exploitation Prediction Fredrik Sætran, Mark Anderson |
14:00 – 15:00 | US Jay Jacobs (Cyentia, US) |
15:00 – 16:00 | Coffee Break & TBD Talk |
16:00 – 17:00 | Keynote : TBC |
18:00 – 19:00 | |
19:00 – 20:00 | Dinner |
Day 2 | |
---|---|
09:00 – 10:00 | Armin Čoralić, Renout Schoen |
10:00 – 11:00 | NO Automated Vulnerability Chaining with CVSS Dr. Martin Eian (mnemonic, NO) |
11:00 – 12:00 | Putting it into PRACTICE: A Proof-of-Concept Prioritization Framework Based on Customization Matt Wixey (Sophos) |
12:00 – 13:00 | Lunch |
13:00 – 14:00 | Forecasting Bang-for-Remediation Matilda Rhode |
14:00 – 15:00 | US Building CVE.ICU: Enhancing Accessibility to Cyber Vulnerability Insights Jerry Gamblin (US) |
15:00 – 16:00 | Coffee Break |
16:00 – 17:00 | US Insights from 3,000 Known Exploited Vulnerabilities: What Can We Learn? Patrick Garrity (VulnCheck, US) |
Fredrik Sætran, Mark Anderson
Predicting vulnerability exploitation likelihood is crucial for effective risk analysis and patching prioritization. We present an open-source system for predicting vulnerability exploitation likelihood, building on EPSS. Our approach offers improved interpretability and user control, allowing practitioners to train custom models for specific contexts. Key focus areas include:
Feature utility: We challenge the trend of using complex and uninterpretable models with unwieldy number of features. Our system enables users to train models using customized feature subsets, while also providing an evaluation of each features utility. This empowers users to make informed decisions when tailoring models to their specific needs. Data completeness: We address the issue of missing data using language models (e.g., CVSSv3 vectors or CWE information) and potential human errors in feature values by exploring methods to predict and incorporate missing information, assessing its impact on model performance. Model interpretability: We investigate the balance between model interpretability and performance by comparing various architectures and examining how prediction formatting affects outcomes Temporal relevance: addresses the problem of static models being quickly out-of-date in a rapidly evolving space. This ties into our exploration of lightweight models with refined feature sets, enabling practitioners to update their models more frequently and easily.
We will demonstrate the system's performance and practical applications, aiming to provide a more flexible and transparent approach to vulnerability exploitation prediction.
Speaker: Fredrik Sætran. Bio Coming Soon.
Speaker: Mark Anderson. Bio Coming Soon.
October 3, 2024 13:00-14:00
Dr. Martin EianDr. Martin Eian (mnemonic, NO)
What is the CVSS score of an asset? The Common Vulnerability Scoring System (CVSS) can help you understand and score a single vulnerability. Assets often have more than one vulnerability, and the manual approach for vulnerability chaining described in the CVSS User Guide does not scale to the number of vulnerabilities managed by organizations today.
We developed an open source tool for automated vulnerability chaining of CVSS vectors. This tool is available on Github [1]. The tool uses the National Vulnerability Database (NVD) as its data source for CVSS vectors and Common Platform Enumeration (CPE) data. Given a list of CVEs, the tool computes the aggregated CVSS vector: what the adversary needs in order to exploit the vulnerability chain, and what the adversary is able to achieve given the set of vulnerabilities.
As an example, consider an asset with two vulnerabilities. The first vulnerability can be exploited remotely, requires no privileges, and provides local access and low user privileges to a web application. The second vulnerability requires local access and low user privileges to the web application, and provides high user privileges to the operating system. The aggregate of these two vulnerabilities is a vulnerability chain that can be exploited remotely, requires no user privileges, and provides high user privileges to the operating system.
The model used by the tool is based on what a vulnerability requires in order to be exploited, and what the vulnerability provides when exploited. A vulnerability might require user privileges, and local or adjacent system access. These requirements are CVSS base metrics present in the NVD data. What a vulnerability provides, however, is not present in the NVD data, so we deduce what the vulnerability provides from the CVSS Impact and Scope metrics.
Dr. Martin Eian is the Head of Research at mnemonic, and he is the Project Manager for the research projects "Semi-Automated Cyber Threat Intelligence (ACT)" and "Threat Ontologies for CyberSecurity Analytics (TOCSA)". He has more than 15 years of work experience in IT security, IT operations, and information security research roles. In addition to his position at mnemonic, he is a member of the Europol EC3 Advisory Group on Internet Security. He has previously presented ACT workshops at the FIRST Conference and at the FIRST CTI Symposium.
October 4, 2024 10:00-11:00
Jerry Gamblin (US)
In the rapidly evolving cybersecurity landscape, timely access to comprehensive vulnerability statistics is paramount for organizations and individuals. The CVE.ICU project was conceived to significantly improve access to such critical vulnerability data. This talk will delve into the architecture and development of CVE.ICU.
At the heart of CVE.ICU lies the integration with the National Vulnerability Database (NVD), the U.S. government repository of standards-based vulnerability management data. We will explore how CVE.ICU harnesses the rich, structured data from NVD to create a user-friendly interface that simplifies the complexity inherent in vulnerability statistics.
A unique aspect of the CVE.ICU project is the utilization of Jupyter Notebook and JupyterBook technologies. Another key to the project's success is the implementation of automated daily builds using GitHub Actions, a feature that ensures the CVE.ICU platform is constantly refreshed with the latest data.
By attending this talk, participants will gain insights into the development and operation of CVE.ICU, and how it stands as a testament to the transformative power of open-source tools and automation in enhancing the accessibility of cyber vulnerability insights.
Speaker: Jerry Gamblin. Bio Coming Soon.
October 4, 2024 14:00-15:00
Benjamin Edwards
Éireann LeverettBenjamin Edwards (US), Éireann Leverett (Concinnity Risks, GB)
The vulnerability enumeration process through the CVE framework has recently rapidly evolved. In particular, we have moved away from a monolithic system in which MITRE (or a few trusted affiliates) are the sole fountain from which all CVE information flows. Instead we have transitioned into a federated system in which vendors, clearinghouses, and foundations are able to commonly enumerate their own vulnerabilities, and disseminate information about them.
This transition was inevitable as the scope of software has expanded to the point where we are seeing 100s of new vulnerabilities cataloged everyday. This federated process would seem like a godsend were it not for the loose requirements around data quality. In this talk, we’ll examine CVE data quality across CNAs and how it’s evolving over time. In particular, we’ll examine the myriad of potential pieces of information that could be included in CVEs, but rarely are by their reporting CNAs. Required fields like “Description” and “References” are sparse and inconsistent, though nearly always present. However a full 63% of vulnerabilities have no structured information about the affected software. 75% are missing metric information, 80% are missing CWE information, and even fewer bother with lesser known CVEv5 fields like “Credit”, “Solution”, “Workaround”, and “Exploit”. Moreover, even when data is present, it is inconsistently formatted and therefor difficult to extract the relevant information.
After the overview of what’s missing we’ll take a proactive approach and create a measure of CNA performance using Item Response Theory. We’ll explore how this finding breaks down across the most prolific and sparse CNAs. In particular MITRE, the most ancient, venerable, and haggard “CNA of last result” tends to perform the worst. Some vendor and specialty CNAs such as Patchstack, Wordfence, and Solar winds provide particularly rich information, while others such as Apple and Intel only provide the bare minimum. In contrast, some clearinghouse CNAs such as the Zero-Day Initiative and INCIBE (National Spanish CERT) and the Swiss National Cyber Security Center perform quite well.
We’ll conclude the talk by considering how these gaps from in CNAs might affect future efforts at forecast. In particular. highlighting how many of the gaps in the CNA process were filled by NIST’s National Vulnerability Database, which is still catching up after a degradation in capacity early in 2024. We’ll also examine how CISA’s new Vulnrichment program is filling similar gaps as an Authorized Data Provider and including additional information. We’ll also discuss how multiple sources can lead to conflicting information, but also the potential for validation and further CNA performance measurement.
Dr. Benjamin Edwards is a security data scientist working at the Cyentia Institute. An expert in ML and statistics, Ben has led research on a variety of security topics including vulnerability management, application security, human risk, Next-gen SIEM, and security program performance. He is an active member of the security community, contributing to both EPSS and CVSSv4. Prior to joining Cyentia, his research examined global attack trends, the effects of security interventions, nation state cybersecurity policy, and the security of ML models.
Éireann Leverett is the co-author of Solving Cyber Risk, and regularly writes about cyber risk perception, articulation, and quantification. He is a co-chair of the Ransomware SIG, and long time DFIR innovator and data scientist. When he's not working in cyber insurance and risk, he likes writing code, papers, and taking long walks in nature.
While his bio is serious; he hates writing bios in the third person, and once placed second in an Eireann Leverett impersonation contest.
October 3, 2024 10:00-11:00
Armin Čoralić, Renout Schoen
Despite being very valuable, many organizations struggle to adapt external scorings of CVEs (e.g. CVSS, EPSS) to their particular context. While attempting to undertake this, organizations typically spend extensive resources on investigating the same CVE’s. We believe in the potential of combining efforts to collectively learn about CVE impact, and we believe in NCSC-NL's ability to play a central role in this. With a trusted collaboration of organizations we are testing a community-driven machine learning model that scores and prioritizes CVE’s.
In this talk, we would like to discuss the technical components that made our approach possible, the model that was built, and first results and learnings of aggregated community input.
Speaker: Armin Čoralić. Bio Coming Soon.
Speaker: Renout Schoen. Bio Coming Soon.
October 4, 2024 09:00-10:00
Matilda Rhode
Vulnerability prioritisation metrics and frameworks typically operate on a per-vulnerability basis but there are inter-vulnerability dynamics which can be leveraged to predict remediation effort. This talk outlines three metrics based on data science that can help support futher prioritisation.
Some metrics (e.g. CVSSv4) include remediation effort but this is per-individual vulnerability. In practice, there are relationships between vulnerabilities which have implications for the efficiency of remediation efforts. This talk discusses additional metrics to predict bang-for-remediation and is a complement to existing prioritisation approaches.
This talk addresses 3 real-world common challenges: (1) backlogs, (2) context-switching and (3) stakeholder relationships and proposes metrics to avoid. This is not an exhaustive list of under-addressed areas but it is a set of representative scenarios where data analytics can support. The concepts are expanded below, each has been assigned a name to make them easier to remember and distinguish.
(1) Batch patch: there may be a significant backlog of vulnerabilities and patching efficiency can be achieved by prioritising those which are easy to patch (covered by existing metrics) and those which patch multiple vulnerabilities at once, this part of the talk discusses the strengths and weaknesses of other prioritisation methods (e.g. CVSS, EPSS, SSVC, VEST, known-exploited) to dovetail with batch-patch predictions.
(2) Remediation flow: For those vulnerabilities requiring more creative remediation solutions, it attempts leverages the velocity of ""flow-states"" and ""deep-work"" to cluster similar vulnerabilities in order that they can be dealt with using recently-acquired skills, knowledge and social networks.
(3) Stakeholder inroads: vulnerability management entails the management of stakeholder relationships with individuals and teams who have diverse and sometimes competing priorities. The remediation effort in practice may significantly depend on the relationship with a (group of) stakeholder(s). Successful vulnerability remediation in one technology-owner's domain may open an opportunity for remediation in a new domain with another stakeholder. This metrics forecasts the stakeholder groups that can be engaged using related remediation activities and seeks to translate this into a ranking metric for prioritisation.
We hope this talk will be of interest to the Vuln4Cast community and would welcome the opportunity to critically discuss these proposed prediction metrics with others.
Speaker: Matilda Rhode. Bio coming soon.
October 4, 2024 13:00-14:00
Patrick GarrityPatrick Garrity (VulnCheck, US)
VulnCheck gathers vulnerability and exploit intelligence from over 400 global sources, supporting security tools and teams worldwide in triaging and prioritizing vulnerabilities. In early 2024, we released the VulnCheck Known Exploited Vulnerabilities (KEV) catalog, which now features roughly 3,000 CVEs with references, ransomware attribution, and curated exploits, available for free to the community. This talk will explore key observations across known exploited vulnerabilities, offering insights that can help forecast future vulnerability exploitation trends.
Patrick Garrity is a security researcher at VulnCheck where he focuses on vulnerabilities, vulnerability exploitation and threat actors.
He is a seasoned cybersecurity professional with over 15 years of experience across solutions engineering, product and security research roles helping build and scale security startups including Duo Security, Censys, Blumira, Nucleus Security and VulnCheck.
October 4, 2024 16:00-17:00
Matt Wixey (Sophos)
Users responsible for prioritizing remediation efforts have several tools and frameworks available to them: CVSS, EPSS, SSVC, and more. While these all take different approaches, they have one thing in common: they assess the characteristics of a vulnerability in some way, and produce a score or recommendation, so that users can prioritize accordingly. These tools can all be useful, and have their place in vulnerability management, but because they typically focus on the vulnerability, rather than the user, their outputs can lack context, be misunderstood or misused, and may have important caveats.
In this talk, I’ll explore a proof-of-concept framework I've developed to address these problems, with a working title of PRACTICE (Prioritizing Remediation by Aggregating Customized and Tailored Information - a Conceptual Example). Before I get to PRACTICE, I’ll provide some context by briefly walking through CVSS, EPSS, SSVC, the KEV Catalog, CVEMap, and other tools and frameworks in the context of prioritization – discussing the results they produce, their drawbacks, and what they tell (and don't tell) users.
I’ll then move on to the main part of the talk: PRACTICE itself. PRACTICE is an experimental, transparent methodology for remediation prioritization, which takes a novel approach to the problem by putting the user and their unique environment - rather than vulnerabilities and their characteristics - at the center of the issue. It allows highly granular, customized input (via external configuration files, which can also be customized) about a user’s specific environment, priorities, and circumstances relating to a given vulnerability - including attack vectors; security controls and their coverage and configuration; impact; remediation effort and the potential for disruption; and more. All the inputs in each of these categories can be weighted individually, so that users get an informed understanding of how a vulnerability affects them uniquely – therefore enabling them to prioritize on a completely tailored basis.
As for the output, there are no calculations, no scores, and no recommendations. Instead, PRACTICE produces a visual, colour-coded, at-a-glance summary of a user’s answers, providing a simple graphical decision-making aid for how a vulnerability might affect their organization, based on their security posture and priorities in relation to that specific vulnerability. Unlike most other tools and frameworks, the whole idea is that, by design, two organizations might see completely different outputs for the same vulnerability, and can therefore prioritize accordingly.
After walking through the philosophy and design of PRACTICE, I’ll show some demos of prioritizing the remediation of some real vulnerabilities, from the perspectives of some hypothetical organizations with different security postures and priorities; discuss future research efforts and the PRACTICE to-do list; talk about how the PRACTICE approach could work at-scale; and encourage attendees to get involved with contributing to the development of PRACTICE as I continue to think about and build on this proof-of-concept.
Matt Wixey is a senior threat researcher at Sophos, where he conducts and reports on security and threat intelligence research. He previously led cybersecurity R&D capabilities at a professional services firm and a law enforcement agency, digging into emerging attack vectors, vulnerabilities, and new technologies. Matt has presented original research at multiple conferences, including Black Hat USA, Black Hat Europe, DEF CON, ISF Annual Congress, 44Con, and BruCon.
October 4, 2024 11:00-12:00
Jamie O'Hare
Systems exposed to the Internet are under threat to known-vulnerability exploitation. While informative, vulnerability forecasting does not reflect the associated Internet-traffic. Various solutions aim to quantify and classify Internet traffic towards enabling greater insight into the threats faced. However, the impact of Internet-wide scanning within this topic has received limited attention. This presentation presents a research methodology seeking to quantify the impact of Internet-wide scanning on Internet-exposed systems.
Speaker: Jamie O'Hare: Bio Coming Soon.
October 3, 2024 11:00-12:00
Jay Jacobs (Cyentia, US)
This talk will present the recently published EPSS research report (July 30th, 2024) and discuss the future of EPSS. The research explores the prevalence, spread and speed of exploitation activity with several surprises and subsequent insight. It also presents some metrics for vulnerabilities and measures the historical performance of EPSS. Also, this summer we are updating the backend for EPSS and I may touch on some of the data challenges we are experiencing. Finally, I will end with a view and discussion of the future of EPSS.
Jay Jacobs is the Chief Data Scientist at Cyentia Institute, the lead data scientist for the Exploit Prediction Scoring System (EPSS) and is co-chair of the EPSS special interest group at FIRST.
October 3, 2024 14:00-15:00