Program Overview

Before Disclosure:

Forecasting vulnerabilities from source code or software projects, SSDLC, and/or bug bounties.

The Zero Day Initiative (ZDI) is the world's largest, vendor-agnostic, bug bounty program and has been evaluating vulnerabilities since 2005. The ZDI does not share vulnerability data, or metadata, outside of its company. However, in this presentation, we will explore the ZDI dataset for anomalies, trends, and statistics as they relate to the FIRST community and explore what predictions can be made, have been made, and have been missed. What's so interesting about this data vs other available datasets?

In 2017, the ZDI became a research CNA and began submitting CVEs as the CNA of record for any vulnerabilities handled through the ZDI program for which a CNA wasn't available. In 2018 ZDI rearchitected its database structure to modernize and facilitate data analysis. This move also allowed ZDI to more easily affect, and adapt to, process changes. Around the same time, the ZDI switched to CVSS3 (base scores) and started evaluating and tracking additional metadata such as CWE. And finally, the ZDI has had limited turnover in its cadre of decision makers for the last 10 years.

As a result, the ZDI has consistently applied CVSS3.x and CWE to over 7000 vulnerabilities since mid 2018. On average, ZDI acts as the CNA for roughly 25% of its disclosed vulnerabilities, however it assigns CVSS and CWE to all cases and submits that data to the CNA. The responsible CNAs are the final decision authorities on their CVE data and the ZDI does not generally challenge, or even compare, it's decisions to that of the CNA. Some CNAs never seem to change the submitted values, some CNAs always seem to change certain values such as the CWE, and other CNAs only change the values when needed. Regardless of the outcome, the ZDI internally retains all its initial decisions, so it has a privileged and unadulterated view of thousands of vulnerabilities. By no means is ZDI data without bias, but the biases are readily identifiable and are generally consistent over the last decade. This presentation will discuss, and invite discussion on, the data, and any conclusions or predictions that can be made with it.

Joshua "Kernelsmith" Smith is a senior vulnerability researcher and the secdevfuzzops (aka "FuzzOps") senior manager at Trend Micro's Zero Day Initiative. He occasionally analyzes zero-day vulnerabilities, but he and his team mostly facilitate the ZDI's operations and the ZDI's vulnerability discovery efforts and reporting. He's currently an active member of the CVE Automation Working Group and a CVSS SIG observer. In a prior life, Josh was a pentester in the United States Air Force and a senior computer security engineer at Johns Hopkins University Applied Physics Laboratory (JHUAPL). He holds a BS in Aeronautical Engineering and an MA in Management of Information Systems as well as a CISSP and RHCSA. Previously, Kernelsmith has spoken at DefCon, DerbyCon, RuxCon, BSides etc., and was an external Metasploit developer.

Travis Erard is a grizzled veteran of the internet. He has worked with companies of all shapes and sizes, including Fortune 500 companies like Merck, Hewlett Packard, and AstraZeneca, as well as on high-visibility projects for the National Geographic Channel. Travis has also worked with startups at various stages and funding levels. He has served in various roles ranging from leadership and management to front line development. In Travis' current role, he is the Vulnerability Intelligence Architect at Trend Micro's Zero Day Initiative.

Before Disclosure:

Forecasting vulnerabilities from source code or software projects, SSDLC, and/or bug bounties.

FOSS is here to stay, displacing more and more its privative counterparts. Advantages of this transition include the exposure of bugs to be fixed by a community of experts. In the same box, however, we find disadvantages like the exposure of security issues that can be exploited by attackers. Today we even have security websites exposing vulnerabilities and exploits online—and despite good practices like responsible disclosure, it is the sheer amount of (external) code what makes everyone ultimately vulnerable.

From that base, this talk puts forward a concept of probability of future vulnerabilities. This is crucial for project management, but also at developer level, to see the risks of not upgrading (or yes upgrading!) a dependency. We show how this probability can, and must, be computed from a project's dependency tree, in a manner that is intimately related to the use of FOSS. We also show that the development history of the project and its dependencies is key to getting useful results.

Finally, we merge the dependency tree and development history of a project into a white-box model, which we use to estimate the probability of future exploits. We show one way to do this for the Java-Maven environment, for which we can use a battery of tools from the formal methods community.

Carlos Estebran Budde received his PhD in Computer Science in 2017 (Universidad Nacional de Córdoba, AR), continued on to do a postdoc at the Universiteit Twente (NL) in collaboration with Dutch Railways until 2021, and since then works as assistant professor at the Università di Trento (IT). Carlos uses his background in formal methods to perform simulation and probabilistic-based analyses, to assess the cybersecurity resilience of systems' models. In 2022 Carlos was awarded a Marie Curie Postdoctoral Fellowship: his ProSVED project studies how security vulnerabilities can be used for the estimation of future exploits.

Fabio Massacci (MEng’92, PhD’98 Computer Engineering, MA’95 in International Relations), married with two children, has been in Rome, Cambridge, Toulouse, Trento, and Amsterdam. He held visiting positions in Durham, Koblenz, Lueven, Marina del Rey, and Oslo. For a full biography on Fabio, see this page: https://fabiomassacci.github.io/

Ranindya Paramitha (also called Nanin) is currently pursuing a PhD at the University of Trento, Italy, focusing on software security under the supervision of Prof. Fabio Massacci. My interest on this field started when I was doing my bachelor in informatics, at Institut Teknologi Bandung, Indonesia. I continued my master on the same field, doing my thesis on mining software repositories for security. I'm grateful that until now, ALL my higher education years are covered by several scholarships. During my bachelor and master periods, I had several internship experiences in some software companies in Indonesia. Despite having some experiences working in industry, I discovered that I enjoy teaching and (later) researching, which encouraged me to pursue my PhD. I also enjoy attending conferences/ schools as they broaden my knowledge while giving me networking opportunity.

With over 10yrs experience working in Cyber Security, Ceri Jones spent most of that time working in government, the UK Civil Service, where she researched people-centred security. Understanding the needs of people, with particular attention on improving security messaging as well as security awareness campaigns. She has been responsible for championing research in the area of Sociotechnical Security and using her expertise to bring research into practice across government projects. Now working in industry bringing the research into practice and trying to influence the way we work with people in the security field.

Prioritising Response

Which to fix, which to patch, which to investigate after an incident. How to choose between different vulnerabilities at different times.

The Cybersecurity and Infrastructure Security Agency (CISA) uses empirical evidence to both create its vulnerability prioritization policies and to execute those policies to prioritize vulnerability reports for action. CISA is the US government agency tasked with leading “the national effort to understand, manage, and reduce risk to our cyber and physical infrastructure.” Vulnerability management is a part of that mission, and vulnerability report triage is a part of vulnerability management. CISA uses the Stakeholder-Specific Vulnerability Categorization (SSVC) for report triage. CISA also supports our stakeholders’ vulnerability prioritization by publishing the Known Exploited Vulnerabilities (KEV) catalog. SSVC structures decisions about vulnerabilities as a logical combination of facts about the vulnerability and how an organization wants to respond to a vulnerability that matches those facts. This document provides an overview of CISA’s experiences with SSVC between Jan 2020 and June 2023.

Elizabeth Cardona is a vulnerability analyst in the Cybersecurity and Infrastructure Security Agency. Working within the Cybersecurity Division's Vulnerability Management Office, she specializes in vulnerability analysis. Elizabeth has helped implement the Stakeholder Specific Vulnerability Categorization (SSVC) and the Known Exploited Vulnerability Catalog (KEV) in CISA. As a former dentist and healthcare provider, Elizabeth has a unique perspective on cybersecurity connecting the human aspect and technology.

Jonathan Spring is a senior member of the technical staff in the CERT division of the Software Engineering Institute at Carnegie Mellon University. Dr. Spring's work focuses on producing reliable evidence in support of crafting effective cybersecurity policies at the operational, organizational, national, and Internet levels. Jono's research and practice interests include incident response, vulnerability management, machine learning, and threat intelligence.

Kevin Donovan Deputy Branch Chief, Vulnerability Response and Coordination (VRC) at Cybersecurity Infrastructure and Security Agency (CISA)

Prioritising Response

Which to fix, which to patch, which to investigate after an incident. How to choose between different vulnerabilities at different times.

Roel van der Jagt has worked in cyber security for 15 years with several Dutch MSSPs. Roel currently works for Tesorion in the role of CERT incident handler with T-CERT. This team can be best described as the firefighters of cyber security. Along with helping organizations during critical incidents, Roel really likes finding the vulnerability being exploited (if there is one).

Time and Magnitude:

Are CVEs changing over time? What is the magnitude of a vulnerability or it's exploitation

A key assumption to forecasting any quantity is that the underlying processes that will generate future data are the same processes that generated previous data. Unfortunately for vulnerability forecasting, this is not the case on multiple levels. Attackers and vulnerability researchers will change their efforts based on changes to the current software landscape. Standards bodies will change the way frameworks are structured in response to perceived and known shortcomings as well as in response to the shifting sands created by developers and researchers. Attacker interest in a vulnerability can bloom or wither depending on dynamics beyond our vision, such as out-of-band exploit dissemination.

How can forecasters work in this fluid landscape? The first step is attempting to identify when particular changes to the underlying structure have occurred. In this talk we’ll focus on the CVE process and related frameworks. We’ll show that a number of technical and procedural changes to the CVE, CWE, OWASP, and CVSS frameworks have altered the trajectory of vulnerability reporting and data. We’ll then dive into modeling techniques that can approximate both the timing and the magnitude of technical changes that impact data. In particular, we’ll present two regression techniques, segmented regression and generalized additive models, as examples of an approach to identifying structural changes to systems of data.

Dr. Benjamin Edwards is a security data scientist working at the Cyentia Institute. An expert in ML and statistics, Ben has led research on a variety of security topics including vulnerability management, application security, human risk, Next-gen SIEM, and security program performance. He is an active member of the security community, contributing to both EPSS and CVSSv4. Prior to joining Cyentia, his research examined global attack trends, the effects of security interventions, nation state cybersecurity policy, and the security of ML models.

Sander Vinberg is a Threat Researcher for F5 Labs. He is the project lead on many of F5 Labs’ intelligence products, including the 2023 Identity Threat Report, the Application Protection Report 2019-2022, the 2021 Credential Stuffing Report, and the Sensor Intelligence Series. Sander is a frequent speaker, having spoken at RSAC, SIRAcon, BSides, the Washington State CISO Forum, BCAware, (ISC)2 events, and Infragard meetings, among others. He holds a master’s degree in Information Management from the University of Washington, as well as bachelor’s degrees in History and African and African-American Studies from the University of Chicago.

Time and Magnitude:

Are CVEs changing over time? What is the magnitude of a vulnerability or it's exploitation

These days it feels like there is a new 'catastrophic' vulnerability every month. Is this actually the case, or simply a symptom of better information exchange and (maybe) some cyber marketing? We aim to examine and understand this dynamic by analyzing historical vulnerability datasets, cyber incident data, and public data feeds. If the rate of major vulns is increasing, when did this start? Where are we heading? Can we predict how many "big" vulnerabilities next year will bring? Additionally, it has been said that once a vulnerability has been found, a trail of follow-ons may often follow. Google reported that in 2022, 17 of 41 zero-day vulnerabilities were variants of previous zero-days. Anecdotally, we know that this behavior happens. But can we see this play out in a meaningful sense over years of vulnerability data? Do specific platforms exhibit this behavior more often than others? Furthermore - are we able to predict which vulnerabilities might be best suited for lots of variations, and might therefore require more complete solutions than targeted patches?

Matthew Berninger is a Principal Cyber Analyst for the Marsh McLennan Cyber Risk Intelligence Center. He has previously led teams in Detection and Response, Data Science, and Cyber Incident Response across private industry and within the U.S. Government. He has an M.S. in Cyber Warfare and Operations from the Naval Postgraduate School and a B.A. in Mathematics from Columbia University. He enjoys baseball, math, and baseball math."