S. Raj Rajagopalan is a Senior Principal Research Scientist at Honeywell Research, where he is tasked with creating sensible technological enhancements to Honeywell’s vast portfolio of control systems to help defend against attacks exemplified by Stuxnet and Flame. He is working closely with the various business units in Honeywell Automation and Control Systems to study and address the problems of market addressability, lifecycle deployment, and cost-effectiveness of cybersecurity solutions. He is also working with the Security Operations Center organization in Honeywell Global Security where he is embedded with a view to studying and analyzing SOC processes as well as to bring Labs research in-house to improve incident investigation and digital forensics techniques. Prior to joining Honeywell, Dr Rajagopalan worked with HP Labs Security Research Group based in Princeton, NJ where he collaborated with Mr. Daniel Moor who is a senior member of HP's Managed Security Service Provider Business Unit to provide appropriate technology such as better visualization and threat ranking. He has helped transition solutions as needed for HP Enterprise Security Product Units such as Tipping Point and ArcSight. Before joining HP Labs Dr. Rajagopalan started his career at Bellcore (later Telcordia) where he worked on building firewall management tools with DARPA sponsorship that earned him several awards. In recent years he has focused his research on inter-disciplinary research. He was on two DHS S&T awards, one on applying tools from Industrial Psychology to study team dynamics in CSIRTs (with George Mason University) and another on applying social insect models from Biology to distributed intrusion detection (with Rutgers University).
Dr. Xinming "Simon" Ou is associate professor of Computer Science at Kansas State University. He received his PhD from Princeton University in 2005. Before joining Kansas State University in 2006, he was a post-doctoral research associate at Purdue University's Center for Education and Research in Information Assurance and Security (CERIAS), and a research associate at Idaho National Laboratory (INL). Dr. Ou's research interests focus on designing better technologies to facilitate cyber defense. He is a recipient of 2010 NSF Faculty Early Career Development (CAREER) Award, and three-time winner of HP Labs Innovation Research Program (IRP) award.
The most critical assets in guarding the nation from cyber terrorists are our cyber defenders -- these are the security analysts, security center operators, incident investigators, etc. who are at the front line of our national defense, working ceaselessly in commercial, academic and government security operations centers (SOCs) to detect, repel, and prevent cyber intrusions into the sprawling cyber infrastructure. Like in any war, our assets need to have adequate arsenal, and have to be trained and re-trained to keep up with the enemy. Both have been extremely challenging. It is now well accepted that training cyber analysts is a long and hard process, and something that keeps us from expanding our defensive operations to match the enormity of the task. Government agencies, commercial SOCs, private consulting companies, universities are all suffering from a severe shortage of trained personnel in this area and yet there is no clear large-scale program in place to train security analysts. Our current approach to training is in an individual one-on-one mode that does not scale. Indeed, as we discovered in many SOCs, no systematic training is taking place at all and this is even culturally accepted as a fact of life for working in those organizations. Adding to this is the lack of effective tools, both commercially and from the research community, to help analysts do their job more efficiently. Without adequate arsenal and training, one is not expected to win in any war.
We took the novel approach of trying to study the problem in its native environment, i.e. rather than study the problem of training as an abstract concept in a lab, we aim to observe and study the difficult art of analysts' job in the security operations center itself. Furthermore, in order to study the rather human problem of learning and teaching cyber analysis, we are applying tools and techniques from the humanities, in particular socio-cultural anthropology. Anthropologists use the term "tacit knowledge" to capture the idea that a lot of the knowledge that people use in their jobs is inside their heads and not written down or documented anywhere. This is especially true in SOCs; indeed many times the analysts who possess the knowledge do not even know how to express it. Added to that is the natural censure against sharing of sensitive information and the cultural belief that one has to learn his/her way through the roughs, and we have a "tribal" knowledge regime wherein critical knowledge is only transferred from human to human through long apprenticeships and inter-personal/trust relationships.
Working with a professional anthropologist, our strategy is to have student interns embedded in various security operations centers with the goal of "learning by participation." Our embeds participate in the SOC just like regular employee trainees and keep detailed field notes on what they observe in their day to day interactions. These analysts process large amounts of data under time-stress conditions when handling cyber threats. The job requires intelligence and high levels of skills but has many mundane/repetitive aspects as well. Adequate tool support is largely lacking and many of the skills and procedures involved are un-codified and undocumented, resulting in a large body of tacit knowledge. We place computer science and anthropology researchers and graduate students trained in both fields into SOCs, working side by side with the analysts. This "participant observation" approach developed in socio-cultural anthropology provides a method and means to access the tacit knowledge of the analysts and to convert it into more explicit knowledge, leading to the development of algorithms that can help automate the tasks. More importantly, the ethnographic fieldwork provides an opportunity to observe real security operation centers' work processes and identify factors that influence the effectiveness and efficiency with which cybersecurity incidents are handled. This may help explain why some cybersecurity problems are hard to address in practice, what roles humans and organizational structures play, and where procedures might be inefficient or completely fail for non-technical reasons. The research is carried out through a collaborative effort involving researchers from Kansas State University and two companies, Honeywell and RedJack, LLC. Results from the research will not only create practical tools that leverage tacit knowledge in security analytics and automate/aid tasks in incident response and forensic analysis, but also informs the training of cybersecurity professionals by making explicit the tacit knowledge of effective security analytics acquired during participant observation.
Our belief is that, rather than using the traditional approach of trying to build tools to address the problems that appear on the surface, we want to study the deeper relationship between the requirements of the job, the incentives and disincentives in the work environment, and the unique attributes of cyber security operations. Thus far we have conducted the fieldwork at Kansas State University's Office of Information Security and Compliance for about a year, where multiple PhD students work side by side with the security analysts in their daily duties. We have identified significant gaps in the tool support that no existing commercial or open-source solutions address, and we have been building tools to help the analysts' job and using the tool building as a means to open up the discussion on the technical details of their job that constitute the tacit knowledge. This has been a highly beneficial experience for both the researchers and the analysts. The tools that resulted from the discussion have tremendously improved the analysts' work performance (by their evaluation) and reduced the amount of labor they would have to put in to perform the repetitive, low-level, mundane tasks. This has also allowed the analysts to focus more of their effort to investigate sophisticated attacks, which in turn fosters more discussion with the research team and thenceforth to more tacit-to-explicit knowledge conversion. This will be extremely helpful in informing what types of training are most effective for new analysts so that they can more quickly acquire the deep analytical skills and not be overwhelmed by the low-level repetitive processes that can be automated.
We are now extending and expanding this effort to studying more SOCs, including the commercial SOC at Honeywell. We would like to find more partners to work with us, so that our study can be more representative and valid. In particular, we will need help to access more SOCs to conduct the fieldwork. What we will need from our collaborators is to dedicate some human resources for doing this fieldwork. For academic collaborators, this could mean sending some students to SOC(s) and having regular meeting with the whole research team to exchange the findings produced from the fieldwork. For industry collaborators, this could mean having an analyst working with the student fieldworkers (apprentices) to train them in doing the job. The collaborating organizations will benefit from a third-party perspective of operational effectiveness, intra-team interactions, and other organizational attributes in the context of cyber security operations. They may also benefit from any tools that the fieldworkers build or help build specifically for the organization. At the end of the project, we expect to write a training manual with do's
and don'ts for organizations employing cyber security operations personnel that are common to such organizations across businesses, academia, and governmental agencies. Collaborating organizations can also benefit by contributing to the framing and prioritization of issues to be addressed in such a manual as well as early access to learning.
Any study such as this can only improve with more participation. We hope to invite the FIRST community of SOC analysts and managers alike to participate in our study and making the derived outcomes truly global.
This work is supported by the National Science Foundation under Grant No. 1314925. Any opinions, findings and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.
