For decades, defense systems have been developed and deployed with a requirement for longevity, but generally without a high priority on cybersecurity. Traditionally, cybersecurity was not an identified need because these systems were typically limited in their external access and contained customized hardware and software. With the digital transformation, today’s defense weapon systems benefit from additional capabilities provided by more embedded processors, increasing volumes of software and external (or networked) connections. For example, Figure 1 illustrates a modern aircraft utilizing processors ranging from basic controllers to complex avionics systems, multiple internal networks and a variety of external connections. Along with these advancements comes the potential for increased vulnerabilities. A recent report from the Government Accountability Office revealed the routine discovery of mission-critical cyber vulnerabilities during operational testing.1
To reduce and mitigate these vulnerabilities, there is a growing emphasis on security risk management processes for defense systems. One example of this is the Department of Defense (DoD) cybersecurity Risk Management Framework (RMF) for integration of cybersecurity into its acquisition programs.2 However, as programs deploy RMF and other cybersecurity controls, there are sometimes unintended consequences. A requirement to log cyber-relevant data, for example, can create volumes of information that overwhelm system maintainers, who then might discard full system logs simply to keep up with the most recently generated data. The situation is more acute for those operating weapon and defense systems. For those warfighters, cyber threats requiring immediate detection and response can potentially interrupt an already complex set of tasks requiring constant attention. Even system designers, while having a longer timeline to create more resistant architectures, face related challenges. It is inevitable that the millions of lines of code in a typical operating system contain vulnerabilities, yet there is no method to efficiently evaluate all the risks. All of these factors contribute to cybersecurity automation being a rich area of research and a capability with the potential to enable humans to respond directly, rapidly and effectively to indicators of malicious activity.
Raytheon’s research in cybersecurity automation covers applications across the lifecycle of weapon systems—from design to deployment. This article describes some key areas of research, providing examples of automation technologies applied to both protecting operational weapon systems and hardening software baselines during design and maintenance.
Automated Event Detection in Embedded Bus Systems
Weapon platforms and systems as well as many complex commercial systems often utilize embedded architectures with processors connected through local networks, backplanes and/or serial buses that communicate via protocols not having built-in security measures. Protecting these embedded systems starts with security functions found in traditional security products such as policy enforcement, intrusion detection, and event log collection and analysis. While these mechanisms are effective for external connections found on ships, aircraft and unmanned aerial vehicles (UAVs), and other Internet Protocol-based communications, the currently available versions of these services often fall short in addressing unique aspects of embedded processing systems. For example, available IP-based security applications are incompatible with the unique protocols and formats used on embedded system backplanes and buses. Also, communications in these environments are typically time-critical, where a delay of a few microseconds in packet delivery can cause undesirable effects. Another important consideration is that weapon system computers are designed to interact with the physical world. They often actuate control surfaces, direct sensors, fire weapons and control propulsion. Any security related applications in this area must ensure system state/context are accounted for.
Raytheon’s research in cyber event detection for MIL-STD-1553B serial data bus systems is addressing some of the challenges that come with these embedded systems. The MIL-STD-1553B is a serial network found on most military and commercial aircraft, as well as some ships and ground vehicles. The focus of Raytheon’s research is to prototype and evaluate various methods for detection and alert of anomalous events that might indicate a cyberattack.
An early discovery of this research was that one aspect of an embedded bus (or network) that helps in cyber hardening is the predictability of the environment. For example, mapping out the processors that exchange requests and responses can expose consistent patterns of communication. This consistency, when coupled with well-defined message formats, means data within frames of information exchanged between any two processors will fall within a predictable range, making anomalous messages that much easier to detect.
Once an anomaly is detected, characterizing the threat and the appropriate response to that threat becomes more difficult. Early prototypes took a “check engine light” approach. Similar to an aircraft’s radar warning receivers, the detector provided an alert with no direct action or evaluation of mission impact. An invaluable tool used in the ongoing research and prototyping is Raytheon’s in-flight cyber effect demonstrator (see Figure 2), built to test and evaluate the technology as well as solicit feedback from the customer community. The demonstrator provides a simulated aircraft environment for injection of cyber effects to evaluate various response scenarios and the effectiveness of the display interface. Feedback from warfighters seeing the demonstration has included the question of false positives. In other words, how do you limit authentic but rare communications, perhaps those occurring during a complex battle scenario, from being identified as potential threats? Many have also asked for additional context with an alert, such as the impact or severity of the threat to the mission.
Raytheon utilizes both automated analytics and machine learning (ML) techniques to explore improved methods of threat characterization and response. By continuously analyzing inter-processor communications and associated system state, the analytics algorithms are able to develop behavioral patterns and identify system activity indicative of a cyber threat in near real-time. ML algorithms can be trained using data from both authentic-but-rare system operating sequences and realistic cyber-attack sequences. These algorithms are then used to develop models which are more effective in not only detecting and characterizing a particular cyber threat but also in reducing the likelihood of a false positive.
Another important aspect of Raytheon’s research in automated event detection is automated (or assisted) operator response to a cyber threat once it has been identified. Pilots and other weapon system operators are subject to high cognitive loads, so it is critical that response interaction be rapid and effective while providing little to no distraction to the mission at hand. Raytheon user experience experts are developing various methods of alert notification, which will include parameters such as severity of impact and degree of certainty along with alternative courses of action. The goal of this effort is to provide operators with positive and actionable information, while allowing them to maintain their overall mission focus.
Automating Secure Configuration of Complex Systems in Continuous Development
In agile development environments, the “velocity” of software development and delivery is a key measure of performance. Ensuring cybersecurity in the process can be particularly challenging. For example, it’s not uncommon for security verification of a complex system to include checking the values and associations of tens of thousands of configuration settings. Moreover, the requirements and standards which govern a secure configuration, such as the Defense Information Systems Agency (DISA) Secure Technical Implementation Guide (STIG), are continually evolving to meet the ever changing nature of today’s cyber threat. Raytheon cyber researchers have developed an approach, incorporating both process and toolset to automate cyber hardening in this dynamic environment.
The primary tool used in this approach is Stigler, a software utility that automates the application of thousands of hardening rules necessary to secure both operating systems and computing infrastructure. Stigler retrieves the STIG from the DISA website, extracts the rulesets and generates script code which can then be applied to harden the target system (Figure 3). Hardening of a complex system involves applying tighter security through configuration settings, generally used to enforce principle of least privilege (PoLP) and mitigate vulnerabilities. PoLP is an approach which limits a general user’s access level and privileges on a system to only those necessary to complete his/her assigned work activities. In addition to the STIG, Stigler can also ingest and translate from other sources of policy such as Security Content Automation Protocol (SCAP)3 files.
In order to confirm that a system remains securely configured, many systems use scanning tools like Nessus® or Assured Compliance Assessment Solution4 (ACAS) for regular and independent verification. Stigler can also translate the STIG rulesets into scan policies in the formats required by these tools, which is important in ensuring that the scanner is using the same version of the STIG.
Stigler automates the painstaking tasks of policy translation, critical system configuration programming and verification, repeating these tasks across any number of systems.
Automated Self-Healing Systems Research
The software used in future weapon and defense systems should be self-diagnosing and self-healing and, during normal operation, have the ability to automatically find, assess and repair its own vulnerabilities and/or unauthorized modifications. This was part of the Defense Advanced Research Projects Agency’s (DARPA) vision for the historic Cyber Grand Challenge competition in 2016,5 where competitors’ systems simultaneously defended, analyzed, patched and attacked without human interaction (Figure 4). Having participated as a finalist in that event, Raytheon has continued to build on the experience, researching automated self-healing techniques that can be applied both to development environments and operational systems.
A recent effort in this research includes a new tool, “Chipper,” that matures and extends some of the features proven in the Cyber Grand Challenge. Chipper automates analysis, assessment and patching of a code base. It starts by factoring programs into composable pieces, and then it orchestrates a combination of static and dynamic analysis tools to evaluate the pre-processed code. This results in more efficient discovery of software vulnerabilities, especially when examining a large code base that utilizes multiple processors. When a potential vulnerability is identified, the tool tests the exploitability of the flaw. Exploitability can be thought of as a measure of risk, which is then used in an automated software triage where easily exploited flaws undergo immediate mitigation while those that cannot be triggered are deferred. For high risk vulnerabilities, Chipper will postulate and test fixes, and then patch the operational code base.
Initial implementations of this tool are in homogenous environments, covering chipsets and operating systems of interest to select Raytheon customers – an important early step toward self-healing systems able to diagnose and fix themselves without loss of service.
Raytheon Research with the Army Research Lab (ARL)
Raytheon employees working with the ARL are performing more basic research into automated cyber defense, including two diverse applications of machine learning.
One example of this work involves the Semi-Supervised Learning for Exploits and Exploit Kits (SSLEEK). SSLEEK is an extensible set of network intrusion detection tools, combining the pre-processing of IP packets with various machine learning algorithms. These efforts have focused mainly on the detection of botnets, which are cooperative instances of malicious software distributed around a network of interest. Recently published findings of this research, using the benchmark CTU-13 data set, show promising results for a combination of pre-processing the packets to calculate an IP address distance metric, and applying a k-NN (k-Nearest Neighbor) machine learning model.6 K-means and Gaussian Mixture Models were also evaluated, but neither showed the consistency of k-NN to this use case.
Another area of research addresses alert response, a common challenge in cybersecurity operations. As events occur, prioritizing alerts can be difficult since the potential impact of a cyber threat greatly depends on the context in which it happens. Rather than defining a priori the attributes of an authentic and high priority event, the ARL team investigated methods for having the environment determine which events need attention. Starting with a large data set consisting of Intrusion Detection System (IDS) alerts from an operational Department of Defense (DoD) system, the team cross-referenced the initial alerts with documented incident reports and used these reports as the baseline for a training set. This data was further processed to extract 23 attributes of each alert and then used to train separate Adaboost and Random Forest machine learning models. When run against test datasets, the team found that a composite of the two models was the most successful, reducing false alarms by greater than 90%, and correctly identifying greater than 90% of the true alerts. These results demonstrated that having this type of system in place would allow analysts to spend more time looking for novel attacks and following up leads.7,8
Raytheon recognizes the challenges of defending cyberspace as the domain expands within the defense industry, and the value automation can bring in making operators, analysts and developers more efficient in eliminating cyber vulnerabilities. Our research spans the entire lifecycle of weapon and defense systems in areas ranging from advanced algorithms to the operational systems using those algorithms, to the automated development and maintenance of tools currently being piloted by Raytheon engineers.
— Jon Goding
— Heather Romero
Nessus® is a registered trademark of Tenable Network Security, Inc.
1 WEAPON SYSTEMS CYBERSECURITY: DoD Just Beginning to Grapple with Scale of Vulnerabilities, GAO-19-128 (2018). U.S. Government Accountability Office, https://www.gao.gov/products/GAO-19-128.
2 DoD Program Manager’s Guidebook for Integrating the Cybersecurity Risk Management Framework (RMF) into the System Acquisition Lifecycle (2015), Office Of The Under Secretary Of Defense For Acquisition, Technology, And Logistics, Washington, D.C.
6 Leslie, N. O. (2018). Using Semi-Supervised Learning for Flow-Based Network Intrusion Detection. In Proceedings of the 23rd International Command and Control Research and Technology Symposium (ICCRTS): Multi-Domain C2. Pensacola, FL: ICCRTS.
7 Shearer, G., Leslie, N. O., Nelson, F. (2018). Integrating Human Knowledge in a Semi-Autonomous Prioritization System: An Approach for Improving Network Intrusion Detection Efficiency. In Proceedings of the 23rd International Command and Control Research and Technology Symposium (ICCRTS): Multi-Domain C2, 6-9 November 2018. Pensacola, FL: ICCRTS.
8 Shearer, G., Leslie, N. O., Ritchey, P., Braun, T., Nelson, F. (2017). IDS Alert Prioritization through Supervised Learning. In Proceedings of the NATO Specialists’ Meeting on Predictive Analytics and Analysis in the Cyber Domain, 10-11 October 2017, Sibiu, Romania: NATO.