Technology Today

2013 Issue 1

System-of-Systems Testbeds

Raytheon’s customers often demand innovative, low-cost, minimal-risk solutions with high technical readiness levels (TRLs1). Many new program pursuits require TRLs of 6 or greater just to compete. This is driving an industry trend toward greater reliance on early prototyping in high fidelity simulation testbeds to demonstrate technology maturity during the proposal and concept design phases.

Raytheon operates several rapidly configurable system-of-systems (SoS2) testbeds. They consist of commercial-off-the-shelf (COTS), government-off-the-shelf (GOTS) and Raytheon-developed models that are used to validate integrated system solutions at the mission level. These testbeds are used as software-, processor-, hardware-, and/or operator-in-the-loop simulation environments used to demonstrate how Raytheon’s solutions meet the needs of the warfighter and have the maturity needed to proceed to the next phase of development. The testbeds also provide an excellent early system development capability to rapidly find and fix system and system of systems integration issues.

Three Raytheon testbeds are highlighted in this article:

  • The Aerospace-Ground Integration (AGI) Testbed, supporting airborne and space platforms.
  • The Joint Force Interoperability and Requirements Evaluation SupraCenter (JFIRES), supporting integrated air and missile defense.
  • The Air Dominance TestBed (ADTB), supporting the weapon system kill chain.
Figure 1. The RM2AGIC white force room (main) is the truth viewing center where
customers watch scenarios unfold and operator performance in executing the mission. The
RM2AGIC blue force room (inset) is where operators rely on their skills and employed tactics
to best utilize blue force assets under their control and successfully complete their mission.

As U.S. Department of Defense (DoD) mission areas begin to overlap and their assets perform multimission roles, there is a need for integrating these mission-area-focused testbeds to provide a broader battlefield-wide simulation capability. Under a corporate Enterprise Modeling and Simulation (EMS) initiative, the first steps toward a more integrated capability were taken by creating the necessary network connectivity between sites; see article “Enterprise Modeling and Simulation (EMS): Enhancing Cross-Company Collaboration to Improve the Quality of Solutions that Raytheon can Offer Our Customers,” for more information on this topic.

Aerospace-Ground Integration (AGI) Testbed

The AGI Testbed was designed to support airborne platform program pursuits for a large customer base. It is currently being used to explore advanced concepts in the area of unmanned airborne systems (UAS). AGI Testbed capabilities are derived from targeted key capabilities across Raytheon; these include weapon models, sensor models, entity generators, communication models, fusion models and command and control algorithms. Raytheon has an eye toward adding immersive simulation expertise in the future.

Raytheon utilizes the AGI Testbed to show its customers how the integrated system solution meets the warfighter’s need. Such demonstrations are conducted at the appropriate security levels so that the effectiveness of system-of-systems solutions can be evaluated within their true context. Raytheon has multiple modeling and simulation (M&S) demonstration facilities where the customer is invited to observe and participate. Raytheon’s Mission Modeling Aerospace Ground Integration Center (RM2AGIC) is one such facility.

The RM2AGIC facility has two main areas for audience observation and participation with regard to live virtual demonstration. The first is the white room theater (Figure 1 main) where the audience can simultaneously view, on three digitally projected screens, operator displays, truth data and engineering displays. As such, the audience can simultaneously observe a) what has been “sensed or detected,” and subsequently shown on the operator’s display; b) what entities are present in the entire virtual scenario; and c) what is occurring at the detailed algorithm level. As the audience members listen and observe the live virtual exercise, they can change the conditions of the exercise at any given time unbeknownst to the blue force operators, who are physically isolated in a separate room. This allows the audience to observe an operator’s behavior as well as the effectiveness of the system when pop-up threats or other sudden changes are made to the environment.

Figure 2. The Raytheon Multiprogram
Testbed is a demonstration flight test
asset used to perform system hardware/
software integration and testing in flight.

Operators conduct the mission in the blue (friendly) force room (Figure 1 inset). Operators can only view data that has been “sensed” and is displayed in the common relevant operating picture (CROP). Because this is a non-scripted, real-time simulation, the scenario outcomes are not predetermined. Outcomes, therefore, vary depending on the skill of the operators, the tactics employed and the weapons systems available to both the blue and red (opposition) forces.

The ability to execute a live virtual demonstration has frequently resulted in follow-up demonstrations per customer request. In one instance, a customer wanted a better understanding of the advantages of employing multisensor and multiplatform sensor fusion algorithms in the context of his mission. A live demonstration was set up where sensors and platforms were added in real-time, under the control of the customer, enabling him to observe how fused track accuracies improved as the sensor/ platform asset mix was changed.

Once an airborne system solution has been verified in RM2AGIC, it is ready for hardware/software integration and testing in flight. Traditionally, the transition between the virtual environment and real hardware has been both time consuming and expensive. This transition is minimized by the advances of faster COTS hardware, the use of experimental test aircraft and the use of a prototype system that is encapsulated within the AGI testbed and employs the same hardware interface definitions as those used within the flight test asset. Following integration lab tests, the prototype system is typically exported to an experimental flight test asset such as the Raytheon Multiprogram Testbed (RMT, Figure 2). The RMT aircraft is equipped with multiple sensing systems and multiple computer racks, allowing Raytheon to demonstrate to the customer the effectiveness of its systems solutions in flight at TRL 6.

Joint Force Interoperability and Requirements Evaluation SupraCenter (JFIRES)

JFIRES is a multiservice, multitheater, multimission prototyping, evaluation and analysis environment with a focus on integrated air and missile defense (IAMD). High-fidelity, real-time, integrated digital simulations complement hardware and software-in-the-loop (HWIL/SIL) capabilities, providing an ideal testbed for SoS-, system-, and component-level evaluation. The scalability and flexibility of the JFIRES open architecture meet Raytheon’s rapid prototyping needs in bringing critical IAMD capabilities to the warfighter.

At the center of JFIRES infrastructure is a highly scalable distributed simulation layer consisting of low- to high-fidelity simulation models and HWIL/SIL elements capable of sharing common, operationally relevant scenarios. Simulated scenario data are exchanged in real time between models by using a COTS implementation of the Object Management Group (OMG®) Data Distribution Service (DDS). Built from this foundation of distributed data, simulated and actual tactical system software and hardware exchange information over simulated tactical data links. This flexibility, illustrated in Figure 3, allows for the rapid configuration and testing of sensors, weapon systems and command and control (C2).

Figure 3. The JFIRES simulation domain (top) provides scenario truth data that drives
the tactical software and hardware being evaluated in the tactical domain (bottom).

This configuration and test process highlight specific functionality of interest and have often been used to demonstrate new concepts for sensor, weapon and engagement resource management, joint track management and integrated fire control.

An example of the JFIRES testbed’s flexibility is a recent adaptation in support of space situation awareness program. On this program, the JFIRES environment was used to perform a set of experiments to assess the potential contribution of Missile Defense Agency (MDA) radars to space situational awareness, and to evaluate architecture alternatives for multimission sensor use.

The first of these JFIRES experiments (iterations) evaluated opportunities for using the basic coverage capabilities of radars to contribute to space situation awareness. The next experiment, using higher-fidelity radar models that included sensor slew and dwell dynamics, investigated radar resource management requirements. Finally, the last of these iterations evaluated multiple prototypes of a sensor resource manager (SRM). The SRM operates within a predefined concept of operations (CONOPS) and other constraints and maximizes sensor utilization efficiency in the gathering of both missile defense and space object data.

By leveraging the existing JFIRES simulation capability, the program rapidly completed the initial set of experiments and provided the customer with needed analysis data to assess the desirability of the SRM concept.

Without an existing simulation framework to build upon, excess funding and time would have been spent developing tools instead of producing analysis results. The study was one of over 20 rapid-response projects that JFIRES completed over the last five years, allowing studies like this one to focus more on the experiment and analysis phases and less on developing new simulation capabilities.

Air Dominance TestBed

A kill chain is the sequence of actions performed by a defense system to destroy an incoming threat. The events that define the kill chain include detection, location, tracking, targeting, engaging, and post-engagement assessment. Flight testing is the standard method used for making technology development and insertion decisions for airborne weapon systems. Although flight testing can potentially provide the most convincing validation, it is usually the most expensive method, especially if it involves multiple elements of the kill chain. M&S is often a cost-effective alternative to live fire exercises. Scenarios involving kill chain elements can be extremely complex and may include intricate system details; thus, it is advantageous to utilize simulation testbeds comprised of high-fidelity kill chain models.

The Air Dominance TestBed (ADTB) is one such simulation environment. Developed, maintained and operated by Raytheon, the ADTB is a national asset that has been supporting customers for over 20 years. The ADTB simulates the entire kill chain at the highest fidelity necessary in order to demonstrate performance to requirements and perform design of experiments and data-intensive analyses.

As shown in Figure 4, the ADTB includes a toolbox containing multiple simulation models that are integrated with tactically representative information processing and exchanges between models. Each model represents a tactical system, key function or key technology in the kill chain. A typical kill chain is implemented by integrating models for a surveillance asset, models for a manned tactical fighter aircraft, models for a weapon associated with the manned tactical fighter, and models for threats of interest. Additionally, warfighter tactics, kill chain interoperability and environmental factors are also implemented as models. Simulation control, data logging, tactical information exchange and entity kinematics are provided by Raytheon’s Air Combat Evaluation Model (ACEM).

Figure 4. The Air Combat Evaluation Model (ACEM) controls sensor, weapon and
other models in the Air Dominance Test Bed (ADTB). ADTB supports the analysis
of current and future kill chains, pre- and post-flight tests and the system engineering
of components.

Most of the models included in the ADTB toolbox are derivatives of the stand-alone models used by subject matter experts to validate requirements and predict flight test performance for their associated kill chain elements. When detailed models of specific kill chain assets are not available, or when high-fidelity is not necessary, the analyst can employ the lower-fidelity engagement-level models embedded within ACEM. These models are sufficient for quickly performing “what if” analyses. As questions of interest tend to vary from customer to customer, it is not uncommon to use a combination of lower- and higher-fidelity models in the course of an ADTB analysis effort.

Because each phase of the kill chain is modeled in the testbed, ADTB provides its customers with detailed and credible data. This helps customers make the right decisions on weapon system deployment, tactics and system upgrades. The ADTB has been used to assess cooperative engagement techniques in air-to-air engagements. The analysis uncovered potential limitations in the tactical code of one of the kill chain elements and eventually resulted in upgrades to the tactical software of the missile. In a similar instance, ADTB was used to generate performance predictions for an upcoming flight test. The preliminary analysis indicated that the desired test shot had a lower than expected chance of successful intercept. Upon further investigation of the testbed output, it was determined that some of the targeting data used by the launch platform was suboptimal for the intended test. Additional analyses were performed with the ADTB, which subsequently led to proposed modifications to the launch platform’s estimation of the target data. The probability of successful intercept improved and the eventual flight test was a success.

Data generated from the kill chain analysis process enables Raytheon and its customers to view the mission from a complete engagement-chain perspective. This, in turn, enables more informed decisions concerning technology development and program upgrades.

1 TRL is a measure used to assess the maturity of evolving technologies.
2 A SoS is defined as a set or arrangement of systems that results when independent and useful systems are integrated into a larger system that delivers unique capabilities [DoD Defense Acquisition Guidebook 2008].

Steve Baba, Tony Curreri, Russell W Lai,
Yuxiang Liu, Stelios Pispitsos and Tony Sabatino

Share This Story



Top of Page