13.3. Run the nodes using a multi-computer setup

In this tutorial, we’ll run things on 2 machines. You could run this tutorial setup theoretically in 3 machines:

fsw : Spacecraft machine that will run the cFS process.
rosgsw : Ground machine from which the user will send commands using ROS2 and will receive robot data.
rosfsw : Flight robot machine that runs ROS2 and is connected to the hardware (e.g. rover, arm)

For this tutorial, we’ll run 2 machines:

Ground machine: rosgsw
Flight machine: fsw and rosfsw

Note

If your machines are in the same network, ROS2 by default will make information such as topics visible among machines. To avoid this, set the ROS_DOMAIN_ID environmental variable to different non-zero values in your rosgsw and rosfsw machines (e.g. 20 and 21), such that by default they do not eavesdrop on each other.

13.3.1. Set up IP addresses:

Start by writing down the IP addresses of your 2 machines. You can use the ifconfig command in your terminal to get this information.

13.3.1.1. fsw machine

In your fsw machine, go to sample_defs/tables:

$ cd ~/test_brash/cFS
$ cd sample_defs/tables

And edit cpu2_sbn_conf_tbl.c . Modify the IPs for Self and rosfsw (processors 2 and 3) to be the IPs for your fsw and rosfsw machines. respectively. Leave the ports (2235, and 2236) as they are.

.Peers =
  {
     {/* [0] - Corresponds to cpu1 and not used in typical brash config */
       .ProcessorID  = 1,
       .SpacecraftID = 0x42,
       .NetNum       = 0,
       .ProtocolName = "UDP",
       .Filters      = {"Remap"},
       .Address      = "10.5.0.3:2234",
       .TaskFlags    = SBN_TASK_RECV},
     {/* [1] - Self */
       .ProcessorID  = 2,
       .SpacecraftID = 0x42,
       .NetNum       = 0,
       .ProtocolName = "UDP",
       .Filters      = {"Remap"},
       .Address      = "XXX.XXX.XXX.XXX:2235", // <== WRITE fsw's IP HERE !!!
       .TaskFlags    = SBN_TASK_POLL},
     {/* [2] - rosfsw */
       .ProcessorID  = 3,
       .SpacecraftID = 0x42,
       .NetNum       = 0,
       .ProtocolName = "UDP",
       .Filters      = {"Remap"},
       .Address      = "XXX.XXX.XXX.XXX:2236", // <== WRITE rosfsw's IP HERE!!!
       .TaskFlags    = SBN_TASK_POLL},
  },
.PeerCnt = 3};

Link Go to cFS/sample_defs/cpu2_cfe_es_startup.scr and add the line for this app as shown below:

CFE_APP, robot_sim,   RobotSimMain,    ROBOT_SIM,    50,   16384, 0x0, 0;
+ CFE_APP, simple_robot_app,   SimpleRobotAppMain,    SIMPLE_ROBOT_APP,    50,   16384, 0x0, 0;
CFE_APP, cf,          CF_AppMain,      CF,           80,   16384, 0x0, 0;
CFE_APP, sntp,        SNTP_Main,       SNTP,         80,   16384, 0x0, 0;

Feel free to delete the robot_sim and rover_app from this file, as we won’t use them. You can keep them as well.

Build cFS, so your changes take effect:

$ cd ~/test_brash/cFS
$ make && make install

13.3.1.2. rosfsw machine

Open a terminal and go to cfe_sbn_plugin in your brash workspace:

cd ~/test_brash/brash
cd src/cfe_sbn_bridge_plugin/cfe_sbn_plugin/config

Open the cfe_sbn_config_multihost.yaml file and change the value of peer_ip to be the IP of your fsw machine:

cfe_sbn_bridge:
  ros__parameters:
    plugin_name: cfe_sbn_plugin.cfe_sbn_plugin
    namespace: /flightsystem
    plugin_params:
      msg_pkg: cfe_msgs
      udp_receive_port: 2236
      udp_receive_ip: '0.0.0.0'
      spacecraft_id: 66
      processor_id: 3
      epoch_delta: 315532800
      rosout_name_max_length: 32
      rosout_msg_max_length: 128
      rosout_file_max_length: 64
      rosout_function_max_length: 32
      # fsw information
      peer_ip: "XXX.XXX.XXX.XXX"         # <== WRITE fsw's IP HERE !!!
      peer_port: 2235             # 2234 for cpu1, 2235 for cpu2
      peer_processor_id: 2        # Processor id (1 or 2, for cpu1 - singlehost setup, or cpu2 - multihost setup)

13.3.1.3. rosgsw machine

Open a terminal and go to cfe_plugin in your brash workspace:

cd ~/test_brash/brash
cd src/cfe_ros2_bridge_plugin/cfe_plugin/config

Open the cfe_config_multihost.yaml file and change the value of udp_command_ip to be the IP of your fsw machine:

cfe_bridge:
  ros__parameters:
    plugin_name: cfe_plugin.cfe_plugin
    namespace: /groundsystem
    plugin_params:
      udp_telemetry_ip: "0.0.0.0"
      udp_command_ip: "XXX.XXX.XXX.XXX" # <== WRITE fsw's IP HERE !!!
      udp_receive_port: 1235
      udp_send_port: 1235       # 1234 for cpu1 (singlehost setup), 1235 for cpu2 (multihost setup)

13.3.2. Running the tutorial in a distributed setup:

Flight machine : Run cFS. Make sure to use cpu2 :

$ cd ~/cFS/build/exe/cpu2
$ ./core-cpu2 2>&1

Ground machine : Run the cfe/ROS2 bridge:

$ cd ~/test_brash/brash
$ source install/setup.bash
$ ros2 launch cfe_plugin cfe_bridge.launch.py cfe_config:=cfe_config_multihost.yaml

Note that we are using the cfe_config argument to use the yaml file which we edited earlier to have our setup IPs.

Start the telemetry:

$ cd ~/test_brash/brash && source install/setup.bash
$ ros2 topic pub --once /groundsystem/to_lab_enable_output_cmd cfe_msgs/msg/TOLABEnableOutputCmdt "{\"payload\":{\"dest_ip\":\"XXX.X.X.X\"}}"

Ground machine : In another terminal, start the UI tools that will allow us to send commands and visualize data from the flight system:
```
$ cd ~/test_brash/brash
$ source install/setup.bash
$ ros2 run brash_application_tools simple_robot_app_ground
```
Flight machine : For this demo, the rosfsw machine is not doing any actual calculation or interacting with an actual robot, so here we’ll just use it to show how ROS2 data can be shown in the flight side using the SBN bridge. Go to the terminal:
```
$ cd ~/test_brash/brash
$ source install/setup.bash
$ ros2 launch cfe_sbn_plugin cfe_sbn_bridge.launch.py cfe_sbn_config:=cfe_sbn_config_multihost.yaml
```
Note that we are using the yaml file that we edited earlier to have our setup IPs.

Flight machine : Finally, run the flight node that simulates the actuator

$ cd ~/test_brash/brash
$ source install/setup.bash
$ ros2 run brash_application_tools simple_robot_app_flight