Integration Candidate: 2020-04-15

.gitignore

@@ -1,2 +1,3 @@
 *.pyc
 cmdUtil
+.DS_Store

Guide-GroundSystem.md

@@ -0,0 +1,178 @@
+# cFS Ground System Version 2.1.5
+
+## cFS Ground System Info
+
+The GroundSystem directory contains the new Ground System project for the cFS, that incorporates the main window to launch commands and telemetry systems, and other utilities like FDL/FUL and FT managers to send and receive files. The main window runs alongside the Routing Service (`RoutingService.py`). The Routing Service handles all incoming data and publishes (PUB/SUB) the data to specific ZeroMQ channels so that the different ground system utilities can receive (subscribe) only the desired data.
+
+This ground system supports commanding and receiving telemetry from multiple spacecrafts using UDP.
+
+The Ground System contains the main window that lets you launch the different utilities.
+
+To start receiving data from the cFS, you need to enable telemetry first. To enable telemetry:
+- click the "Start Command System" button from the main window,
+- then from the Command System Main Page click the "Enable Tlm" button (you will need to enter the target/destination IP address as an input to this command).
+
+Note: The Main Window needs to be opened at all times so that the telemetry messages can be forwarded to the Telemetry System.
+
+The Ground System will automatically detect the spacecraft when it starts sending the telemetry, and it will be added to the IP addresses list. You can select the spacecraft from the list, and start Telemetry System to receive its data. If 'All' spacecrafts are selected, you can start Telemetry System to display the packet count from multiple spacecrafts (if it detected more than one).
+
+Future enhancements:
+1. Detect different spacecrafts based on telemetry header (spacecraft `id`) data instead of using the spacecraft IP address.
+1. Add instructions for Windows.
+
+## Install and run
+
+Before launching the Ground System make sure that:
+- PyQt4 is installed,
+- PyZMQ is installed,
+- cmdUtil is compiled.
+
+Installing and running cFS Ground System on Ubuntu:
+
+1. ```sudo apt-get install python3-pyqt4```
+1. ```sudo apt-get install python3-zmq```
+1. ```sudo apt-get install libcanberra-gtk-module```
+1. ```cd Subsystems/cmdUtil/ && make```
+1. ```python3 GroundSystem.py```
+
+The historically included instructions for running on macOS or CentOS are included at the bottom of this document for reference.
+Please note that instructions have not been maintained.
+Welcoming instruction contributions if any of these are your platform of choice.
+
+## Adding new flight software application to ground system command GUI
+
+This section was made to help developers who are adding core Flight Software (cFS) Applications to the Python-based Ground System that comes with this cFS distribution.
+
+The `CHeaderParser.py` program that should be found in:
+
+```GroundSystem/Subsystems/cmdGui```
+
+Is an interactive, command-line based program to help walk developers through the process of adding custom cFS applications to the Ground System. Along with `CHeaderParser.py` is a configuration file that CHeaderParser uses to find the proper header files for your "new" cFS application. This file is named `CHeaderParser-hdr-paths.txt`, and should be placed in the same directory as `CHeaderParser.py`.
+
+Expected file structure:
+```
+cFE-6.4.x-OSS-release/cfe/tools/cFS-GroundSystem/Subsystems/cmdGui/CHeaderParser.py
+cFE-6.4.x-OSS-release/cfe/tools/cFS-GroundSystem/Subsystems/cmdGui/CHeaderParser-hdr-paths.txt
+cFE-6.4.x-OSS-release/cfe/tools/cFS-GroundSystem/Subsystems/cmdGui/CommandFiles/
+cFE-6.4.x-OSS-release/cfe/tools/cFS-GroundSystem/Subsystems/cmdGui/ParameterFiles/
+cFE-6.4.x-OSS-release/cfe/tools/cFS-GroundSystem/Subsystems/cmdGui/command-pages.txt
+```
+Steps to adding application commands to the Ground System:
+
+1. Edit `CHeaderParser-hdr-paths.txt`:
+   1. Locate any header files that contain command code definitions or command structure definitions. These files typically end in `*app_msg.h` or `*app_msgdefs.h` but could be named anything.
+   1. Add each one of the paths to a new line in `CHeaderParser-hdr-paths.txt`.
+   1. Comment out any paths/lines that aren't needed with `#` (at the beginning of the line).
+
+1. Run CHeaderParser:
+   1. Call CHeaderParser using python: `python3 CHeaderParser.py`
+   1. The program will prompt you to enter a filename for the application. This will create a pickle file for your application named `CommandFiles/<user_defined_name>`. Notice that this file will be stored in the `CommandFiles` directory. This same filename will be used in `command-pages.txt` later.
+   1. Type `yes` if any commands in your application have parameters. The program will then look through the provided header files for `definitions.pick` which-ever definitions describe related command codes (one at a time, the program will prompt you for the next command code after all parameters have been added for the current command).
+   1. Select the appropriate command structure for the selected command. The program will show all structures that it could find in the provided header files. Enter the index of the command structure (the corresponding index should be above the command structure).
+   1. Select any parameters from the structure that apply. Once you have selected all applicable lines from the command structure, enter `-1` to finish. This will create a pickle file for the command/parameters named `ParameterFiles/<command_name>`. Notice that this file will be stored in the `ParameterFiles` directory.
+
+1. Update `command-pages.txt` (CSV):
+   1. Column 1 - Title of your application (whatever you want it called).
+   1. Column 2 - filename of your application (chosen in Step 2.ii) under `CommandFiles` directory.
+   1. Column 3 - Message ID for Application Commands (typically defined in `mission_msgids.h`).
+   1. Column 4 - Endianess (default little endian: `LE`).
+   1. Column 5 - PyGUI Page (default: `UdpCommands.py`).
+   1. Column 6 - Command Send Address (default: `127.0.0.1`).
+   1. Column 7 - Command Send Port (default: `1234`).
+
+   Notes:
+   - USE ONLY SPACES, NO TABS (Remember, it's Python).
+   - Don't leave any empty lines in `command-pages.txt`, this could cause errors when running `GroundSystem.py` and `CommandSystem.py`.
+
+After completing these steps, restart the Ground System and the changes should have taken affect.
+
+## Common issues and troubleshooting
+
+### Issue: Cannot Send Command, receiving the "[Errno 8] Exec format error"
+
+Traceback:
+
+```
+Calling cmdUtil from Parameter.py
+Traceback (most recent call last):
+  File "Parameter.py", line 100, in ProcessSendButton
+    subprocess.Popen(cmd_args, stdout=subprocess.PIPE)
+  File "/usr/lib/python3.7/subprocess.py", line 642, in __init__
+    errread, errwrite)
+  File "/usr/lib/python3.7/subprocess.py", line 1234, in _execute_child
+    raise child_exception
+OSError: [Errno 8] Exec format error
+```
+
+Fix: This problem is most likely caused by calling `cmdUtil` without compiling it.  This issue has also been found to occur when the `cmdUtil` executable has been compressed/decompressed. To fix this problem, use the `Makefile` inside of the `cmdUtil` directory to compile or recompile (in the case after decompressing).
+
+```
+$ cd Subsystems/cmdUtil
+$ make
+$ cd ../..
+$ python3 GroundSystem.py
+```
+
+## Historically included instructions for running on macOS or CentOS
+
+### Installing and running cFS Ground System on macOS, using Homebrew
+
+```
+$ brew install pyqt
+$ brew install zeromq
+$ ( cd Subsystems/cmdUtil/ && make )
+$ python GroundSystem.py
+```
+
+### Installing and running cFS Ground System on CentOS 6
+
+#### Update yum
+```
+$ su
+$ <type password>
+$ yum -y update
+```
+
+#### Install pip and python-development ####
+If you are on a 64-bit CentOS / RHEL based system:
+```
+$ wget http://download.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm
+$ rpm -ivh epel-release-6-8.noarch.rpm
+```
+
+If you are on a 32-bit CentOS / RHEL based system:
+```
+$ wget http://download.fedoraproject.org/pub/epel/6/i386/epel-release-6-8.noarch.rpm
+$ rpm -ivh epel-release-6-8.noarch.rpm
+$ yum install -y python-pip
+$ yum install -y python-devel
+```
+
+#### Install zeroMQ and pyZMQ messaging system ####
+```
+$ yum install -y uuid-devel
+$ yum install -y pkgconfig
+$ yum install -y libtool
+$ yum install -y gcc-c++
+$ wget http://download.zeromq.org/zeromq-4.0.5.tar.gz
+$ tar xzvf zeromq-4.0.5.tar.gz
+$ cd zeromq-4.0.5
+$ ./configure
+$ make
+$ make install
+$ echo /usr/local/lib > /etc/ld.so.conf.d/local.conf
+$ ldconfig
+$ pip install pyzmq
+```
+
+#### Install pyQT4 ####
+```
+$ yum install -y PyQt4
+$ yum install -y qt qt-demos qt-designer qt4 qt4-designer
+```
+
+
+#### Running Ground System ###
+```
+$ python GroundSystem.py
+```

Guide-GroundSystem.txt

@@ -1,4 +1,4 @@
-cFS Ground System Version 2.1.6
+cFS Ground System Version 2.1.7
 
 cFS Ground System Info:
 

README.md

@@ -6,31 +6,42 @@ This lab application is a non-flight utility ground system to interact with the
 
 See Guide-GroundSystem.txt for more information.
 
-## Version Notes
+## Version History
 
-- 2.1.6: DEVELOPMENT
-  - Cmd code (and checksum) are always in the same place 
+##### Development Build: 2.1.7  
+  - Commands and Telemetry definitions now match code
+  - See https://github.com/nasa/cFS-GroundSystem/pull/74
+
+##### Development Build: 2.1.6
+  - Cmd code (and checksum) are always in the same place
   - See https://github.com/nasa/cFS-GroundSystem/pull/69
-- 2.1.5: DEVELOPMENT
+
+##### Development Build: 2.1.5
   - Updated build instructions for Python 3
   - See https://github.com/nasa/cFS-GroundSystem/pull/64
-- 2.1.4: DEVELOPMENT
+
+##### Development Build: 2.1.4
   - Finish conversion to python 3
   - cmdutil now accepts --word as alias to --long
   - See https://github.com/nasa/cFS-GroundSystem/pull/54
-- 2.1.3: DEVELOPMENT
+
+##### Development Build: 2.1.3
   - Minor updates to work with python 3
   - No longer compatible with python 2.7
   - Note issue #50 is to update the related documentation
   - See https://github.com/nasa/cFS-GroundSystem/pull/47
-- 2.1.2: DEVELOPMENT
+
+##### Development Build: 2.1.2
   - Minor updates (see https://github.com/nasa/cFS-GroundSystem/pull/39)
-- 2.1.1: DEVELOPMENT
+
+##### Development Build: 2.1.1
   - Minor updates (see https://github.com/nasa/cFS-GroundSystem/pull/36)
-- **2.1.0 OFFICIAL RELEASE**:
+
+### ***OFFICIAL RELEASE 2.1.0***
   - Minor updates (see https://github.com/nasa/cFS-GroundSystem/pull/26)
   - Released as part of cFE 6.7.0, Apache 2.0
-- **2.0.90a OFFICIAL RELEASE**:
+
+### ***OFFICIAL RELEASE 2.0.90a***
   - Released as part of cFE 6.6.0a, Apache 2.0
 
 ## Known issues

Subsystems/cmdGui/CHeaderParser-hdr-paths.txt

@@ -15,3 +15,9 @@
 #
 #../../../../apps/to_lab/fsw/src/to_lab_msg.h
 #../../../../apps/ci_lab/fsw/src/ci_lab_msg.h
+#../../../../apps/sample_app/fsw/src/sample_app_msg.h
+../../../../cfe/fsw/cfe-core/src/inc/cfe_es_msg.h
+#../../../../cfe/fsw/cfe-core/src/inc/cfe_time_msg.h
+#../../../../cfe/fsw/cfe-core/src/inc/cfe_evs_msg.h
+#../../../../cfe/fsw/cfe-core/src/inc/cfe_sb_msg.h
+#../../../../cfe/fsw/cfe-core/src/inc/cfe_tbl_msg.h

Subsystems/cmdGui/CHeaderParser.py

@@ -435,7 +435,7 @@ def getFileList(self, filename='CHeaderParser-hdr-paths.txt'):
 
 				# If no type could match, ask user for data type
 				if dataTypeNew == '':
-					typeNew = parser.findKeyword(dataTypesOrig[-1])
+					#typeNew = parser.findKeyword(dataTypesOrig[-1])
 					dataTypeNew = input('Data type for '+paramNames[-1]+' not found. Please enter new data type by hand: ')
 
 				dataTypesNew.append(dataTypeNew)

Subsystems/cmdGui/HTMLDocsParser.py

@@ -66,9 +66,10 @@ def findDataTypeNew(self, dataTypeOrig, paramName, paramLen, paramDesc):
 	# Determines character array size for string types
 	#
 	def findStringLen(self, keyword):
-		hdr_files = glob.glob('../../../build/cpu1/inc/*.h')
+		hdr_files = glob.glob('../../../build/inc/*.h')
+		hdr_files += glob.glob('../../../build/cpu1/inc/*.h')
+		hdr_files += glob.glob('../../../build/native/inc/*.h')
 		hdr_files += glob.glob('../../fsw/cfe-core/src/inc/cfe_*.h')
-		hdr_files += glob.glob('../../fsw/mission_inc/cfe_mission_cfg.h')
 		val = ''
 		found = False
 		i = 0

Subsystems/cmdGui/command-pages.txt

@@ -21,6 +21,17 @@
 #  Note(1): Lines starting with a # will be skipped
 #  Note(2): Please do not leave a blank line at the end of the file!
 #
+Executive Services,         CFE_ES_CMD,         0x1806, LE, UdpCommands.py,  127.0.0.1,   1234
+Software Bus,               CFE_SB_CMD,         0x1803, LE, UdpCommands.py,  127.0.0.1,   1234
+Table Services,             CFE_TBL_CMD,        0x1804, LE, UdpCommands.py,  127.0.0.1,   1234
+Time Services,              CFE_TIME_CMD,       0x1805, LE, UdpCommands.py,  127.0.0.1,   1234
+Event Services,             CFE_EVS_CMD,        0x1801, LE, UdpCommands.py,  127.0.0.1,   1234
+Command Ingest,             CI_LAB_CMD,         0x1884, LE, UdpCommands.py,  127.0.0.1,   1234
+Telemetry Output,           TO_LAB_CMD,         0x1880, LE, UdpCommands.py,  127.0.0.1,   1234
+Sample App,                 SAMPLE_APP_CMD,     0x1882, LE, UdpCommands.py,  127.0.0.1,   1234
+Spare,                                    ,     0x0000, LE, UdpCommands.py,  127.0.0.1,   1234
+Spare,                                    ,     0x0000, LE, UdpCommands.py,  127.0.0.1,   1234
+LEGACY DEFINITIONS,                       ,     0x0000, LE, UdpCommands.py,  127.0.0.1,   1234
 Executive Services (CPU1),  cfe__es__msg_8h,    0x1806, LE, UdpCommands.py,  127.0.0.1,   1234
 Software Bus (CPU1),        cfe__sb__msg_8h,    0x1803, LE, UdpCommands.py,  127.0.0.1,   1234
 Table Services (CPU1),      cfe__tbl__msg_8h,   0x1804, LE, UdpCommands.py,  127.0.0.1,   1234

Subsystems/cmdGui/quick-buttons.txt

@@ -20,6 +20,14 @@
 # Note(3): Subsystem name must match name in command-pages.txt exactly
 # Note(4): Command name need not match name in command definition file; shorten name if possible
 #
-Telemetry Output LAB,       to_lab_msg,         Enable Tlm,    6, 0x1880, LE, 127.0.0.1, 1234, TO_OUTPUT_ENABLE_CC
+Telemetry Output LAB,       to_lab_msg,         Enable Tlm,    6, 0x1880, LE, 127.0.0.1, 1234,  TO_OUTPUT_ENABLE_CC
 Executive Services (CPU1),  cfe__es__msg_8h,    ES No-Op,      0, 0x1806, LE, 127.0.0.1, 1234, struct_c_f_e___e_s___no_args_cmd__t
 Time Services (CPU1),       cfe__time__msg_8h,  Time No-Op,    0, 0x1805, LE, 127.0.0.1, 1234, struct_c_f_e___t_i_m_e___no_args_cmd__t
+Telemetry Output,            TO_LAB_CMD,        Enable Tlm,    6, 0x1880, LE, 127.0.0.1,  1234, TO_OUTPUT_ENABLE_CC
+Executive Services,         CFE_ES_CMD,         ES No-Op,      0, 0x1806, LE, 127.0.0.1,  1234, CFE_ES_NOOP_CC
+Software Bus,               CFE_SB_CMD,         SB No-Op,      0, 0x1803, LE, 127.0.0.1,  1234, CFE_SB_NOOP_CC
+Table Services,             CFE_TBL_CMD,        TBL No-Op,      0, 0x1804, LE, 127.0.0.1,  1234, CFE_TBL_NOOP_CC
+Time Services,              CFE_TIME_CMD,       Time No-Op,    0, 0x1805, LE, 127.0.0.1,  1234, CFE_TIME_NOOP_CC
+Event Services,             CFE_EVS_CMD,        EVS No-Op,     0, 0x1801, LE, 127.0.0.1,  1234, CFE_EVS_NOOP_CC
+Command Ingest,             CI_LAB_CMD,         CI No-Op,      0, 0x1884, LE, 127.0.0.1,  1234, CI_LAB_NOOP_CC
+Sample App,                 SAMPLE_APP_CMD,     Sample No-Op,  0, 0x1882, LE, 127.0.0.1,  1234, SAMPLE_APP_NOOP_CC

Subsystems/tlmGUI/cfe-es-app-tlm.txt

@@ -0,0 +1,19 @@
+Name,                 20, 20,  s, Str, NULL,        NULL,        NULL,       NULL
+EntryPoint,           40, 20,  s, Str, NULL,        NULL,        NULL,       NULL
+FileName,             60, 64,  s, Str, NULL,        NULL,        NULL,       NULL
+StackSize,            124,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+ModuleId,             128,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+AddressesAreValid,    132,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+CodeAddress,          136,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+CodeSize,             140,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+DataAddress,          144,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+DataSize,             148,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+BSSAddress,           152,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+BSSSize,              156,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+StartAddress,         160,  4,  I, Hex, NULL,        NULL,        NULL,       NULL
+ExceptionAction,      164,  2,  H, Dec, NULL,        NULL,        NULL,       NULL
+Priority,             166,  2,  H, Dec, NULL,        NULL,        NULL,       NULL
+MainTaskId,           168,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+ExecutionCounter,     172,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+MainTaskName,         176,  20, s, Str, NULL,        NULL,        NULL,       NULL
+NumOfChildTasks,      196,  4,  I, Dec, NULL,        NULL,        NULL,       NULL

Subsystems/tlmGUI/cfe-es-blockstats_1-tlm.txt

@@ -0,0 +1,9 @@
+BlockSize,           16,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumCreated,          20,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumFree,             24,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+BlockSize,           28,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumCreated,          32,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumFree,             36,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+BlockSize,           40,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumCreated,          44,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumFree,             46,  4,  I, Dec, NULL,        NULL,        NULL,       NULL

Subsystems/tlmGUI/cfe-es-memstats-tlm.txt

@@ -0,0 +1,5 @@
+PoolHandle,                    20,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+PoolSize,                      24,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumBlocksRequested,            28,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+CheckErrCtr,                   32,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+NumFreeBytes,                  36,  4,  I, Dec, NULL,        NULL,        NULL,       NULL

Subsystems/tlmGUI/cfe-es-shell-tlm.txt

@@ -0,0 +1 @@
+ShellOutput,             16, 64,  s, Str, NULL,        NULL,        NULL,       NULL

Subsystems/tlmGUI/cfe-evs-hk-tlm.txt

@@ -0,0 +1,15 @@
+#
+Command Counter,        12,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+CommandErrorCounter,    13,  1,  B, Dec, NULL,        NULL,        NULL,       NULL  
+MessageFormatMode,      14,  1,  B, Dec, NULL,        NULL,        NULL,       NULL  
+MessageTruncCounter,    15,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+UnregisteredAppCounter, 16,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+OutputPort,             17,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+LogFullFlag,            18,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+LogMode,                19,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+MessageSendCounter,     20,  2,  H, Dec, NULL,        NULL,        NULL,       NULL
+LogOverflowCounter,     22,  2,  H, Dec, NULL,        NULL,        NULL,       NULL
+LogEnabled,             24,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+Spare1,                 25,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+Spare2,                 26,  1,  B, Dec, NULL,        NULL,        NULL,       NULL
+Spare3,                 27,  1,  B, Dec, NULL,        NULL,        NULL,       NULL

Subsystems/tlmGUI/cfe-sb-allsubs-tlm.txt

@@ -0,0 +1,3 @@
+PktSegment,                    16,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+TotalSegments,                 20,  4,  I, Dec, NULL,        NULL,        NULL,       NULL
+Entries,                       24,  4,  I, Dec, NULL,        NULL,        NULL,       NULL