Add union support (#41)

* -Write macro records correctly. Fixes #7 * -Cleanup * -Update unit testing for draco patterns. WIP. * -Update eclipse project * -Minimally functional target_symbol * -Do not store row id for symbol target updates * -Update github CI * -Docker config for CI * -Point submodules to github * -Cleanup * -Add symbols and encoding mappings. WIP. * -Add symbols and encoding mappings. WIP. * Writer DWARF5 ecodings to encodings table * -Add minimally functional encodings implementation. -TODO:Cleanup. * -Update Makefile * -Update gitignore * -Cleanup * -Remove Ubuntu18 from github actions since it is not available anymore. -https://github.blog/changelog/2022-08-09-github-actions-the-ubuntu-18-04-actions-runner-image-is-being-deprecated-and-will-be-removed-by-12-1-22/ * -Remove dead code * -Cleanup * -Cleanup * -Cleanup * -Include unit test code in format checks. -Format unit test code. * -Update unit tests * -Configure coveralls.io for CI * -Configure coveralls.io for CI * -Configure coveralls.io for CI * -Configure coveralls.io for CI * -Configure coveralls.io for CI * -Configure coveralls.io for CI * -Configure coveralls.io for CI * -Configure coveralls.io for CI * -Update docs * -Update unit tests for new schema. WIP. -TODO:Add function to follow target symbols to test typdef'd types. -TODO:Add unit tests for new macro and elf image features. * -Add followTargetSymbol. Useful for testing typedef'd types. -Update unit tests. * -Use CFE_ES_HousekeepingTlm_Payload for unit testing. * -Update docs * -Update docs * -Extract ELF image data from 64-bit targets properly. -Add ElfClass to ElfFile * -Update docs * -Update unit-test test files * -Remove libdwarf from unit test code test file * -Remove dead code * -Remove dead code * -Add 32-bit make recipes. -Add 32-bit unit tests. * -Update 32-bit unit testing. * -Update docs * -Remove dead code * -Update Logger unit tests -Remove dead code * -Update unit tests * -Update docs * -Cleanup * -Cleanup * -Cleanup * -Add macro unit test. -Update Dockerfile * -Create test files that comply with DWARF4 and DWARF5. Allows us to verify support of DWARF4 as more features are added in the future. * -Unit test DWARF4 and DWARF5. WIP. * -Add DWARF4 testing to Dockerfile * -Cleanup * -Update docs * -Update docs * -Update Catch 2 due to issue on Ubuntu22:catchorg/Catch2#2178 * -pathIndex handling for DWARF5. TODO:This changes for DWARF4, so add alternative for DWARF4. -Remove DisplayDie function for now as it blows up the stack with big c++ names inside DWARF. -TODO:Rewrite DisplayDie function -Add include to src/Artifact.h, needed for Ubuntu22. * -Handle DWARF dbg source files for DWARF4 and 5. -TODO:Figure out what to do about Ubuntu 20/22 behavior producing different DWARF src file line number info for the same DWARF version. * -Add group number as CLI argument. WIP. -Issues of interest:#36, #35 * -Group number argument. Useful for getting DWARF data from multiple COMBAT sections, like macros inside of unlinked ELF files. * -Add unit tests for macros across multiple COMDAT groups. * -Cleanup * -Update docs * -Update unit tests. * -Update Dockerfile for Ubuntu22 * -Update CI * -Update Dockerfiles for Ubuntu20 and Ubuntu22 * -Update Makefile * -Update CI * -Update CI * -Test DWARF version in unit tests * -Update docs * -Update docs * -Update docs * -Update docs. WIP. * -Update docs * -Cleanup * -Cleanup * -Update docs. -Remove DECL code from pointer types. Pointer types do not have DECL coords. * -Add unit tests for artifacts * -Cleanup * -Update test_file1 * -Remove dead code * -Update tests in Docker files * -Update docs * -Update docs * -Update docs * -Update unit tests * -Add support for unions. WIP. * -Add support for unions. WIP. * -Update CI to build Ubuntu20 first. * -Update CI to build Ubuntu20 first. * -Format files * -Update union object used for unit tests * -Add user to docker dev configs * -Add support for unions. WIP. -Update unit tests for unions. WIP. * -Minimally functional union support. -TODO:Upgrade to C++17 and std::optional for union fields, which will be NULL (or empty). * -Remove dead code * -Upgrade to C++17. -Use optional for byte offset of fields. -When byte offset value is empty, write NULL to db. This allows to make a distinction between unions and structs. -TODO:Finish union unit testing. -TODO:Add union support documentation. * -Update docs * -Remove duplicate recipes from Makefile * -Handle default case in getdbgSourceFile * -Update unit testing for union support * -Update unit testing for union support * -Update docs * -Update docs * -Use std::optional for symbol encoding * -Add vscode config * -Add tests for padding. Fixes #21 * -Give root access to dev docker user * -Use std::optional for getdbgSourceFile * -Do not attempt to add padding for unions. -Add checks to padding function. * -Add warning message when field byte offsets do not exist * -Add warning message when field byte offsets do not exist * -Use git tag and latest commit as version value * -Log warning when there is no DW_AT_name attribute --------- Co-authored-by: Mathew Benson <[email protected]>
WindhoverLabs · Nov 4, 2024 · 302145b · 302145b
1 parent 282a1af
commit 302145b
Show file tree

Hide file tree

Showing 22 changed files with 853 additions and 189 deletions.
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
@@ -31,16 +31,20 @@ jobs:
       #     push: false  # Do not push the image
       #     tags: juicer:latest  # Local tag for the built image
 
-      - name: Build Docker image For Ubuntu22
-        run: docker build --no-cache -t juicer:latest -f Dockerfile.ubuntu22 . 
 
       - name: Build Docker image For Ubuntu20
         run: docker build --no-cache -t juicer:latest -f Dockerfile.ubuntu20 .
+
       - name: Copy coverage report to host
         run: docker image ls && img_id=$(docker create juicer:latest) && docker cp $img_id:/home/docker/juicer/coverage.gcov .
+
       - name: publish to coveralls.io
         run: wget https://github.com/coverallsapp/coverage-reporter/releases/download/v0.6.14/coveralls-linux && chmod a+x ./coveralls-linux && COVERALLS_REPO_TOKEN=${{ secrets.COVERALLS_REPO_TOKEN }} ./coveralls-linux 
 
+      - name: Build Docker image For Ubuntu22
+        run: docker build --no-cache -t juicer:latest -f Dockerfile.ubuntu22 . 
+
+
       - name: Build Docker image For Ubuntu18
         run: docker build --no-cache -t juicer:latest -f Dockerfile.ubuntu18 . 
 
diff --git a/.vscode/c_cpp_properties.json b/.vscode/c_cpp_properties.json
@@ -0,0 +1,17 @@
+{
+    "configurations": [
+        {
+            "name": "Linux",
+            "includePath": [
+                "${workspaceFolder}/**",
+                "/usr/include/libdwarf"
+            ],
+            "defines": [],
+            "compilerPath": "/usr/bin/clang",
+            "cStandard": "c17",
+            "cppStandard": "c++17",
+            "intelliSenseMode": "linux-clang-x64"
+        }
+    ],
+    "version": 4
+}
diff --git a/.vscode/settings.json b/.vscode/settings.json
@@ -0,0 +1,87 @@
+{
+  "configurations": [
+    {
+      "name": "x86",
+      "includePath": [
+        "/usr/include/libdwarf"
+      ],
+      "browse": {
+        "limitSymbolsToIncludedHeaders": true,
+        "databaseFilename": ""
+      }
+    }
+  ],
+  "files.associations": {
+    "macro_test.h": "c",
+    "stdint.h": "c",
+    "cctype": "cpp",
+    "clocale": "cpp",
+    "cmath": "cpp",
+    "cstdarg": "cpp",
+    "cstddef": "cpp",
+    "cstdio": "cpp",
+    "cstdlib": "cpp",
+    "cstring": "cpp",
+    "ctime": "cpp",
+    "cwchar": "cpp",
+    "cwctype": "cpp",
+    "array": "cpp",
+    "atomic": "cpp",
+    "strstream": "cpp",
+    "*.tcc": "cpp",
+    "bitset": "cpp",
+    "chrono": "cpp",
+    "complex": "cpp",
+    "condition_variable": "cpp",
+    "cstdint": "cpp",
+    "deque": "cpp",
+    "list": "cpp",
+    "unordered_map": "cpp",
+    "vector": "cpp",
+    "exception": "cpp",
+    "algorithm": "cpp",
+    "functional": "cpp",
+    "iterator": "cpp",
+    "map": "cpp",
+    "memory": "cpp",
+    "memory_resource": "cpp",
+    "numeric": "cpp",
+    "optional": "cpp",
+    "random": "cpp",
+    "ratio": "cpp",
+    "regex": "cpp",
+    "set": "cpp",
+    "string": "cpp",
+    "string_view": "cpp",
+    "system_error": "cpp",
+    "tuple": "cpp",
+    "type_traits": "cpp",
+    "utility": "cpp",
+    "fstream": "cpp",
+    "future": "cpp",
+    "initializer_list": "cpp",
+    "iomanip": "cpp",
+    "iosfwd": "cpp",
+    "iostream": "cpp",
+    "istream": "cpp",
+    "limits": "cpp",
+    "mutex": "cpp",
+    "new": "cpp",
+    "ostream": "cpp",
+    "shared_mutex": "cpp",
+    "sstream": "cpp",
+    "stdexcept": "cpp",
+    "streambuf": "cpp",
+    "thread": "cpp",
+    "cinttypes": "cpp",
+    "typeinfo": "cpp",
+    "variant": "cpp",
+    "bit": "cpp",
+    "compare": "cpp",
+    "concepts": "cpp",
+    "numbers": "cpp",
+    "semaphore": "cpp",
+    "stop_token": "cpp"
+  },
+  "editor.formatOnSave": true
+}
diff --git a/Dockerfile.ubuntu18 b/Dockerfile.ubuntu18
@@ -50,6 +50,11 @@ RUN ./juicer-ut "[main_test#20]"
 RUN ./juicer-ut "[Module]"
 RUN ./juicer-ut "[Symbol]"
 
+RUN cd /home/docker/juicer && make coverage
+#Useful for CI
+RUN cd /home/docker/juicer &&  gcovr --filter /home/docker/juicer/src/ --object-directory /home/docker/juicer/build/ut_obj/ --xml  coverage.gcov 
+
+
 
 
 
diff --git a/Dockerfile.ubuntu18.dev b/Dockerfile.ubuntu18.dev
@@ -20,3 +20,11 @@ RUN mkdir /home/docker/juicer
 WORKDIR /home/docker/juicer
 
 
+
+RUN useradd -ms /bin/bash juicer-dev
+RUN usermod -aG sudo juicer-dev
+RUN echo "juicer-dev ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
+
+USER juicer-dev
+
+
diff --git a/Dockerfile.ubuntu20.dev b/Dockerfile.ubuntu20.dev
@@ -27,3 +27,9 @@ RUN apt-get install -y clang-format
 RUN mkdir /home/docker
 RUN mkdir /home/docker/juicer
 WORKDIR /home/docker/juicer
+
+RUN useradd -ms /bin/bash juicer-dev
+RUN usermod -aG sudo juicer-dev
+RUN echo "juicer-dev ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
+
+USER juicer-dev
diff --git a/Dockerfile.ubuntu22.dev b/Dockerfile.ubuntu22.dev
@@ -19,3 +19,9 @@ RUN apt-get install -y gcovr
 RUN mkdir /home/docker
 RUN mkdir /home/docker/juicer
 WORKDIR /home/docker/juicer
+
+RUN useradd -ms /bin/bash juicer-dev
+RUN usermod -aG sudo juicer-dev
+RUN echo "juicer-dev ALL=(ALL) NOPASSWD:ALL" >> /etc/sudoers
+
+USER juicer-dev
diff --git a/Images/union_fields_tbl.png b/Images/union_fields_tbl.png
diff --git a/Images/union_symbol_tbl.png b/Images/union_symbol_tbl.png
diff --git a/Makefile b/Makefile
@@ -38,10 +38,12 @@ UT_SRC_32     := $(wildcard $(UT_SRC_DIR)/test_file*.cpp)
 UT_OBJ_32     := $(UT_SRC_32:$(UT_SRC_DIR)/test_file%.cpp=$(UT_OBJ_32BIT_DIR)/test_file%.o)
 UT_OBJ_32     := $(UT_OBJ_32:$(UT_SRC_DIR)/test_file%.cpp=$(UT_OBJ_32BIT_DIR)/test_file%.o)
 
+GIT_VERSION   := "$(shell git describe --tags --abbrev=0)($(shell  git rev-parse --short HEAD))"
+
 
 # Set target flags
-CPPFLAGS            := -MMD -MP -std=c++14 -fmessage-length=0 $(INCLUDES)
-CFLAGS              := -Wall -g3 
+CPPFLAGS            := -MMD -MP -std=c++17 -fmessage-length=0 $(INCLUDES)
+CFLAGS              := -Wall -g3 -DJUICER_VERSION=\"$(GIT_VERSION)\"
 CFLAGS_32BIT        := -Wall -g3 -m32 
 LDFLAGS             := -Llib
 LDLIBS              := -lm -ldwarf -lsqlite3 -lelf -lcrypto
@@ -123,7 +125,6 @@ clean:
 -include $(UT_OBJ:.o=.d)
 -include $(OBJ:.o=.d)
 
-
 docker-ubuntu18-build:
 	@sudo docker build --no-cache -t juicer:ubuntu18 -f Dockerfile.ubuntu18 .
 

diff --git a/README.md b/README.md
@@ -1,6 +1,6 @@
 
 [![Run juicer tests](https://github.com/WindhoverLabs/juicer/actions/workflows/ci.yaml/badge.svg)](https://github.com/WindhoverLabs/juicer/actions/workflows/ci.yaml)
-[![Coverage Status](https://coveralls.io/repos/github/WindhoverLabs/juicer/badge.svg?branch=unit_test_updates)](https://coveralls.io/github/WindhoverLabs/juicer?branch=unit_test_updates)
+[![Coverage Status](https://coveralls.io/repos/github/WindhoverLabs/juicer/badge.svg?branch=unit_test_updates)](https://coveralls.io/github/WindhoverLabs/juicer?branch=develop)
 
 # Table of Contents
 1. [Dependencies](#dependencies)
@@ -17,6 +17,7 @@
 12. [Notes On Multiple DWARF Versions](#multiple_dwarf_versions)
 13. [Bitfields](#Bitfields)
 14. [Docker Dev Environments](#docker_dev_env) 
+15. [Union Support](#union_support) 
 
 ## Dependencies <a name="dependencies"></a>
 * `libdwarf-dev`
@@ -322,6 +323,7 @@ As juicer evolves, dwarf support will grow and evolve as well. At the moment, we
 |  DW_MACRO_define      | This tag represents define macros such as "#define CFE_MISSION_ES_PERF_MAX_IDS 128"|
 | DW_AT_decl_file       | This tag represents the file where a certain symbol is declared. Very useful for traceability of source code.|
 | DW_AT_encoding | The encoding of base type. For example; "unsigned int" will have encoding "DW_ATE_unsigned". This is what the "encodings" table is for in the SQLITE db.|
+| DW_TAG_union_type | The tag used for unions. For example; `union Object { int32_t id; int32_t bit_field : 10; uint8_t data[4];};`.|
 
 For more details on the DWARF debugging format, go on [here](http://www.dwarfstd.org/doc/DWARF4.pdf).
 
@@ -433,5 +435,29 @@ For example `make docker-ubuntu22-build-dev` will start a dev environment inside
 The repo is mounted as a volume under "/home/docker/juicer" so developers can make their changes on the host and build inside the container.
 
 
+# Union Support <a name="union_support"></a>
 
-Documentation updated on September 19, 2024
+Given the following union:
+
+```C
+union Object
+{
+    int32_t id;
+    int32_t bit_field : 10;
+    uint8_t data[4];
+};
+
+```
+
+The union is represented as such in the SQLITE db:
+
+
+![union_symbol](Images/union_symbol_tbl.png "symbols-table")
+
+![union_fields](Images/union_fields_tbl.png "fields-table")
+
+Notice that the byte_offset is `NULL` for the fields that belong to the union. This is how consumers (such as [auto-yamcs](https://github.com/WindhoverLabs/auto-yamcs)) can make a distinction between unions and structs.
+
+
+
+Documentation updated on September 19, 2024
diff --git a/src/Field.cpp b/src/Field.cpp
@@ -10,7 +10,7 @@
 Field::Field(Symbol& inSymbol, Symbol& inType)
     : symbol{inSymbol},  // @suppress("Symbol is not resolved")
       name{""},
-      byte_offset{0},
+      byte_offset{std::nullopt},
       type{inType},  // @suppress("Symbol is not resolved")
       dimensionList{},
       little_endian{false},
@@ -22,7 +22,7 @@ Field::Field(Symbol& inSymbol, Symbol& inType)
                     type.getName().c_str(), dimensionList.toString(), little_endian ? "LE" : "BE");
 }
 
-Field::Field(Symbol& inSymbol, std::string& inName, uint32_t inByteOffset, Symbol& inType, DimensionList& inDimensionList, bool inLittleEndian,
+Field::Field(Symbol& inSymbol, std::string& inName, std::optional<uint32_t> inByteOffset, Symbol& inType, DimensionList& inDimensionList, bool inLittleEndian,
              uint32_t inBitSize, uint32_t inBitOffset)
     : symbol{inSymbol},  // @suppress("Symbol is not resolved")
       name{inName},      // @suppress("Symbol is not resolved")
@@ -41,7 +41,8 @@ Field::Field(Symbol& inSymbol, std::string& inName, uint32_t inByteOffset, Symbo
                     type.getName().c_str(), dimensionList.toString(), little_endian ? "LE" : "BE");
 }
 
-Field::Field(Symbol& inSymbol, std::string& inName, uint32_t inByteOffset, Symbol& inType, bool inLittleEndian, uint32_t inBitSize, uint32_t inBitOffset)
+Field::Field(Symbol& inSymbol, std::string& inName, std::optional<uint32_t> inByteOffset, Symbol& inType, bool inLittleEndian, uint32_t inBitSize,
+             uint32_t inBitOffset)
     : symbol{inSymbol},  // @suppress("Symbol is not resolved")
       name{inName},      // @suppress("Symbol is not resolved")
       byte_offset{inByteOffset},
@@ -60,13 +61,13 @@ Field::Field(Symbol& inSymbol, std::string& inName, uint32_t inByteOffset, Symbo
 }
 Field::~Field() {}
 
-uint32_t     Field::getByteOffset() const { return byte_offset; }
+std::optional<uint32_t> Field::getByteOffset() const { return byte_offset; }
 
-bool         Field::isLittleEndian() const { return little_endian; }
+bool                    Field::isLittleEndian() const { return little_endian; }
 
-std::string& Field::getName() { return name; }
+std::string&            Field::getName() { return name; }
 
-void         Field::setName(const std::string& inName)
+void                    Field::setName(const std::string& inName)
 {
     logger.logDebug("Field %s::%s  renamed  to %s.", symbol.getName().c_str(), name.c_str(), inName.c_str());
 

diff --git a/src/Field.h b/src/Field.h
@@ -12,6 +12,7 @@
 
 #include <stdint.h>
 
+#include <optional>
 #include <vector>
 
 #include "DimensionList.h"
@@ -30,50 +31,51 @@ class Field
 {
    public:
     Field(Symbol &symbol, Symbol &type);
-    Field(Symbol &symbol, std::string &name, uint32_t byte_offset, Symbol &type, DimensionList &dimensionList, bool little_endian, uint32_t inBitSize = 0,
+    Field(Symbol &symbol, std::string &name, std::optional<uint32_t> byte_offset, Symbol &type, DimensionList &dimensionList, bool little_endian,
+          uint32_t inBitSize = 0, uint32_t inBitOffset = 0);
+    Field(Symbol &symbol, std::string &name, std::optional<uint32_t> byte_offset, Symbol &type, bool little_endian, uint32_t inBitSize = 0,
           uint32_t inBitOffset = 0);
-    Field(Symbol &symbol, std::string &name, uint32_t byte_offset, Symbol &type, bool little_endian, uint32_t inBitSize = 0, uint32_t inBitOffset = 0);
     virtual ~Field();
-    uint32_t           getByteOffset() const;
-    bool               isLittleEndian() const;
-    uint32_t           getMultiplicity() const;
-    uint32_t           getArraySize() const;
-    void               setMultiplicity(uint32_t multiplicity);
-    std::string       &getName();
-    void               setName(const std::string &name);
-    Symbol            &getSymbol() const;
-    Symbol            &getType();
-    uint32_t           getId(void) const;
-    void               setId(uint32_t newId);
-    uint32_t           getBitOffset() const;
-    void               setBitOffset(uint32_t bitOffset);
-    uint32_t           getBitSize() const;
-    void               setBitSize(uint32_t bitSize);
-    bool               isBitField(void);
-    DimensionList     &getDimensionList();
-    bool               isArray(void) const;
-    std::string        getDimensionListStr();
+    std::optional<uint32_t> getByteOffset() const;
+    bool                    isLittleEndian() const;
+    uint32_t                getMultiplicity() const;
+    uint32_t                getArraySize() const;
+    void                    setMultiplicity(uint32_t multiplicity);
+    std::string            &getName();
+    void                    setName(const std::string &name);
+    Symbol                 &getSymbol() const;
+    Symbol                 &getType();
+    uint32_t                getId(void) const;
+    void                    setId(uint32_t newId);
+    uint32_t                getBitOffset() const;
+    void                    setBitOffset(uint32_t bitOffset);
+    uint32_t                getBitSize() const;
+    void                    setBitSize(uint32_t bitSize);
+    bool                    isBitField(void);
+    DimensionList          &getDimensionList();
+    bool                    isArray(void) const;
+    std::string             getDimensionListStr();
 
-    const std::string &getShortDescription() const { return short_description; }
+    const std::string      &getShortDescription() const { return short_description; }
 
-    const std::string &getLongDescription() const { return long_description; }
+    const std::string      &getLongDescription() const { return long_description; }
 
    private:
-    Symbol       &symbol;
-    std::string   name;
-    uint32_t      byte_offset;
-    Symbol       &type;
-    DimensionList dimensionList;
-    bool          little_endian;
+    Symbol                 &symbol;
+    std::string             name;
+    std::optional<uint32_t> byte_offset{std::nullopt};
+    Symbol                 &type;
+    DimensionList           dimensionList;
+    bool                    little_endian;
     /*bit fields members.
      * If this field is not bit-packed, then the bit_size and bit_offset are 0.*/
-    uint32_t      bit_offset;
-    uint32_t      bit_size;
-    Logger        logger;
-    uint32_t      id;
+    uint32_t                bit_offset;
+    uint32_t                bit_size;
+    Logger                  logger;
+    uint32_t                id;
 
-    std::string   short_description;
-    std::string   long_description;
+    std::string             short_description;
+    std::string             long_description;
 };
 
 #endif /* FIELD_H_ */