Skip to content

Commit

Permalink
Add Enumerated Data Types
Browse files Browse the repository at this point in the history
TODO: Write a better commit message
  • Loading branch information
davisp committed May 15, 2023
1 parent 8afb588 commit 32a8576
Show file tree
Hide file tree
Showing 57 changed files with 5,945 additions and 100 deletions.
7 changes: 6 additions & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -219,7 +219,12 @@ else()
elseif (CMAKE_BUILD_TYPE MATCHES "RelWithDebInfo")
add_compile_options(-DNDEBUG -O3 -g3 -ggdb3 -gdwarf-3)
elseif (CMAKE_BUILD_TYPE MATCHES "Coverage")
add_compile_options(-DDEBUG -g3 -gdwarf-3 --coverage)
if (CMAKE_CXX_COMPILER_ID MATCHES "Clang")
add_compile_options(-DDEBUG -O0 -g3 --coverage -fprofile-instr-generate -fcoverage-mapping)
add_link_options(--coverage -fprofile-instr-generate -fcoverage-mapping)
else()
add_compile_options(-DDEBUG -g3 -gdwarf-3 --coverage)
endif()
endif()

# Use -Wno-literal-suffix on Linux with C++ sources.
Expand Down
37 changes: 37 additions & 0 deletions PJD_TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,37 @@
Chores Left
===

* Testing
* Throw error if enumeration has longer length than the integer width of the attribute
* Require the attribute type to be integral when setting an enumeration
* Don't allow signed integer values for attributes with enumeratiojns? Does R need this?
* Require cell_val_num == 1 for attributes
* Don't forget to test drop and then add attribute with enumeration.

* Missing APIs
* array schema evolution C API
* QueryConditon C API

* Miscellany
* Add Enumeration::dump(FILE* out)
* When serializing array schema, check that any enumerations set on attributes actually exist
* Array::get_enumeration - might have to check all load schema versions?
* Do loaded enumerations need serialized?
* QueryAstNode rewrite_enumeration_condition - Need to adjust bit widths
* Document enumeration format changes in the storage format docs

* REST Serialization ToDOs
* Enumerations initial implementation
* Add ASTNode::use_enumeration to serialization code - If not, all remote queries are against enumeration values
* Add attribute enumeration name

* Use generate_uri instead of reimplementing the thinger if I did that

* Coverage Reports
* Add totals row to TOC
* Do we generate files for headers with no executable lines? If not, track whcih ones are fake so that headers are green in the TOC when they only have non-executable changes
* Add jump-to-first-uncovered link


* Future TODO:
* Update schema consolidation and vaccuming for enumerations
28 changes: 28 additions & 0 deletions format_spec/enumeration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
---
title: Enumerations
---

## Main Structure

```
my_array # array folder
| ...
|_ schema # ArraySchema directory named `__schema`
|_ enumerations # Enumeration directory named `__enumerations`
| |_ enumeration # enumeration data with names `__t1_t2_uuid_v`
Enumeration data is stored in a subdirectory of the [array schema][./array_schema.md]
directory. Enumerations are stored using [Generic Tiles][./generic_tile.md].
Data stored in the generic tile follows the current format as of version 19.
| **Field** | **Type** | **Description** |
| :--- | :--- | :--- |
| Version number | `uint32_t` | Format version number of the generic tile |
| Datatype | `uint8_t` | The datatype of the enumeration values |
| Cell Val Num | `uint32_t` | The cell val num of the enumeration values |
| Ordered | `bool` | Whether the enumeration values should be considered ordered |
| Data Size | `uint64_t` | The number of bytes used to store the values |
| Data | `uint8_t` * Data Size | The data for the enumeration values |
| Offsets Size | `uint64_t` | The number of bytes used to store offsets if cell_var_num is TILEDB_VAR_NUM |
| Offsets | `uint8_t` * Offsets Size | The offsets data for the enumeration if cell_var_num is TILEDB_VAR_NUM |
Loading

0 comments on commit 32a8576

Please sign in to comment.