Reversible typecasts (parent task)

[johnkerl]

When we ingest data into TileDB from, say, AnnData, we need to remap a few things to be storable:

• Categorical strings to strings
• bool to uint8
• Unicode array dimensions to ASCII dimensions -- necessary for TileDB at present
• Unicode array attributes to ASCII array attributes -- TileDB allows attributes to be Unicode, but, they are not queryable via the QueryCondition API. See also Queryability of dataframe attribute columns #99.

Source links:

• util_ann.decategoricalize
• util._to_tiledb_supported_array_type

Related: #105

Item 2 will be addressed in TileDB Core 2.10, where we'll have native boolean support. The remaining ones, though, are ongoing concerns.

Proposal for item 1:

• Use TileDB group/array metadata tags
• Indicate the following at the SOMA-group level:
  • Source provenance (e.g. anndata, seurat)
  • Language and package version of the code that wrote the data
• Indicate the following at the SOMA-array level:
  • Either a dict from all columns to their original types, or, just the 'exceptions' (e.g. categorical-string columns)

Proposal for items 3 and 4:

• See Queryability of dataframe attribute columns #99

[johnkerl]

Split out remaining issues to #866 and #867. cc @maniarathi.