TileDB-API support needs

[johnkerl]

From @bkmartinjr -- TBD whether we continue to track this as a GitHub issue -- for now this is a starting place.

Tactical (near-term, important for launch of CELLxGENE atlas):

• Unicode support
  [ ] Unicode dims
  [x] Unicode attrs with the ability to apply QueryCondition (value filter) on them
  • See also detailed exposition at Queryability of dataframe attribute columns #99
  • See also Restore ability to store and query non-ASCII dataframe attributes #274 (dedicated issue)
    [x] Fix core crash which will enable SOMADataFrame (dense) implementation
  • Fixed
• Core O(1) "row count" support for sparse arrays
  • See also .shape attribute of arrays created with from_h5ad generates error #10
  • See also Allow shape accessors on X et al. #125
  • Currently tiledb.open('my-sparse-array').shape returns domain information, not nnz which is the hope

Mid-term (important to make atlas usable at scale):

[ ] Dictionary encoding -- will reduce size/cost of many obs/var columns which have small cardinality categories

• Including discussion of how/when to use this, and under what circumstances its use is and/or will be most beneficial

Strategic (mid-to-long term):

[ ] Partitioning support for distributed compute
[ ] Native/C++ support for Arrow Tensor I/O (eg, CSR) #382

Bugs:
[x] SIGFPE in a particular query using default python buffer size: SC 21775

• Fixed in core 2.13

[johnkerl]

cc @bkmartinjr @ihnorton @Shelnutt2 -- while we do (and should) track issues internally using Shortcut, we must also connect the dots for external stakeholders. That's what this issue is for.

[johnkerl]

See #866 and #867. Remainings are addressed already, with exception of query-partitioning support which will flow outward from TileDB core releases.