Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

add enumerated/categorical support #604

Closed
bkmartinjr opened this issue Jul 5, 2023 · 4 comments · Fixed by #1064
Closed

add enumerated/categorical support #604

bkmartinjr opened this issue Jul 5, 2023 · 4 comments · Fixed by #1064
Assignees
@bkmartinjr
Labels
backlog-2023H2 Work considered for 2023 H2 but de-prioritized due to resources census builder Related to the census builder tool P1 Priority 1 - Improvement with wide impact, fix within 1 week

Comments

@bkmartinjr
Copy link
Contributor

tiledbsoma is working on the addition of native support for categorical/enumerated types, a la' pandas.Categorical.

When this work lands, the Census obs/var dataframes need to incorporate it for all label columns that are repetiive (e.g., assay_ontology_term_id, etc), as there should be a significant performance improvement, as well as a usability improvement (will materialize into Categoricals or Factors)
@pablo-gar
Copy link
Contributor

PR on TileDB-SOMA single-cell-data/TileDB-SOMA#1511

@pablo-gar pablo-gar added census builder Related to the census builder tool P1 Priority 1 - Improvement with wide impact, fix within 1 week and removed census builder Related to the census builder tool labels Jul 5, 2023
@pablo-gar pablo-gar added the backlog-2023H2 Work considered for 2023 H2 but de-prioritized due to resources label Oct 11, 2023
@bkmartinjr
Copy link
Contributor Author

Currently blcoked by single-cell-data/TileDB-SOMA#1973

@bkmartinjr
Copy link
Contributor Author

PR #896 adds enum support, but disabled by a feature flag until issues upstream are resolved. Toggling the feature flag is currently waiting on:

@bkmartinjr bkmartinjr mentioned this issue Dec 21, 2023
Merged
3 tasks
@bkmartinjr
Copy link
Contributor Author

Latest update: blocked by filter query fix which is slated for TileDB core 2.21 at the earliest.

@bkmartinjr bkmartinjr mentioned this issue Mar 22, 2024
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@bkmartinjr bkmartinjr

Labels
backlog-2023H2 Work considered for 2023 H2 but de-prioritized due to resources census builder Related to the census builder tool P1 Priority 1 - Improvement with wide impact, fix within 1 week
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[builder] enable Arrow Dictionary feature flag chanzuckerberg/cellxgene-census
2 participants
@bkmartinjr @pablo-gar