Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Define a path to promote a the functionality of cellxgene_census.experimental.util._csr_iter.X_sparse_iter as stable #794

Closed
pablo-gar opened this issue Oct 5, 2023 · 2 comments
Labels
P1 Priority 1 - Improvement with wide impact, fix within 1 week python api Related to the API user request

Comments

@pablo-gar
Copy link
Contributor

Description

cellxgene_census.experimental.util._csr_iter.X_sparse_iter has been found to be of great utility for both internal and external cases. Since it has not been advertised and it is experimental, users have mentioned the need to gain the functionality that this method has.

Context

The recurrent issue is the following:

  • I need a memory efficient way to access sparse matrices for downstream analysis. I often require:
    • An iterator that provides csr chunks
    • Full rows per chunk, i.e. no disjoint chunks.
    • Re-indexed positional indices, i.e. no soma_joinids

Potential solutions

  • Promote this method as stable in cellxgene_census
  • Find a path to move functionality down to tiledbsoma. Probably requires more design work to ensure we have the best way to integrate with tieldbsoma
@pablo-gar pablo-gar added python api Related to the API P1 Priority 1 - Improvement with wide impact, fix within 1 week user request labels Oct 5, 2023
@bkmartinjr
Copy link
Contributor

@bkmartinjr
Copy link
Contributor

this is resolved by single-cell-data/TileDB-SOMA#1792

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
P1 Priority 1 - Improvement with wide impact, fix within 1 week python api Related to the API user request
Projects
None yet
Development

No branches or pull requests

2 participants