Skip to content

Releases: single-cell-data/TileDB-SOMA

TileDB-SOMA 1.15.2

20 Dec 19:21
ac4e912
Compare
Choose a tag to compare

New features added

  • [Backport release-1.15] [python] Update add_new_collection method to support Scene collections by @github-actions in #3499

Bug fixes

  • [Backport release-1.15] [python] Make ExperimentAxisQuery inherit somacore, restore its .close by @github-actions in #3500

Minor changes

  • [Backport release-1.15] [c++] Use const references and std::string_view in create function by @github-actions in #3483
  • [Backport release-1.15] [c++] Make destructors of base classes virtual by @github-actions in #3487
  • [Backport release-1.15] [python] Fix a test deprecation warning by @github-actions in #3488
  • [Backport release-1.15] [python] tiledbsoma.__init__.py: restore lint checks by @github-actions in #3498
  • [r] TileDB-SOMA 1.15.2 by @johnkerl in #3497
  • [Backport release-1.15] [c++, python] fastercsx: improved duplicate coordinate handling by @github-actions in #3485
  • [Backport release-1.15] [c++] Set overriding user-provided prefix to be off by default by @github-actions in #3489
  • [Backport release-1.15] [c++/ci] Disable leak detection and add ASAN CI job to all C++ PRs by @github-actions in #3492
  • [Backport release-1.15] [python] Filter spatial disclaimer from pytest by @github-actions in #3496

Full Changelog: 1.15.1...1.15.2

TileDB-SOMA 1.15.1

19 Dec 14:35
80e1825
Compare
Choose a tag to compare

The primary bugfix for this release is #3479, which is a backport of #3476. This releases fixes the error message

TypeError: ExperimentAxisQuery() takes no arguments

Bug fixes

  • [Backport release-1.15] [python] Re-enable tiledbsoma.ExperimentAxisQuery by @github-actions in #3479
  • [docs] Fix broken tutorial-doc symlinks in release-1.15 by @johnkerl in #3466
  • [Backport release-1.15] [python/c++] Fix invalid memory read in fastercsx by @github-actions in #3464
  • [Backport release-1.15] [python] Fix stochastic failure of to_spatialdata method by @github-actions in #3477
  • [Backport release-1.15] [python] Add extra asserts for SpatialData tests by @github-actions in #3474
  • [Backport release-1.15][r] Write group-level string metadata as TILEDB_STRING_UTF8; 1.15.1 by @johnkerl in #3481

Minor changes

  • [Backport release-1.15] [python/c++] Deprecate config_options_from_schema in favor of new function by @github-actions in #3465
  • [Backport release-1.15] [c++, r, python] read_next clean-up by @github-actions in #3480
  • [Backport release-1.15] [python] Convert from Optional[Foo] to Foo | None by @github-actions in #3467

Full Changelog: 1.15.0...1.15.1

TileDB-SOMA 1.15.0

17 Dec 16:17
e5d19d0
Compare
Choose a tag to compare

Summary

This release contains several important developments:

  • Spatial transcriptomics: alpha version of the feature.
  • Performance improvements: particularly for tiledbsoma.io.to_anndata / tiledbsoma.io.to_h5ad
  • Seurat I/O: incremental work for Seurat v5
  • New shape feature
  • Documentation infrastructure: more thorough class coverage, and more clickable links in the Python docs
  • The TileDB-Py package is no longer a required dependency.

New features added

Spatial transcriptomics

  • [python] Add to_spatial_data method to ExperimentAxisQuery #3431
  • [python] Export full experiment to SpatialData #3409
  • [python] Fix typos and import order in the spatial outgest #3381
  • [python] Fix output type when exporting PointCloudDataFrame to SpatialData points #3408
  • [python] Update Scene create method in the Python API #3356
  • [python] Change import experimental -> io.spatial in test #3394
  • [python] Add support for exporting to SpatialData without transform #3392
  • [python] Rename to_spatial_data -> to_spatialdata #3449
  • [python] Move experimental module to io.spatial #3384
  • [python] Add export for MultiscaleImage to SpatialData by @jp-dark in #3355
  • [python] Add PointCloudDataFrame to SpatialData points model by @jp-dark in #3357
  • [python] Add MultiscaleImage level SpatialData exporter by @jp-dark in #3342
  • [python] Spatial changes required for somacore 1.0.24 by @jp-dark in #3333
  • [python] Fix and test SpatialData transformation exporter by @jp-dark in #3330
  • [python] Rewrite Xarray support to use dask arrays by @jp-dark in #3325
  • [python] Return identity for MultiscaleImage transform to/from level 0 by @jp-dark in #3323
  • [python] Add MultiscaleImage methods to access level URI by @jp-dark in #3322
  • [python] Add export for PointCloudDataFrame to SpatialData shapes by @jp-dark in #3298
  • [python] Create xarray backend for DataArray types by @jp-dark in #3243
  • [python] Update spatial API to match somacore 1.0.23 by @jp-dark in #3274
  • [c++] Cast vertex list to WKB by @XanthosXanthopoulos in #3222
  • [c++] Geometry Dataframe by @XanthosXanthopoulos in #3212

Performance improvements

  • [python] Remove double open For SOMAArray reads #3293
  • [python,c++] Improve error handling in IntIndexer #3441
  • [python] Some memory-reduction for append mode #3404
  • [python] Extend #3354 to categoricals of arbitrary value type #3415
  • [python] Protect against huge enum-of-strings input #3354
  • [python] Optimization of ExperimentAxisQuery to_anndata by @bkmartinjr in #3359
  • [python] Fix incorrect type handling in ExperimentAxisQuery sparse reader by @bkmartinjr in #3375
  • [python/C++] Release GIL in IntIndexer C++ code by @bkmartinjr in #3344
  • [python] Partition sparse matrix reads in tiledbsoma.io.to_anndata by @bkmartinjr in #3328
  • [python/c++] COO to CSX conversion optimization by @bkmartinjr in #3304
  • [python] Performance improvement for experiment-level resize by @johnkerl in #3308
  • [python] Add ManagedQuery bindings by @nguyenv in #3281
  • [python] Ingest somacore classes by @ryan-williams in #3307
  • [c++] Fix heap management in fastercsx by @johnkerl in #3365
  • [c++] Add ManagedQuery constructor that takes SOMAArray by @nguyenv in #3279
  • [c++] Add ManagedQuery::set_layout by @nguyenv in #3278
  • [c++] Use schema from tiledb::Array instead of ManagedQuery by @nguyenv in #3277
  • [c++] Move casting from SOMAArray to ManagedQuery by @nguyenv in #3275

Seurat I/O

  • [r] Add initial support for ragged array writing for Seurat v5 by @mojaveazure in #2523

New shape

  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 4/n #3372
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 3 of 4 #3371
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 2/n #3370
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 1 of 4 #3369
  • [c++] Use valid ASCII for wide-as-possible string current domain #3367
  • [c++] Match zero-basing for old/new sparse-array shapes #3439
  • [r] Add check_only support for domain/shape updates #3400
  • [python/r] Update docstrings for domain argument to create #3396
  • [r] Vignette for new-shape feature #3302
  • [c++] Fix upgrade-soma-joinid-shape for dataframes having non-standard dimensions #3416
  • [python/r] Docstring audit for new shape #3399
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 4 of 4 by @johnkerl in #3372
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 3 of 4 by @johnkerl in #3371
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 2 of 4 by @johnkerl in #3370
  • [python/r] Remove 2.27-related feature flag by @johnkerl in #3368
  • [r/python] Revert #3300 by @johnkerl in #3358
  • [python/r] Enforce dataframe domain lower bound == 0 by @johnkerl in #3300
  • [c++] Use valid ASCII for wide-as-possible string current domain by @johnkerl in #3367
  • [python] Use new shape in one more spot by @johnkerl in #3321
  • [python] Fix bad merge of soma-shapes notebook by @johnkerl in #3309
  • [python] Tutorial notebook for the new-shape feature by @johnkerl in #3294
  • [r] Data refresh for new shapes by @johnkerl in #3303
  • [python/r] Move beyond the new-shape feature flag by @johnkerl in #3301
  • [python] Fix dense example data for core 2.27 by @johnkerl in #3296
  • [python/docs] Replace experiment directories with .tgz files in source control by @johnkerl in #3295
  • [python] Doc updates for new-shape feature by @johnkerl in #3285
  • [python] Run upgrade-shapes on notebook example experiments by @johnkerl in #3289
  • [python] Include tiledbsoma_upgrade_shape for DenseNDArray by @johnkerl in #3288
  • [python] More auto-detect of function_name_for_messages by @johnkerl in #3286
  • [r] More fixes for unit-test cases with dense + core 2.27 by @johnkerl in #3280
  • [python] Fix last 2.27+Python+dense failing test case by @johnkerl in #3269
  • [r] Fixes for dense+2.27 by @johnkerl in #3270
  • [python] Fix 3D/4D cases with core 2.27 by @johnkerl in #3268
  • [c++] Apply subarrays for dense reads and writes by @johnkerl in #3263
  • [python] Fix some dense+2.27 failing test cases by @johnkerl in #3265
  • [python] Centralize sparse/dense pybind11 shape methods by @johnkerl in #3261
  • [python] Re-run, and update, example notebooks with new shape feature by @johnkerl in #3290

Upcoming deprecation notices

  • The .used_shape accessor for SparseNDArray and DenseNDArray is scheduled for removal in 1.16: as of 1.15 the .shape accessor replaces it.

Bug fixes

  • [python] ExperimentAxisQuery: to_anndata obsp/varp as sparse matrices #3387
  • [python] Fix conditional fall-through in SOMAObject.open #3446
  • [c++] Fix bug with sliced Arrow-table writes, with string columns #3433
  • [python, c++] Fix misleading DoesNotExistErrors in open factory #3402
  • [c++] ManagedQuery::set_condition should not reset entire instance state #3418
  • [r] Update changelog and develop version for #3395 #3401
  • [r] Adjust blockwise + re-indexer to return condensed matrix, not full domain #3395
  • [python] Re-introduce tiledb_ctx for SOMATileDBContext #3335

Minor changes

Documentation infastructure

Packaging and CI

  • [python] Pin dask version 2024.11.2 to unblock CI #3403
  • [python] Pre-emptively avoid SciPy 1.15, pending a bugfix #3445
  • [c++] Use core 2.27.0 #3421
  • [c++] Use core 2.27.0.rc5 #3410
  • [c++/python/r] Use core 2.27.0-rc3 by @johnkerl in #3366
  • [packaging] More failure info in update-tiledb-versions.py by @johnkerl in #3363
  • [python] Run pre-commit autoupdate by @johnkerl in #3326
  • [r/python] Note on dev installs and spdlog failures by @johnkerl in #3324
  • [r/ci] Remove tiledb-r specific install by @mojaveazure in #3319
  • [r/ci] Try to unbreak r-valgrind CI by @johnkerl in #3318
  • [r/ci] Avoid log truncation with "Last 13 lines of output" by @johnkerl in #3313
  • [r] Run tests in closer to internal-dependency order by @johnkerl in #3311
  • [python] Unbreak CI with anndata==0.11 by @johnkerl in #3305
  • [python] Fix some warnings at anndata==0.11 by @johnkerl in #3306

Remove TileDB-Py as a required dependency

  • [python] Update README.md to remove tiledb-py by @nguyenv in #3346

Other internals

  • [python] Remove use of deprecated schema-display function #3380
  • [r] Use standard name zzz.R for init code by @johnkerl in #3332
  • [r] Address two more compiler warnings by @johnkerl in #3320
  • [c++] Fix a compiler warning by @johnkerl in #3310
  • [python] Consolidate all _set_coords methods into single _util function by @nguyenv in #3292
  • [python] Rename set_reader_coords to set_coords by @johnkerl in #3253
  • [python] Use somacore 1.0.26 #3460
  • [python] Use somacore 1.0.25 #3455
  • [c++] Add ASAN run of CI to weekly CI job #3434

TileDB-SOMA 1.15.0rc4

22 Nov 22:16
684cd69
Compare
Choose a tag to compare
TileDB-SOMA 1.15.0rc4 Pre-release
Pre-release

This is a release candidate for 1.15.0 which we anticipate releasing in earlly December 2024. It contains several important developments.

  • Spatial transcriptomics: alpha version of the feature.
  • Performance improvements: particularly for tiledbsoma.io.to_anndata / tiledbsoma.io.to_h5ad
  • Seurat I/O: incremental work for Seurat v5
  • New shape feature: Python tutorial here; Academy docs to be shipped by the time of 1.15.0
  • Documentation infrastructure: please see the pre-release docs (note latest in the URL)
  • For developers only: We now build our C++ library using C++ 20. This doesn't affect non-developer installs, which will get artifacts from PyPI/r-universe/Conda.
  • The TileDB-Py package is no longer a required dependency.

Upcoming deprecation notices

  • The .used_shape accessor for SparseNDArray and DenseNDArray is scheduled for removal in 1.16: as of 1.15 the .shape accessor replaces it.

Spatial transcriptomics

  • [python] Add export for MultiscaleImage to SpatialData by @jp-dark in #3355
  • [python] Add PointCloudDataFrame to SpatialData points model by @jp-dark in #3357
  • [python] Add MultiscaleImage level SpatialData exporter by @jp-dark in #3342
  • [python] Spatial changes required for somacore 1.0.24 by @jp-dark in #3333
  • [python] Fix and test SpatialData transformation exporter by @jp-dark in #3330
  • [python] Rewrite Xarray support to use dask arrays by @jp-dark in #3325
  • [python] Return identity for MultiscaleImage transform to/from level 0 by @jp-dark in #3323
  • [python] Add MultiscaleImage methods to access level URI by @jp-dark in #3322
  • [python] Add export for PointCloudDataFrame to SpatialData shapes by @jp-dark in #3298
  • [python] Create xarray backend for DataArray types by @jp-dark in #3243
  • [python] Update spatial API to match somacore 1.0.23 by @jp-dark in #3274
  • [c++] Cast vertex list to WKB by @XanthosXanthopoulos in #3222
  • [c++] Geometry Dataframe by @XanthosXanthopoulos in #3212

Performance improvements

Seurat I/O

  • [r] Add initial support for ragged array writing for Seurat v5 by @mojaveazure in #2523

New shape

  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 4 of 4 by @johnkerl in #3372
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 3 of 4 by @johnkerl in #3371
  • [c++] Some use_current_domain unit-test/feature-flag teardown, part 2 of 4 by @johnkerl in #3370
  • [python/r] Remove 2.27-related feature flag by @johnkerl in #3368
  • [r/python] Revert #3300 by @johnkerl in #3358
  • [python/r] Enforce dataframe domain lower bound == 0 by @johnkerl in #3300
  • [c++] Use valid ASCII for wide-as-possible string current domain by @johnkerl in #3367
  • [python] Use new shape in one more spot by @johnkerl in #3321
  • [python] Fix bad merge of soma-shapes notebook by @johnkerl in #3309
  • [python] Tutorial notebook for the new-shape feature by @johnkerl in #3294
  • [r] Data refresh for new shapes by @johnkerl in #3303
  • [python/r] Move beyond the new-shape feature flag by @johnkerl in #3301
  • [python] Fix dense example data for core 2.27 by @johnkerl in #3296
  • [python/docs] Replace experiment directories with .tgz files in source control by @johnkerl in #3295
  • [python] Doc updates for new-shape feature by @johnkerl in #3285
  • [python] Run upgrade-shapes on notebook example experiments by @johnkerl in #3289
  • [python] Include tiledbsoma_upgrade_shape for DenseNDArray by @johnkerl in #3288
  • [python] More auto-detect of function_name_for_messages by @johnkerl in #3286
  • [r] More fixes for unit-test cases with dense + core 2.27 by @johnkerl in #3280
  • [python] Fix last 2.27+Python+dense failing test case by @johnkerl in #3269
  • [r] Fixes for dense+2.27 by @johnkerl in #3270
  • [python] Fix 3D/4D cases with core 2.27 by @johnkerl in #3268
  • [c++] Apply subarrays for dense reads and writes by @johnkerl in #3263
  • [python] Fix some dense+2.27 failing test cases by @johnkerl in #3265
  • [python] Centralize sparse/dense pybind11 shape methods by @johnkerl in #3261
  • [python] Re-run, and update, example notebooks with new shape feature by @johnkerl in #3290

Documentation infastructure

C++20

Packaging and CI

Remove TileDB-Py as a required dependency

  • [python] Update README.md to remove tiledb-py by @nguyenv in #3346

Miscellaneous internals

TileDB-SOMA 1.15.0rc3

30 Oct 14:27
d41a025
Compare
Choose a tag to compare
TileDB-SOMA 1.15.0rc3 Pre-release
Pre-release

Summary

  • NumPy 2 support
  • Python 3.12 support
  • Significant work on spatial transcriptomics
  • Progress on new shape (#2407) to be released with 1.15.0
  • The TileDB-Py package is no longer a dependency

Changes

  • [Backport release-1.15] [python] Support NumPy 2 by @github-actions in #3248
  • [Backport release-1.15] [python] Use typeguard==4.4.0, somacore==1.0.22 by @github-actions in #3254
  • [Backport release-1.15] [python] Rename set_reader_coords to set_coords by @github-actions in #3255
  • File add/removes for bulk-sync from main to release-1.15 by @johnkerl in #3258
  • Bulk-sync main to release-1.15 by @johnkerl in #3251
  • Bulk sync main -> release-1.15 by @johnkerl in #3256
  • [r] Prepare for 1.15.0rc3 by @johnkerl in #3257

Full Changelog: 1.15.0rc2...1.15.0rc3

See also: 1.15.0rc2...main

TileDB-SOMA 1.14.5

23 Oct 20:14
afca166
Compare
Choose a tag to compare

This is a single-bug-fix release

  • [Backport release-1.14] [c++/python] Check arrow_is_string_type in _update_dataframe by @github-actions in #3225
  • [r] Prepare for 1.14.5 by @johnkerl in #3226

Full Changelog: 1.14.4...1.14.5

1.14.4

22 Oct 18:32
e3ec4de
Compare
Choose a tag to compare

What's Changed

This release does not affect any Python code.

Full Changelog: 1.14.3...1.14.4

TileDB-SOMA 1.14.3

08 Oct 14:04
85339bd
Compare
Choose a tag to compare

What's Changed

Full Changelog: 1.14.2...1.14.3

TileDB-SOMA 1.15.0rc2

07 Oct 22:15
29ed055
Compare
Choose a tag to compare
TileDB-SOMA 1.15.0rc2 Pre-release
Pre-release

The main content of this release candidate is a technical preview of the new spatial-transcriptomics featureset.

New features added

  • [python] Update TileDB-SOMA to use the somacore release with spatial datatypes by @jp-dark in #3078
  • [python] Add Python implementation of the PointCloudDataFrame class by @jp-dark in #3109
  • [python/c++] Update TileDB-SOMA to somacore 1.0.19 by @jp-dark in #3098
  • [python/r/c++] Update allowed soma types and the metadata encoding version by @jp-dark in #3086
  • [python] Add implementation of the MultiscaleImage to main by @jp-dark in #3112
  • [python] Add the Python implementation of the Scene collection by @jp-dark in #3116
  • [python] Fix MultiscaleImage.read_spatial_region full region read by @jp-dark in #3128
  • [python] Create Group base class for Collection by @jp-dark in #3024
  • [python] Drop extra index array schema name by @jp-dark in #3033
  • [python] Update coordinate space <-> JSON converters by @nguyenv in #3114
  • Add Visium ingestor and sample spatial notebook by @jp-dark in #3123
  • [c++] Add SOMAMultiscaleImage class by @jp-dark in #3094
  • [c++] Add SOMAScene class by @jp-dark in #3093
  • [c++] Add SOMAPointCloudDataFrame class by @jp-dark in #3092

Breaking changes

Bug fixes

  • [Backport release-1.15] [python] Drop anndata version pin; use CSCDataset/CSRDataset by @github-actions in #3144
  • [python] Avoid RCs of dependent packages for our RC by @johnkerl in #3142
  • [Backport release-1.15] Do not fatal on 1.15-related assertions by @github-actions in #3138
  • [python] Fix retrieval of named levels from a MultiscaleImage by @aaronwolen in #3119
  • [python] Fix f-strings in multiscaleimage error messages by @aaronwolen in #3118
  • [c++] Fix bug in nnz of variant-indexed dataframes by @johnkerl in #2990

Minor changes

Full Changelog: 1.14.2...1.15.0rc2

TileDB-SOMA 1.15.0rc1

05 Oct 00:20
2221b98
Compare
Choose a tag to compare
TileDB-SOMA 1.15.0rc1 Pre-release
Pre-release

The main content of this release candidate is a technical preview of the new spatial-transcriptomics featureset.

New features added

  • [python] Update TileDB-SOMA to use the somacore release with spatial datatypes by @jp-dark in #3078
  • [python] Add Python implementation of the PointCloudDataFrame class by @jp-dark in #3109
  • [python/c++] Update TileDB-SOMA to somacore 1.0.19 by @jp-dark in #3098
  • [python/r/c++] Update allowed soma types and the metadata encoding version by @jp-dark in #3086
  • [python] Add implementation of the MultiscaleImage to main by @jp-dark in #3112
  • [python] Add the Python implementation of the Scene collection by @jp-dark in #3116
  • [python] Fix MultiscaleImage.read_spatial_region full region read by @jp-dark in #3128
  • [python] Create Group base class for Collection by @jp-dark in #3024
  • [python] Drop extra index array schema name by @jp-dark in #3033
  • [python] Update coordinate space <-> JSON converters by @nguyenv in #3114
  • Add Visium ingestor and sample spatial notebook by @jp-dark in #3123
  • [c++] Add SOMAMultiscaleImage class by @jp-dark in #3094
  • [c++] Add SOMAScene class by @jp-dark in #3093
  • [c++] Add SOMAPointCloudDataFrame class by @jp-dark in #3092

Breaking changes

Bug fixes

  • [Backport release-1.15] Do not fatal on 1.15-related assertions by @github-actions in #3138
  • [python] Fix retrieval of named levels from a MultiscaleImage by @aaronwolen in #3119
  • [python] Fix f-strings in multiscaleimage error messages by @aaronwolen in #3118
  • [c++] Fix bug in nnz of variant-indexed dataframes by @johnkerl in #2990

Minor changes

Full Changelog: 1.14.2...1.15.0rc0