Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Faster indexing #456

Closed
wants to merge 7 commits into from
Closed

[WIP] Faster indexing #456

wants to merge 7 commits into from

Conversation

luizirber
Copy link
Member

@luizirber luizirber commented Mar 21, 2018

  • Split nodes into internal and leaves
  • allow adding leaves without updating internal nodes (trigger it later)

Checklist

  • Is it mergeable?
  • make test Did it pass the tests?
  • make coverage Is the new code covered?
  • Did it change the command-line interface? Only additions are allowed
    without a major version increment. Changing file formats also requires a
    major version number increment.
  • Was a spellchecker run on the source code and documentation after
    changes were made?

@codecov-io
Copy link

codecov-io commented Mar 21, 2018

Codecov Report

Merging #456 into master will decrease coverage by 0.68%.
The diff coverage is 82.28%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master     #456      +/-   ##
==========================================
- Coverage   89.26%   88.58%   -0.69%     
==========================================
  Files          27       27              
  Lines        4238     4414     +176     
  Branches       39       39              
==========================================
+ Hits         3783     3910     +127     
- Misses        455      504      +49
Impacted Files Coverage Δ
sourmash/__main__.py 96% <ø> (ø) ⬆️
sourmash/commands.py 85.66% <10.25%> (-3.43%) ⬇️
sourmash/sbt_storage.py 87.87% <90.9%> (+2.48%) ⬆️
sourmash/sbt.py 86.22% <94.57%> (+0.51%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0d69b79...863c255. Read the comment docs.

@luizirber luizirber force-pushed the refactor/faster_indexing branch from 25bca36 to 90b7b6b Compare May 3, 2018 20:56
@luizirber luizirber force-pushed the refactor/faster_indexing branch from 90b7b6b to 82d0c18 Compare October 17, 2018 00:25
@luizirber luizirber mentioned this pull request Oct 17, 2018
5 tasks
@luizirber luizirber force-pushed the refactor/faster_indexing branch from 82d0c18 to b4ef3f9 Compare December 18, 2018 20:53
@luizirber luizirber force-pushed the refactor/faster_indexing branch from b4ef3f9 to 210f8fc Compare January 15, 2019 04:33
@luizirber luizirber force-pushed the refactor/faster_indexing branch 2 times, most recently from 84c1e56 to 091cb98 Compare January 28, 2019 19:13
luizirber and others added 7 commits July 7, 2019 20:22
refactor SBT to keep internal nodes and leaves in separate dicts
make nodes and missing nodes 'private'
fill_up implemented and working
Disable updates during indexing
small cleanups
* start support for loading IPFS data without a local node

* start prepare command

* implement prepare cmd

* add sbt prepare test data

* more tests, fix migrate

* working on preloading

* add stub unload method

* fix test
@luizirber luizirber force-pushed the refactor/faster_indexing branch from 15b76a1 to 863c255 Compare July 7, 2019 23:58
@luizirber luizirber mentioned this pull request Jul 8, 2019
5 tasks
@luizirber luizirber closed this Mar 27, 2022
@luizirber luizirber deleted the refactor/faster_indexing branch March 27, 2022 22:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants