Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[MRG] comments update for ZipFileLinearIndex #1442

Merged
merged 97 commits into from
Apr 4, 2021
Merged
Changes from all commits
Commits
Show all changes
97 commits
Select commit Hold shift + click to select a range
6bacb41
implement a simple ZipFileLinearIndex class
ctb Feb 25, 2021
61245eb
fix load_file_as_signatures
ctb Feb 25, 2021
31b0ca4
Merge branch 'latest' of github.com:dib-lab/sourmash into add/zipfile…
ctb Feb 26, 2021
b22248b
add tests for zipfile searching etc.
ctb Feb 26, 2021
8c9c9ae
add sig describe test for loading from zipfile
ctb Feb 26, 2021
7087d82
fix load_file_as_index to support zipfiles
ctb Feb 26, 2021
c75bd83
rename force; add docstrings
ctb Feb 26, 2021
da3a58c
Merge branch 'latest' into add/zipfile_index
ctb Mar 5, 2021
92e5fdc
add an IndexOfIndexes class
ctb Mar 6, 2021
5c71e11
rename to MultiIndex
ctb Mar 7, 2021
85efdaf
switch to using MultiIndex for loading from a directory
ctb Mar 7, 2021
04f9de1
some more MultiIndex tests
ctb Mar 7, 2021
201a89a
add test of MultiIndex.signatures
ctb Mar 7, 2021
07d2c32
add docstring for MultiIndex
ctb Mar 7, 2021
61d15c3
stop special-casing SIGLISTs
ctb Mar 7, 2021
16f9ee2
fix test to match more informative error message
ctb Mar 7, 2021
c6bf314
switch to using LinearIndex.load for stdin, too
ctb Mar 7, 2021
dd0f3b8
add __len__ to MultiIndex
ctb Mar 8, 2021
9211a74
add check_csv to check for appropriate filename loading info
ctb Mar 8, 2021
75069ff
add comment
ctb Mar 8, 2021
d2294fb
Merge branch 'latest' of github.com:dib-lab/sourmash into add/multi_i…
ctb Mar 9, 2021
9f39623
fix databases load
ctb Mar 9, 2021
ac63cf8
more tests needed
ctb Mar 9, 2021
d5059eb
Merge branch 'latest' into add/multi_index
ctb Mar 9, 2021
3e06dbf
Merge branch 'latest' of github.com:dib-lab/sourmash into add/multi_i…
ctb Mar 9, 2021
5590d70
add tests for incompatible signatures
ctb Mar 9, 2021
14891bd
add filter to LinearIndex and MultiIndex
ctb Mar 9, 2021
40395ff
clean up sourmash_args some more
ctb Mar 9, 2021
8c51452
Merge branch 'latest' of github.com:dib-lab/sourmash into add/multi_i…
ctb Mar 9, 2021
fbf3bb9
Merge branch 'latest' into add/multi_index
ctb Mar 12, 2021
abd84b2
Merge branch 'latest' of github.com:dib-lab/sourmash into add/zipfile…
ctb Mar 12, 2021
dd52be6
Merge branch 'latest' of github.com:dib-lab/sourmash into add/multi_i…
ctb Mar 24, 2021
f377dc4
shift loading over to Index classes
ctb Mar 24, 2021
250c49a
refactor, fix tests
ctb Mar 24, 2021
9a921f9
switch to a list of loader functions
ctb Mar 25, 2021
780fb71
comments, docstrings, and tests passing
ctb Mar 26, 2021
d261963
update to use f strings throughout sourmash_args.py
ctb Mar 26, 2021
4b4174e
Merge branch 'latest' of github.com:dib-lab/sourmash into refactor/db…
ctb Mar 26, 2021
93fca04
add docstrings
ctb Mar 26, 2021
0203357
update comments
ctb Mar 26, 2021
cd53f02
Merge branch 'latest' of github.com:dib-lab/sourmash into refactor/db…
ctb Mar 26, 2021
8a0200a
remove unnecessary changes
ctb Mar 26, 2021
e9df90f
revert to original test
ctb Mar 26, 2021
9e427e3
remove unneeded comment
ctb Mar 26, 2021
0dd390a
clean up a bit
ctb Mar 26, 2021
2c0ee29
debugging update
ctb Mar 27, 2021
edcb483
better exception raising and capture for signature parsing
ctb Mar 27, 2021
3f6c3f2
more specific error message
ctb Mar 27, 2021
78dbb1d
revert change in favor of creating new issue
ctb Mar 27, 2021
229b1d7
add commentary => TODO
ctb Mar 28, 2021
20ed9f0
add tests for MultiIndex.load_from_directory; fix traverse code
ctb Mar 28, 2021
16a119e
switch lca summarize over to usig MultiIndex
ctb Mar 28, 2021
cb1e8a3
switch to using MultiIndex in categorize
ctb Mar 28, 2021
c9e176d
remove LoadSingleSignatures
ctb Mar 28, 2021
8f914f1
test errors in lca database loading
ctb Mar 28, 2021
a43b011
remove unneeded categorize code
ctb Mar 28, 2021
15328ae
add testme info
ctb Mar 28, 2021
f674232
verified that this was tested
ctb Mar 28, 2021
01c54c0
remove testme comments
ctb Mar 28, 2021
ae3f66d
add tests for MultiIndex.load_from_file_list
ctb Mar 28, 2021
7f52d7c
refactor select, add scaled/num/abund
ctb Mar 28, 2021
dde14fd
more work
ctb Mar 28, 2021
3f498a4
catch ValueError from db.select
ctb Mar 29, 2021
df19926
update debug print to sys.stder
ctb Mar 29, 2021
e8233ca
fix scaled check for LCA database
ctb Mar 29, 2021
b44c3cf
add debug_literal
ctb Mar 29, 2021
7133ac1
break things when filter returns empty Index
ctb Mar 29, 2021
f5f1c9c
fix scaled check for SBT
ctb Mar 29, 2021
d6f156f
fix a few tests
ctb Mar 30, 2021
785a9a4
fix LCA database ksize message & test
ctb Mar 30, 2021
23d7ac4
flag for removal
ctb Mar 30, 2021
efc07cd
Merge branch 'latest' of github.com:dib-lab/sourmash into refactor/si…
ctb Mar 30, 2021
12399e7
add 'containment' to 'select'
ctb Mar 31, 2021
b6a4dff
Merge branch 'latest' into refactor/db_load_multiindex
ctb Mar 31, 2021
2b7acb9
fix remaining tests
ctb Mar 31, 2021
f663426
Merge branch 'refactor/db_load_multiindex' into refactor/siglist_loading
ctb Mar 31, 2021
9aae1cb
update comments
ctb Mar 31, 2021
2630be2
remove all the cruft, yay
ctb Mar 31, 2021
4f1a7fe
added 'is_database' flag for nicer UX
ctb Mar 31, 2021
736ddf3
remove overly broad exception catching
ctb Mar 31, 2021
16719ce
add docstrings
ctb Mar 31, 2021
6d8663e
document downsampling foo
ctb Mar 31, 2021
9832810
Merge branch 'latest' of github.com:dib-lab/sourmash into add/zipfile…
ctb Apr 1, 2021
4854325
Merge branch 'refactor/siglist_loading' into merge
ctb Apr 1, 2021
c4de8fb
update for additional test files
ctb Apr 1, 2021
31194bf
update ZipFileLinearIndex for new selector criteria
ctb Apr 1, 2021
be502ab
remove leftover code fragment
ctb Apr 1, 2021
c072866
Merge branch 'latest' of github.com:dib-lab/sourmash into add/zipfile…
ctb Apr 2, 2021
ec13d03
add zipfile API tests; use .location
ctb Apr 3, 2021
0339ed7
update docs to include zipfile collections
ctb Apr 3, 2021
9b016dd
add zipfile loading tests
ctb Apr 3, 2021
7c0e54a
add __len__ to ZipFileLinearIndex and test MultiIndex load of zipfile
ctb Apr 3, 2021
d755f95
Update doc/command-line.md
ctb Apr 3, 2021
b0f5241
Merge branch 'latest' into add/zipfile_index
ctb Apr 3, 2021
6ce3ce5
add test of incompatible sig search for zipfile
ctb Apr 3, 2021
56184ff
document load_signatures behavior for zipfile
ctb Apr 3, 2021
7adf978
Merge branch 'latest' of github.com:dib-lab/sourmash into doc/zipfile…
ctb Apr 3, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions src/sourmash/index.py
Original file line number Diff line number Diff line change
Expand Up @@ -268,6 +268,9 @@ def signatures(self):

# now load all the signatures and select on ksize/moltype:
selection_dict = self.selection_dict

# note: if 'fp' doesn't contain a valid JSON signature,
# load_signatures will silently fail & yield nothing.
for ss in load_signatures(fp):
if selection_dict:
if select_signature(ss, **self.selection_dict):
Expand Down