Arbitrary directories indexed in version 0.7.0

[gkiar]

Hey team,

I'm using the new 0.7.0 release and encountered an "undocumented feature" that's been a bit annoying for me. First of all, thanks for the descriptive release notes - it was really easy to transition from 0.6.5 -> 0.7.0 just going through the list.

What I'm experiencing is that I have arbitrary directories in my dataset, such as "derivatives_old" and "derivatives_phacked_save_for_later", etc., and while derivatives isn't being indexed automatically, and then some files in sub-directories there are being identified erroneously as my raw data file.

I can't seem to find a way to cleanly exclude these from being indexed. Am I just breaking bids and should put these all inside the derivatives folder?

[tyarkoni]

Yes, you're breaking BIDS. Per the spec, derivatives have to go either under derivatives/, or in a completely separate location not under the raw project.

That said, it would be a good idea to have the ability to exclude arbitrary directories/files from indexing (and I'll come back to this at some point once the 0.8 refactoring is done). One way you can do that at the moment is to drop a config file (I believe it's named layout.json by default) into your BIDS project, and then specify an exclude directive that lists the directories you want to exclude. This is not well-documented (you'll have to look at the grabbit code and test examples), but should work. But probably the better solution is to get your derivatives into derivatives, at which point it's easy to select which ones to index.

[gkiar]

Noted - thanks!

[yarikoptic]

oh, @AdinaWagner , look above at the hidden treasure disclosed by Tal which could be of help ;-)

[adswa]

already drafting one of these and looking forward to get rid of our hardcode-hacking ;-)

[tyarkoni]

Keep in mind this hidden treasure will be going away fairly soon...

[tyarkoni]

This is now supported in 0.8 via the ignore argument, so I'm closing the issue.