[ENH] BEP031: Microscopy

[mariehbourget]

This PR aims to incorporate Microscopy to the bids-validator (BEP031, @mariehbourget, @jcohenadad)

We have encountered some issues with the schema and with an external library and would appreciate some feedback and guidance before we continue.

Questions about schema

We had a working version of the microscopy validator in commit 5d0a27e before we merged recent changes from master (commit f2588ed). If we understand correctly, the bids-validator now uses the yaml files from the schema to validate the datasets. We will make the appropriate changes related to that in order to have a working version again but we have some questions first:

1. Is the validation through yaml schema files mandatory or should we still be able to run it like the previous version?
2. When using yaml files, are those files duplicated from the bids-specification/src/schema?
3. If yes, is there any sort of "synchronisation" between bids-spec and bids-validator or should we update them in both repos?

Note: Some yaml files were not included for Microscopy in bids-spec to facilitate community review. If we have to add them to bids-validator, I would like to add them as well in bids-spec for consistency.

Questions about external libraries

For microscopy, we need to read metadata from the XML header of OME-TIFF files. We are able to do that via the exifreader, esm and xml2js libraries.
However, when installing exifreader with npm, the node_module bids-validator is automatically uninstalled.
We think we might not do the installation correctly or there might be a security issue with xmldom (a dependance of exifreader). Running npm audit gives us this message:

As we are not very experienced with javascript, we would need help with integrating and installing external librairies with the bids-validator. Thank you in advance!

[codecov]

Codecov Report

Merging #1359 (bd78f58) into master (c1c9615) will decrease coverage by 0.72%.
The diff coverage is 75.87%.

@@            Coverage Diff             @@
##           master    #1359      +/-   ##
==========================================
- Coverage   84.83%   84.10%   -0.73%     
==========================================
  Files          83       90       +7     
  Lines        3356     3631     +275     
  Branches     1011     1098      +87     
==========================================
+ Hits         2847     3054     +207     
- Misses        430      483      +53     
- Partials       79       94      +15     

Impacted Files	Coverage Δ
bids-validator/utils/files/readBuffer.js	40.00% <40.00%> (ø)
bids-validator/validators/bids/groupFileTypes.js	88.88% <44.44%> (-11.12%)	⬇️
bids-validator/validators/tsv/tsv.js	83.08% <45.09%> (-12.37%)	⬇️
bids-validator/utils/summary/collectModalities.js	91.30% <50.00%> (-1.88%)	⬇️
bids-validator/validators/bids/fullTest.js	95.34% <57.14%> (-3.39%)	⬇️
bids-validator/validators/microscopy/ometiff.js	73.61% <73.61%> (ø)
bids-validator/utils/files/readOMEFile.js	88.88% <88.88%> (ø)
bids-validator/validators/microscopy/validate.js	89.28% <89.28%> (ø)
...tor/validators/microscopy/validateTiffSignature.js	90.00% <90.00%> (ø)
bids-validator/validators/json/json.js	99.11% <94.73%> (-0.89%)	⬇️
... and 4 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update c1c9615...bd78f58. Read the comment docs.

[mariehbourget]

Hi @bids-standard/maintainers,
The implementation of the microscopy validator in this PR is complete and ready for review. 😃

Note: The ExifReader package that we use to read the XML header in OME-TIFF files seems to support only "regular TIFF" files and not "BigTIFF" files. So in cases where the files are "BigTIFF", the consistency checks between the OME metadata and the JSON metadata is skipped with a warning to the user.

Thanks to @etiennebergeron and @konantian for their help with the validator implementation! 🚀

[rwblair]

Only one comment in, in readOEMFile, thoughts on potential memory issues, doesn't necessarily have to be fixed at this time but I wanted to see if any had any opinons/thoughts on it.

Other than that any issues are addressed in this PR:
neuropoly#7