Update bulk_validation_referential_integrity_check notebook to concur with refscan (no false positives)

[dwinston]

In this branch, I updated docs/nb/bulk_validation_referential_integrity_check.ipynb so that it prefers a union of a LinkML slot's any_of ranges, when present, to the value of range (cf. refscan's get_names_of_classes_in_effective_range_of_slot). Furthermore, I ditch use of concurrent.futures.ThreadPoolExecutor, as its usage results in inconsistent collections of errors from run to run. Tuning efficiency as needed will be raised as a separate issue.

Details

Now returns 33 "not found" errors and zero "invalid type" errors on using nmdc-schema v11.1.0 on /global/cfs/projectdirs/m3408/nmdc-mongodumps/dump_nmdc-prod_2024-11-25_20-12-02/nmdc, in alignment with the corresponding refscan_report.20241126_083003_UTC.schema_v11.1.0.nmdc.violations.tsv.

...

Related issue(s)

Fixes #576

...

Related subsystem(s)

• Runtime API (except the Minter)
• Minter
• Dagster
• Project documentation (in the docs directory)
• Translators (metadata ingest pipelines)
• MongoDB migrations
• Other

Testing

• I tested these changes (explain below)
• I did not test these changes

I tested these changes by...

Documentation

• I have not checked for relevant documentation yet (e.g. in the docs directory)
• I have updated all relevant documentation so it will remain accurate
• Other (explain below)

Maintainability

• Every Python function I defined includes a docstring (test functions are exempt from this)
• Every Python function parameter I introduced includes a type hint (e.g. study_id: str)
• All "to do" or "fix me" Python comments I added begin with either # TODO or # FIXME
• I used black to format all the Python files I created/modified
• The PR title is in the imperative mood (e.g. "Do X") and not the declarative mood (e.g. "Does X" or "Did X")

[dwinston]

Merged in order to focus reviewer bandwidth on #696. As is, this PR does not "operationalize" anything.

[eecavanna]

FYI, @dwinston, as of about 24 hours ago (via this PR), the daily dumps are being created with the --gzip CLI option to mongodump. Before that, the --gzip CLI option was not being included. So, the mongorestore command here may require a change (e.g. the addition of its --gzip CLI option—assuming you're restoring a dump that was created after about 24 hours ago).

[dwinston]

Interestingly, the mongorestore works without an explicit --gzip. smart.