Reconceptualized biosample data collection

[anngvu]

Initial work on #334. Cross-check with https://dash.harvard.edu/bitstream/handle/1/32725809/Neurology-2016-Hanemann-S40-8.pdf?sequence=1&isAllowed=y.

These templates require a lot of changes in workflows that want to adopt them. I think switching over wholesale for NF is going to be difficult to manage because it's very different from HTAN/Gray Foundation, so I think the original idea was to selectively offer/pilot this for just 1-2 projects that might actually benefit.

This is a prototype (not production use yet) that we merge but need to followup with additional work of setup + testing and understand where to refine things as needed:

• Set up of a folder called "Biosample" or "Clinical Data" so data could be stored there.
• Set up and test implementation of cross-validation/linkage.
• Figure out how to extract clinical data so somehow so they still show up in the portal.

If above goes well, we'll actually:

• Advertise this new feature to the community and funders such as NTAP.
• Select and discuss rollout for projects that are a good fit for this.
• Add docs/educational materials for contributors on how to use this new and workflow.

The preview links to review are mainly:

• HumanIndividualTemplate
• AnimalIndividualTemplate

So if you think these aren't great names for the templates, I based them on ADKP (though for Gray Foundation, HumanIndividualTemplate ~ CohortCoreTemplate, which I think sounds better). We can discuss the best template naming as well.

Update:
@cconrad8 is primary reviewer.

[github-actions]

Test Suite Report

Template Generation

template	result	link
AnimalIndividualTemplate	😄	template link
BiospecimenTemplate	😄	template link
ClinicalAssayTemplate	😄	template link
EpigeneticsAssayTemplate	😄	template link
FlowCytometryTemplate	😄	template link
GenomicsAssayTemplate	😄	template link
GenomicsAssayTemplateExtended	😄	template link
HumanCohortTemplate	😄	template link
ImagingAssayTemplate	😄	template link
LightScatteringAssayTemplate	😄	template link
MRIAssayTemplate	😄	template link
PharmacokineticsAssayTemplate	😄	template link
PlateBasedReporterAssayTemplate	😄	template link
ProcessedAlignedReadsTemplate	😄	template link
ProcessedExpressionTemplate	😄	template link
ProcessedVariantCallsTemplate	😄	template link
ProteomicsAssayTemplate	😄	template link
ProtocolTemplate	😄	template link
RNASeqTemplate	😄	template link
ScRNASeqTemplate	😄	template link
UpdateMilestoneReport	😄	template link
WESTemplate	😄	template link
WGSTemplate	😄	template link

Manifest Validation

manifest	result	expectation
GenomicsAssayTemplate_0.csv	😄	Lists can be blank if attr not required using ‘list like’ rule
GenomicsAssayTemplate_1.csv	😄	Mixing blanks and regular list values works
GenomicsAssayTemplate_2.csv	😄	Conditional validation for attributes is currently not supported
ScRNASeqTemplate_0.csv	😄	Single list val works by using ‘list like’ rule
ScRNASeqTemplate_1.csv	😄	Fail because of missing data in required field libraryStrand

[cconrad8]

Nice! it looks like Manifestation Enum was already used here? so the drop downs in the google sheet are showing up with the manifestations rather than what was intended?

nf-metadata-dictionary/modules/DCC/Portal.yaml

Line 63 in 494fe54

ManifestationEnum:

[cconrad8]

And just as a note here that in Gray Foundation the structure is
'Data' > 'Patient Cohort Data'> 'synapse_storage_manifest_cohortcoretemplate.csv'
in case we want to do something similar

[allaway]

We discussed this in our 1-1.

Key takeaways:

-this is great!
-let's consider pilot this manifest in the Sarin project and in Batch 5 of the NF1 biobank project (cc @jaybee84 )
-let's take a look at the 2023 Sundby paper for any updates that we could incorporate
-let's share with NTAP to see if we can align/adopt parts of their clinical data model
-let's get feedback from Sundby and Rhodes on the NF1 CDM slack
-consider changing name of manifest as described in PR text (prefer Cohort over HumanIndividual)
-this is good to merge as-is and then we can update after feedback