BEP Proposal: Atlas specification

[jdkent]

Your idea

EDIT

github draft for current discussion: https://github.com/PESTILLILAB/bids-specification/blob/bep038/src/atlas.md

the google doc draft (deprecated in favor of the github draft) : https://docs.google.com/document/d/1RxW4cARr3-EiBEcXjLpSIVidvnUSHE7yJCUY91i5TfM/edit?usp=sharing

[PeerHerholz]

We currently aim to organize a meeting to further discuss the atlas-related BEP developments in #1379.

[CPernet]

yo, count me in

[PeerHerholz]

Hi @bids-standard/maintainers,

as we aim to share our draft with the community to receive and integrate feedback soon, we would like to ask if it would be possible to move our BEP to the next stage. More specifically, this refers to step 5 of the BEP development guide, ie making the BEP official and obtaining a number. We followed and conducted steps 1-3 and would now like to access if the requirements for step 4 were fulfilled and if not, what would be missing to achieve this.

We are aware of the current status concerning the two atlas-related BEPs and one of our team (myself) will attend the next maintainers meeting to discuss if the outlined procedure is feasible. However, based on the so-far positive nature of the received feedback on this matter we nevertheless already wanted to ask regarding the further steps.

If you have any questions, please don't hesitate to ask.
We're looking forward to working with you on this.

Best, Peer

[PeerHerholz]

Hi everyone,

I just wanted to follow up on my earlier message: would it be possible to evaluate if the requirements for step 4 were fulfilled and if not, indicate what we would need to do to achieve this?

It would be great to hear from you.

Best, Peer

[CPernet]

@effigies you were there at the steering group meeting, and while we are still a little unsure of some specific aspects, I cannot see any reasons not to move on stage 4 - IMO +1

[sappelhoff]

reposted from other thread:

Hi @PeerHerholz, thanks for the patience -- I am not sure I'm the correct person to judge this, but to obtain a number you may open a PR to the bids-website repo by editing these files. We can then discuss in that PR and potentially approve and merge it, and then you'll have the official number :-)

• https://github.com/bids-standard/bids-website/tree/gh-pages/_bep_collection
• https://github.com/bids-standard/bids-website/blob/gh-pages/_data/beps.yml

[PeerHerholz]

Hi @sappelhoff,

thanks for getting back to us and the information. We opened a respective PR here.

Looking forward to the discussion!

Cheers, Peer

[PeerHerholz]

Hello @bids-standard/maintainers, @bids-standard/steering & everyone,

I hope you're doing fine.

We (@jdkent, @dorahermes, @francopestilli, @dlevitas, @poldrack, @CPernet, @melanieganz, @mnoergaard, @mathesong, @bendhouseart, @MaxvandenBoom and others) continued to work on BEP038 - Atlases. We would like to finalize Step 9 of the BEP development process: Incorporate the feedback and strive for consensus. . Thus, I thought about creating a dedicated issue within which we can track what is still needed in order to do so.

If y'all could have a look at the draft again and share your thoughts re the current status and if something/what definitely still needs to be addressed, that would be great! I'll start a list below and will keep editing it based on your comments.

Needs to be addressed

• PR [SCHEMA][ENH] Remove atlas entity and replace it with seg in prep of BEP038 #1579
• Electrophysiology example/pointer
• "atlas" as "DatasetType", ie no modality tag(s)
• outline differences and files of BEP17

Would be cool to address but not necessary

Thanks so much again for all your help and effort, we highly appreciated it.

[melanieganz]

Thanks @PeerHerholz! And please check out the atlas PR and voice your opinions, so we ensure to address all concerns before moving forward.

[PeerHerholz]

Thanks @melanieganz! I/we keep monitoring the PR closely but so far we haven't noticed any major issues concerning the BEP. IMHO it's better to see what the PR leads to and then adapt the BEP respectively as this would greatly enhance adaptation and streamline the process (I think for the BEP this would most likely mean smaller changes, ie changing some definitions and naming patterns).

[PeerHerholz]

Hi everyone,

given that #1579 was merged, I think we can proceed with finalizing the draft.

Could I please ask everyone, the moderators/leads (@eduff, @jdkent) and contributors (@dorahermes, @francopestilli, @dlevitas, @poldrack, @CPernet, @melanieganz, @mnoergaard, @mathesong, @bendhouseart, @MaxvandenBoom, @dlevitas) to have another look at the BEP and indicate problems that should be addressed before we're moving forward (except the ones listed above)?

Thank you very much for your continued supported, we highly appreciate it.

Cheers, Peer

[CPernet]

• PR [SCHEMA][ENH] Remove atlas entity and replace it with seg in prep of BEP038 #1579
• Electrophysiology example --> given the scope, we said no need of that - correct?
• "atlas" as "DatasetType", ie no modality tag(s) --> I still think we need to constrain a little more the _*map.[ext] proposal but also we should move along with it to progress

[PeerHerholz]

Thx for the reply and comments @CPernet!

Electrophysiology example --> given the scope, we said no need of that - correct?

Sure thing, we can remove it for the time being and add one if needed.

"atlas" as "DatasetType", ie no modality tag(s) --> I still think we need to constrain a little more the _*map.[ext] proposal but also we should move along with it to progress

I'm actually not sure if #1602 applies here, ie for atlases, or did you mean PET-related things more generally? However, I agree that we should try to move forward.

[PeerHerholz]

@melanieganz I added the needed discussion re BEP017 files to the list.

[PeerHerholz]

@melanieganz and @CPernet, I was trying to come up with an approach to exclude BEP017-related files in this BEP for the time being but am not sure how to implement this best. We could simply ignore them, knowing that we will introduce/add them later, ie once BEP017 is through but I wanted to ask for feedback re this. WDYT?

[CPernet]

is that the files to check? PR are fine but a pain to edit, can you invite people to your fork instead

[PeerHerholz]

Hi @CPernet,

sorry for my unclear message. I was referring to the _relmat files that we aim to introduce via BEP017 but already use in this BEP to denote e.g. spatial distance between nodes of an atlas.

[CPernet]

?? @melanieganz and I were wondering about community feedback -- posting the google doc to the community?

[PeerHerholz]

Hi @CPernet,

the google doc was shared earlier this year and promoted at different events (e.g. conferences and workshops). Are you suggesting to make another call for feedback?

Cheers, Peer

[CPernet]

well no one else but 'us' look at it, and that's a little problematic -- nothing says we can keep working on the markdown, but another public email would be good IMO

[CPernet]

re good your point about relmat -- sine we want this BEP first, we have to define it here.

[CPernet]

can you point me to the markdown doc? do you need help - it is an important extension for Open Neuro PET so @bendhouseart can definitively spend time on that

[PeerHerholz]

Hi Cyril,

well no one else but 'us' look at it, and that's a little problematic -- nothing says we can keep working on the markdown, but another public email would be good IMO

yeah, I would be cool to get more ideas and feedback. However, I'm not sure how likely this is given the BEPdevelopment process so far, of this and other BEPs. It seems to be the case that the development of most BEPs is done by the group of folks initially proposing the BEP and then supported by the steering group/maintainers and adjacent folks. We're currently brainstorming ideas to address this and make the BEP development process more streamlined (ie find ways to engage more folks and don't get stuck in a waiting for feedback-loop).

re good your point about relmat -- sine we want this BEP first, we have to define it here.

So, introducing the files here but outlining that they're part of BEP017 (and thus could change respectively)? I think @melanieganz proposed a different solution (IIRC, sorry if I didn't).

can you point me to the markdown doc? do you need help - it is an important extension for Open Neuro PET so @bendhouseart can definitively spend time on that

We're still working on the Google Doc, as we want to follow the BEP development guidelines as closely as possible. Ie, moving only to GitHub and markdown once the draft is considered mature enough.

P.S.: Sorry for closing and re-opening the issue, I hit buttons by mistake.

[CPernet]

Yes - let's not get stuck in the waiting loop ... I say let's fork the repo and start the markdown. In parallel, send another community email JIC.

[PeerHerholz]

Hi folks,

as we would like to bring this BEP to the next stage of the development process, ie moving it to GitHub, we would like to

• address the, currently, final discussion point, ie files related to BEP017 (@melanieganz WDYT?)
• afterwards "freeze" the draft of the BEP and get feedback from the maintainers and steering group re if we can advance

The second point will also include another announcement over the mailing list and social media to ask others for feedback.
Please let us know if you don't agree with this plan.

Cheers, Peer

[melanieganz]

Dear @PeerHerholz,

sorry, for leaving you hanging, I simply missed this thread.

So regarding the files related to BEP017 (see below your earlier comment), I would vote for not mentioning them in the atlas BEP at all. They are no introduced yet in the spec and otherwise we would have to wait for BEP017 before merging the straightforward and basically relatively finished BEP038. So I would remove them for now and then mention in BEP017 that files can be saved according to the atlas spec and give examples there. And then one can always later on make a minor change and add more atlas examples once BEP017 is finished and merged.

I just think it's a bad idea to keep holdign the straightforward atlas BEP that goes way beyond BEP017 back.

So remove those, freeze draft and get a date with the maintainers and afterwards steering group scheduled.

[melanieganz]

Just to clarify, @oesteban here you are addressing cons of @PeerHerholz point 1, correct?

Point 1 is regarding The "derivative" dataset type

Wrt the status of the derivatives, the common derivatives define some guidelines for the derivatives, but there are approximately 5-6 other BEPs going on that also define this (BEP011, 012, 016, 023 per modality plus BEP017 and 038). So this is a huge ongoing effort between all modalities and there are not fixed decisions yet, correct?

So we do agree that
a) derivatives are not yet properly defined and b) it is questionable at this point if derivatives ever will be able to be validated as we know do for raw data.

So what exactly is the issue with defining an atlas when as a derivative with the examples provided in the PR? They do not break any of the existing modality specific derivatives - we took great care to invite and solicit feedback from other modalities like the EEG, MEG and MR crowd and based our experience on PET.

Can you please in one to two sentences formulate the a con list? The whole argumentation is quite convoluted for me to follow.

[melanieganz]

And @oesteban wrt your other reply which I believe addresses point 3 of @PeerHerholz's list just as I my comment did. Your con would be that the atlas should be named differently and that it's not versionable?

I would like to address this in two parts:

1. Naming - the atlas you are referring to is a PET atlas and it is using a controlled vocabulary defined in the PET raw specification. The naming follows the definitions defined in PET raw and it's called PS13 due to the radiotracer. A PET atlas can depend on the tracer used and in this case it's just the distribution of 11^[C]PS13, but in other cases it can be an absolute measure and then it can be relevant to name it by the receptor (e.g. 5-HT_2A), but to still indicate the tracers used (11^[C]Cimbi36) since it depends on the tracer.

2. I see exactly the strength of validation and versioning in sharing atlasses on Openneuro as standalone datasets! And therefore I would like to be able to validate atlasses and share them traceably on Openneuro. Because then I can totally refer to a specific version of a dataset on Openneuro.
   And again, I think we conceptualize atlases very differently. I see an atlas like I also defined it in the .yml file. It's not something that gets updated all the time.
   For example, our work on serotonin atlases will not be able to be repeated by anyone for some time - simply because there is not a lab in the world that has that amount of serotonin images. And this is exactly why it is so useful for the rest of the neuroimaging community and why it is worth to be able to share them standardized and clearly understandable.
   We had always shared them openly (which has actually lead to some people just re-distributing them - even wrongly because they misunderstood how stuff was processed - under a different license). But the standard and how we shared them wasn't defined which made reuse hard, gave rise to misunderstandings and misuse. Now we are defining the standard and we will be creating even more stand alone atlases with all the PET data we can get our hands on in order to enable e.g. fMRI people to get a seed region that is actually defined by the relevant functional region such as the D1 or D2 distribution and not by structural regions. I believe this has a huge potential for neuroscientific research.

[pwighton]

Hi All,

I'm a big fan of this BEP and would like to thank @PeerHerholz and everyone else for their hard work putting this together.

I have some questions about suffixes. Of the three suffixes introduced (_desg/_probseg/_mask), none of them seem suitable for quantitative maps. Should we introduce a new suffix (_scalar or _qmap or something) for quantitative maps? Also, the suffix _mimap (molecular imaging map) used in the PS13 example, which is the preferred suffix of the OpenNeuroPET group is not currently defined.

I'm not exactly sure how this process works, but I'm happy to open a PR to the bep038 branch of PESTILLILAB/bids-specification to address this if folks are agreeable.

[effigies]

dseg, probseg and mask are defined in derivatives: https://bids-specification.readthedocs.io/en/stable/derivatives/imaging.html

mimap will need to be addressed by BEP23.

[francopestilli]

hi @oesteban @CPernet @melanieganz @yarikoptic @effigies etc,

Thanks for the heartful discussion. I have been observing. I want to move forward with the BIDS-Connectivity project this BEP needs to be resolved.

A summary of what I understand the issues might be (more things have been said, I understand but I am trying to close):
-A- Lack of distinction between Atlas and Template.
-B- Location of where Atlases and Templates should be saved.

As a general background, Templates in this context are Atlases in a specific space (say the sub-01 space). Atlases instead are

(1) If we change, the name from Atlas to Template (see A). @oesteban believes this BEP becomes unnecessary because derivatives can already be saved.
(2) If we do make this an Atlas BEP and not a subject-specific template we need to agree on a location to save the Atlases
(3) Perhaps a combination of A and B is necessary?

Recently, @TheChymera and @yarikoptic started what seems to be a very similar BEP here, should these two be merged or the discussion coordinated?

In general, Atlases are coming to be recognized here as something that BIDS will need to manage and data archives and platforms will need to be able to host atlases for purposes of analysis of individuals or groups.

@oesteban can you try to make a proposal that does not require the deletion of the current BEP, but a either a change in its operations or a repurpose? Otherwise, I fear the support seems to go in favor of keeping a version of this BEP.

But I do not want to be the judge of that, we had a similar roadblock for a different issue for DWi and I think we have mechanisms in place to resolve the issue, but I was hoping we could find a solution here before we go that route.

[Edited for clarity]

[TheChymera]

(2) [...] started what seems to be a very similar BEP here [...]

Well, if this is almost-finished, and we're just derailing the discussion, perhaps this could be a subsequent BEP. It's not just that one dataset, there are numerous “libraries” of some feature or another, which I think are best contextualized as atlases. They're no different from segmentation areas in the sense that they can be used as spatial references, and the things which are important for them are a superset of what's covered in BEP038. Resolution, corresponding template, that's all good.

The two I've been using are the ABI gene expression and the ABI connectivity atlases, and what they need would be a seed- and expression- field, though that's too specific. What I am thinking, is perhaps a feature- field would be good, as an adaptable key-value pair for to the core aspect of the specific atlas. Alternatively the individual feature maps could be stacked in an additional dimension, and that dimension could be documented via the JSON, though I find that much less transparent, and it would create enormous files due to the wealth of features and that these are not binary maps.

In any case let me know, here on the other issue. I'd be happy to help with either extending this, or starting a new BEP.

[francopestilli]

(2) [...] started what seems to be a very similar BEP here [...]

Well, if this is almost-finished, and we're just derailing the discussion, perhaps this could be a subsequent BEP. It's not just that one dataset, there are numerous “libraries” of some feature or another, which I think are best contextualized as atlases. They're no different from segmentation areas in the sense that they can be used as spatial references, and the things which are important for them are a superset of what's covered in BEP038. Resolution, corresponding template, that's all good.

The two I've been using are the ABI gene expression and the ABI connectivity atlases, and what they need would be a seed- and expression- field, though that's too specific. What I am thinking, is perhaps a feature- field would be good, as an adaptable key-value pair for to the core aspect of the specific atlas. Alternatively the individual feature maps could be stacked in an additional dimension, and that dimension could be documented via the JSON, though I find that much less transparent, and it would create enormous files due to the wealth of features and that these are not binary maps.

In any case let me know, here on the other issue. I'd be happy to help with either extending this, or starting a new BEP.

I think this request (adding feature- is similar to @pwighton's request in our discussion here.

it seems that we need ways to define content types of images in an Atlas. Does it seem reasonable?

[TheChymera]

ways to define content types of images in an Atlas

Yes. The way I understand it, and perhaps I don't, is that currently the BEP assumes parcellation atlasses. The category range is defined broadly “such as landmarks, labeled brain regions, parcellated surfaces, and other stereotaxy-referenced tables, or textual matter” but I see that as all boiling down to parcellation.

it seems that we need ways to define content types of images in an Atlas

I would agree, but is this something which should be done now, perhaps at the cost of delaying the BEP, or is it something that can be easily tacked on later?

[CPernet]

case 1 with atlas shared at the root works fine - it is compliant with the current spec (except the entity atlas) , @oesteban was fine with this, I think (TBC).

@TheChymera this does NOT boil down to parcellation - re qMRI and PET (ie quantitative voxel wise values)
@francopestilli Changing atlas to template does not reflect how we thought about that - but @oesteban proposed an alternative IMO @effigies distinction space -- template -- atlas was the most meaningful

I must admit - I'm also puzzled by how to move forward.

[PeerHerholz]

Hello everyone,

sorry again for the late reply but I'm most likely afk for the rest of this week as well but we will be back next week.

Just to follow up on the discussion on how to move forward. IMHO it becomes increasingly more challenging to follow all the discussion points and arguments in this thread. Please don't get me wrong: I think all of them are warranted and very helpful. However, I also think a more precise and clear overview would be beneficial for folks. This essentially goes back to my proposal of a collection of use cases/examples based on the different positions that were brought up (ie atlasvs. tpl, at root, etc.) and respective pros and cons. With that, we would be able to see how things would look like/would have to be implemented and can collect benefits/disadvantages wrt the current spec and (potential) further developments. This collection could then also serve as the basis for a potential vote, should we decide to go that route.
This collection could be added to the GoogleDoc draft so that we have everything in one place and everyone interested can participate.

WDYT?

Cheers and thanks again, Peer

[oesteban]

Meta/moderation of the BEP:

@PeerHerholz @francopestilli @melanieganz @effigies @CPernet - I'll shortly send an email proposing the logistics of how to go forward with this BEP. As you'll see in the email, it's definitely not perfect, but I think it could be piloted as a more specific way for all BEPs to work out. I agree with Peer that the process of BEPs is very undefined beyond "we discuss on Google Docs until a PR to the specs is created". Most (if not all) BEPs struggle to have such a clean and straightforward trajectory.

Specific answers in this BEP:

@francopestilli
@oesteban can you try to make a proposal that does not require the deletion of the current BEP, but a either a change in its operations or a repurpose?

I didn't get a sense that my effort will be considered, so I didn't go ahead and try. But I can give it a go if it will be actually considered - I don't think it requires much (beyond the main talking point we've already initiated above).

@francopestilli
@oesteban believes this BEP becomes unnecessary because derivatives can already be saved.

Not at all, and please, I would ask everyone not to simplify my interventions this way because it very badly represents my view (@francopestilli, I know this is a consequence of my poorly explaining rather than an intended oversimplification).

The real (and very important) value of this BEP is on the metadata descriptions. That is already there, and I am in favor of that. The structure may change, but the metadata descriptions will remain practically unmodified. It worries me a little that the discussion is perceived as a complete slamming of the BEP when it is not.

One potential way to move this forward more rapidly is to split the atlas metadata and the file structure into two different BEPs. I can anticipate that atlas metadata will be a low-hanging fruit to go in smoothly; I see the file structure and the new DatasetType as very problematic for the points discussed above and below. (highlight for @PeerHerholz @francopestilli)

@melanieganz
b) it is questionable at this point if derivatives ever will be able to be validated as we know do for raw data.

This admittedly worries me. If the validator is "flexibilized" with derivatives, as a result of realizing that derivatives is such a wild space that the validation cannot be implemented, then I would stop here and recommend everyone to move into a self-sufficient data description such as NIDM and hope that the NIDM-BIDS BEP sees the light one day.

This also deflects the discussion over the point that the new DatasetType opens the door to having datasets with two possible (and equally valid) encodings within BIDS. This is problematic (with dangerous consequences to reproducibility) beyond whether the validator can distinguish them and also needs addressing. For more on this point, please check my PR #1530 (although I see this duality way more problematic than what my PR tried to address).

@melanieganz

1. Naming - the atlas you are referring to is a PET atlas and it is using a controlled vocabulary defined in the PET raw specification

It seems my argument was poorly delivered. I'm not suggesting it doesn't follow the BIDS specs. What I meant is that, if the DatasetType were to be "atlas", then BIDS will need some sort of "registry" of names so that your choice "PS13" is not overshadowed by another researcher, and when papers come out, mentioning "PS13" unambiguously refers to your template (atlas), not others that happen to have been called the same way. As we describe in the templateflow paper, ambiguity is a big issue for reproducibility and expanding flexibility here will have negative consequences overall.

@melanieganz
I see exactly the strength of validation and versioning in sharing atlasses on Openneuro as standalone datasets! And therefore I would like to be able to validate atlasses and share them traceably on Openneuro. Because then I can totally refer to a specific version of a dataset on Openneuro.

There are a few issues I see with this:

• My first point is that you can already share atlases on OpenNeuro as a standalone derivative dataset. You will be able to do so more reproducibly when all the mandatory and recommended metadata prescribed by this BEP are integrated in the specs. BUT, as a derivative dataset. Creating an alternative view (as in, a numpy array view) of the dataset as an "atlas" DatasetType poses big issues to the BIDS Validator as both are totally valid, and reporting having used the WhateverAtlasName atlas will not univocally point at the actual atlas used (unless the authors of the paper are careful in reporting, which is not a given as it is clear in the abundant literature and particularly Carp's papers). This stems from the fact that, while a numpy array view points at a single array in memory, here we are talking about different actual datasets, and therefore, breaking the synchrony will be unavoidable. You could probably fine tune DataLad (or similar solutions such as just symlinks in a filesystem that supports it) to have these views linking against the same annex, but I don't think it is a good idea for BIDS to count on that.
• This is something TemplateFlow is already doing today. We decided to start TemplateFlow because we saw this specific domain of neuroimaging uncovered by BIDS and we anticipated that BIDS would not ever [fully] cover it. This is to say: I think it is great how this BEP will help disseminate atlas-like derivatives more reproducibly with a common language of metadata items and values, but I don't think BIDS can resolve the problems of acting as a registry of atlas names or rely specifically on OpenNeuro's (or, say, TemplateFlow, for the same reason) features to distribute atlases.

@yarikoptic thanks a lot for the paper and the picture you posted. Following the figure with the ontology, we can clearly see how BIDS encodes the most important concept -the subject- on the left side** -yellow, "spatial module"- (with session IMHO omitted as assumed) with entities and file structure tools, and the concepts on the right side (green, "semantic module") more predominantly with metadata. The argument I'm trying to bring about is that template is a concept that falls within the spatial module of the ontology, while atlas is a semantic construct (right side of the ontology).

** EDIT: changed wording order for clarity

[poldrack]

This thread has honestly gone way past the point where I have time to follow it in detail, but I do want to respond to @oesteban's last point: The argument I'm trying to bring about is that template is a concept that

falls within the spatial module of the ontology, while atlas is a semantic construct (right side of the ontology).

Isn't an atlas really a mapping between the semantic domain and the spatial domain? e.g. mapping of a particular anatomical region or neurotransmitter density (semantics concepts from neuroscience) into a location in the sterotactic space?

[oesteban]

Isn't an atlas really a mapping between the semantic domain and the spatial domain? e.g. mapping of a particular anatomical region or neurotransmitter density (semantics concepts from neuroscience) into a location in the sterotactic space?

I would not dare contradict your argument(s) @poldrack :D.

My point was more about Yarik's paper - within the thought framework of the figure he pasted there, it becomes clear that templates belong more on the left side (spatial domain) and atlases more on the right side (semantic domain). Obviously, there are many grades and colors to definitions and concepts, and so I agree there's a connection between the idea of an atlas and the spatial domain, which goes through the idea of template as I discuss next.

Circling back to your question, Isn't an atlas really a mapping between the semantic domain and the spatial domain?, of course, yes, but as very nicely put by @effigies above, that mapping needs to go necessarily through the concept of templates (T1w, T2w, or any other MRI, radioligand receptor binding, etc.), which map a particular property that may be used to engender individual or group stereotaxy.

Obviously, the actual brains Talairach used to engender stereotaxy and establish a reference where the brain could be atlased are way less semantic or abstract than digital brains obtained with 3D imaging.

This is to make an epistemic argument supporting the equivalence between template (averaging individuals or repetitions of a single individual) and subject as a coordinate-system-engendering spatial reference.

There may be a day when the scanner will produce TheOneAndBestAtlas' parcellations and region-wise averaged fMRI time series of the brain directly - but I doubt it will allow us to discard the original fMRI time series or the anatomical images when retrieving the data from the scanner. Because then that atlas' reference is lost, and any possibility of understanding how things were generated with it also is.

EDIT: Perhaps one point was missing above, in the sense that @yarikoptic's suggestion of looking into the paper he proposed is to me very important - it would allow moving from "how individuals look at this problem" (i.e., in terms of I like this or that) into "how this problem can be categorized and resolved with continuity to BIDS' principles and foundations".

[CPernet]

@oesteban what we need in the google doc is the proposed changes for cases 1, 2, 3 - along with whatever name changes are needed (maybe use a different colors) -- then we can ask the technical committee to decide since otherwise we are stuck -- that rule was decided by the steering group committee last week (to be merged FYI https://github.com/bids-standard/bids-extensions/pull/29/files)

[oesteban]

@CPernet

@oesteban what we need in the google doc is the proposed changes for cases 1, 2, 3 - along with whatever name changes are needed (maybe use a different colors) -- then we can ask the technical committee to decide since otherwise we are stuck -- that rule was decided by the steering group committee last week (to be merged FYI https://github.com/bids-standard/bids-extensions/pull/29/files)

This is a misinterpretation of bids-standard/bids-extensions#29 for the reasons I have provided there bids-standard/bids-extensions#29 (comment) IMHO that call to the technical committee cannot be done if the BEP is not in the PR phase unless I'm misinterpreting the governance.

No, I do not intend to work on a Google Doc document that is out of date and should not be used (as we have discussed above). I can work as a PR against @francopestilli's lab Github repo, but this is precisely why I proposed over email to fork the specs to work in this PR, to make it open to everyone and under the control of BIDS.

[CPernet]

the PR reflects exactly the google doc -- and the steering group decided on that procedure

[CPernet]

but feel free to make a parallel PR for the technical committee to review

[oesteban]

the PR reflects exactly the google doc

What PR?

[CPernet]

Apologies, I've been too quick -- in several instances above we linked to the PR, i.e. the Atlas BEP is/was ready --> you can see it there https://github.com/PESTILLILAB/bids-specification/blob/bep038/src/atlas.md.

What I was proposing is to use your framework to write the 3 use cases/examples that the BEP deals with. I think that's the best way for everyone to appreciate the changes you want to make. Without those 3 cases fully fleshed out, it can be difficult to follow, Then we can see if we can reach a consensus, if and only if we cannot, would we need to ask an 'external' group to look into it (as per BEP guideline PR). I hope this makes sense to you. Thx.

[oesteban]

What I was proposing is to use your framework to write the 3 use cases/examples that the BEP deals with.

I don't know what my framework is in this context. Are you asking me to write a PR against PESTILLILAB/bids-specification?

What I was proposing is to use your framework to write the 3 use cases/examples that the BEP deals with. I think that's the best way for everyone to appreciate the changes you want to make.

I have done that above, but okay if a PR against the fork would be a better way of driving the conversation.

Expand to see my concerns about the "BEP guideline PR"

Then we can see if we can reach a consensus, if and only if we cannot, would we need to ask an 'external' group to look into it (as per BEP guideline PR).

The "BEP guideline PR" has so many problems that it is absolutely uninterpretable. However, as per the BIDS' bylaws, it is the BIDS SC who decides its interpretation anyway. In effect, the "BEP guideline PR" is just creating a loophole to move the responsibility for making an ad-hoc decision on a BEP to a group (we don't know the name yet) of referees instead of having the SC behind that decision.

Before proceeding that way, I have several questions (please expand if interested):

• Who is going to decide the point where "consensus" has not been achieved? As per the "BEP guideline PR", that looks like the BEP leads, but nothing is said on how the mechanism is triggered - simple majority of BEP leads? how's the voting? That part of the "BEP guideline PR" would need a lot of work to make it actionable.
• When is this decision going to happen? When does the procedure become "unlocking" a stuck BEP rather than silencing diverging opinions? Is that one week? one year? a 10% of the current lifespan of the BEP?
• Who is going to be part of that group of referees? What is their level of experience with BIDS? How involved with this BEP will they be? Because it is not the same to participate in a conversation compared to fine-tuning my proposal for a tribunal of experts.
• How the "alternatives" are to be evaluated? What are the criteria and guidelines that these referees will be given?

Linking bids-standard/bids-extensions#29 as all these questions should be in that thread.

[effigies]

I think we should go ahead and open a PR for BEP038. I suggest that we push the current PESTILLILAB/bep038 branch to upstream/bep038 and open a PR from it. That would mean that PRs against the BEP will now be discussed in the main repository and less likely to go unnoticed, as well as get rendered on ReadTheDocs, for easier review.

@oesteban I think it would be good to make a concrete PR against that branch, once it's created. I've tried to keep up with this thread, but still don't understand what alternative you would like to see.

I did say that I would also try to make a PR against the PESTILLILAB branch a few weeks ago, but have not had the time. I will try to get to that ASAP. Please do not wait on me to get a PR in if you end up having time to put together a proposal first.

[melanieganz]

Dear all,

I wanted to highlight that @effigies has turned the PESTILLI lab PR into a proper BIDS repo PR now and that we are continuing/restarting the discussions there.

We really want to get your input and see your alternative proposals for how we can share atlases. So please check it out!

[oesteban]

Thanks @effigies. You suggested in the PR to have the conversation here - but I think it actually would be better to lock this conversation and redirect everyone to the PR as the main discussion point about the BEP.

[effigies]

I suggest that future conversations be focused in individual PRs against the bep038 branch, rather than creating extensive discussion in that PR, which could grow into something sprawling like this issue. I think it would be good to have very specific proposals that can be discussed and accepted/rejected.

I'm not a huge fan of locking conversations, but I can if it seems like it will be difficult to keep new posts to this thread to a minimum.