Segment Anything from META: Network implementation. Has someone started already? #6357
Comments
vgonzalezd
changed the title
vgonzalezd
changed the title
|
Hi @yashika-git, yes, welcome to your contribution. |
|
Thanks, I've started to look how it can be implemented. |
|
Could someone please provide a little more details about the expected outcome of the implementation? |
|
Hi @yashika-git, yes first I think we should enable users to load the existing pre-trained weights, also I think they could infer and fine-tune the model on their custom datasets. |
|
Hi @KumoLiu, I have a very naive question: What exactly needs to be done to integrate SAM with MONAI? Does it involve modifying SAM's codebase to utilize MONAI's existing code, such as MONAI's networks and layers? |
|
Hi @yashika-git, For existing parts that can be replaced by the MONAI, you should use the components of MONAI as much as possible. |
Fixes #7356 ### Description Add relative positinoal embedding in attention block as described in https://arxiv.org/pdf/2112.01526.pdf Largely inspired by https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/image_encoder.py Can be useful for #6357 ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: vgrau98 <[email protected]> Signed-off-by: vgrau98 <[email protected]> Co-authored-by: Eric Kerfoot <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: YunLiu <[email protected]>
Fixes Project-MONAI#7356 ### Description Add relative positinoal embedding in attention block as described in https://arxiv.org/pdf/2112.01526.pdf Largely inspired by https://github.com/facebookresearch/segment-anything/blob/main/segment_anything/modeling/image_encoder.py Can be useful for Project-MONAI#6357 ### Types of changes <!--- Put an `x` in all the boxes that apply, and remove the not applicable items --> - [x] Non-breaking change (fix or new feature that would not break existing functionality). - [ ] Breaking change (fix or new feature that would cause existing functionality to change). - [ ] New tests added to cover the changes. - [ ] Integration tests passed locally by running `./runtests.sh -f -u --net --coverage`. - [ ] Quick tests passed locally by running `./runtests.sh --quick --unittests --disttests`. - [x] In-line docstrings updated. - [x] Documentation updated, tested `make html` command in the `docs/` folder. --------- Signed-off-by: vgrau98 <[email protected]> Signed-off-by: vgrau98 <[email protected]> Co-authored-by: Eric Kerfoot <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: YunLiu <[email protected]> Signed-off-by: Mark Graham <[email protected]>
Is your feature request related to a problem? Please describe.
Is it someone implementing in MONAI the segment anything model in MONAI?
https://github.com/facebookresearch/segment-anything
My concern is the weight matching. We can help with it if someone starts already.
Describe the solution you'd like
This will make more students download and use MONAI
For sure it will be an useful network for MONAILABEL
Describe alternatives you've considered
N/A
Additional context
Add any other context or screenshots about the feature request here.
The text was updated successfully, but these errors were encountered: