Adding Segment Anything to the model zoo!

[brimoor]

Polishes up @Rusteam's work from #3019.

Segment Anything is now officially available in the FiftyOne Model Zoo! 📈

Box prompt example

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset(
    "quickstart", max_samples=25, shuffle=True, seed=51
)

model = foz.load_zoo_model("segment-anything-vitb-torch")

# Prompt with boxes
dataset.apply_model(
    model,
    label_field="segmentations",
    prompt_field="ground_truth",
)

session = fo.launch_app(dataset)

Keypoint prompt example

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset(
    "coco-2017",
    split="validation",
    label_types="detections",
    classes=["person"],
    max_samples=25,
    only_matching=True,
)

# Generate some keypoints
model = foz.load_zoo_model("keypoint-rcnn-resnet50-fpn-coco-torch")
dataset.default_skeleton = model.skeleton
dataset.apply_model(model, label_field="gt")

model = foz.load_zoo_model("segment-anything-vitb-torch")

# Prompt with keypoints
dataset.apply_model(
    model,
    label_field="segmentations",
    prompt_field="gt_keypoints",
)

session = fo.launch_app(dataset)

Automatic segmentation example

import fiftyone as fo
import fiftyone.zoo as foz

dataset = foz.load_zoo_dataset(
    "quickstart", max_samples=5, shuffle=True, seed=51
)

model = foz.load_zoo_model("segment-anything-vitb-torch")

# Automatic segmentation
dataset.apply_model(model, label_field="auto")

session = fo.launch_app(dataset)

[jacobmarks]

LGTM and it works on my machine 👍 . The one thing worth noting is that it doesn't install segment-anything for you. You need to install the package for it to work. Might consider adding the instructions:

pip install git+https://github.com/facebookresearch/segment-anything.git

[brimoor]

LGTM and it works on my machine 👍 . The one thing worth noting is that it doesn't install segment-anything for you. You need to install the package for it to work. Might consider adding the instructions:

pip install git+https://github.com/facebookresearch/segment-anything.git

FYI- without segment-anything installed, this command:

import fiftyone.zoo as foz

model = foz.load_zoo_model("segment-anything-vitb-torch")

should show a helpful error message:

PackageError: The requested operation requires that 'segment-anything' is installed on your machine.

If you think this error is inaccurate, you can set `fiftyone.config.requirement_error_level` to 1 (warning) or 2 (ignore).
See https://docs.voxel51.com/user_guide/config.html for details.

If you want the dependency to be automatically installed, one can use:

model = foz.load_zoo_model("segment-anything-vitb-torch", install_requirements=True)