Problems with real-time segmentation accuracy

[jkff00]

Hi authors,

Your work is so great! I'm trying to deploy it to do real-world task. There's a problem that's bothering me right now.

For instances, looking at the confidence maps and background masks for single frames, there are a considerable number of objects in the indoor environment that are segmented into the background, resulting in large voids. This is usually due to the exceptionally high category regression score of 0.9++ for the background category.

1.This could potentially lead to missing semantics in some areas of the global map, is there any way around this. (e.g. Whether COCO_PANOPTIC_CLASSES needs to be replaced according to a specific scenario)

2.SEEM-based mask regression is based on predefined categories, whereas the SAM+CLIP combination seems to be category-independent, and whether this makes a difference when deployed on open-set data (e.g. unknown category).