READMem: Robust Embedding Association for a Diverse Memory in Unconstrained Video Object Segmentation
by Stéphane Vujasinović, Sebastian Bullinger, Stefan Becker, Norbert Scherer-Negenborn, Michael Arens and Rainer Stiefelhagen
TL;DR: We manage the memory of STM like sVOS methods to better deal with long video. To attain long-term performance we estimate the inter-frame diversity of the base memory and integrate the embeddings of an incoming frame into the memory if it enhances the diversity. In return, we are able to limit the number of memory slots and deal with unconstrained video sequences without hindering the performance on short sequences and alleviate the need for a sampling interval.
[arXiv] - [BMVC Proceeding Paper]/[SUPP.] - [Video] - [Poster] - [BMVC Page]
- Our poster was mentioned during the honorable mentions! 😄
The following plots illustrate performance variations among sVOS baselines with and without our READMem extension on the LV1 dataset. The first plot showcases changes when varying the sampling interval
But check out our paper and supplementary material for more qualitative and quantitative results!
The documentation is split in the following seperate markdown files:
@inproceedings{Vujasinovic_2023_BMVC,
author = {Stephane Vujasinovic and Sebastian Bullinger and Stefan Becker and Norbert Scherer-Negenborn and Michael Arens and Rainer Stiefelhagen},
title = {READMem: Robust Embedding Association for a Diverse Memory in Unconstrained Video Object Segmentation},
booktitle = {34th British Machine Vision Conference 2023, {BMVC} 2023, Aberdeen, UK, November 20-24, 2023},
publisher = {BMVA},
year = {2023},
url = {https://papers.bmvc2023.org/0603.pdf}
}