frame selection and frame aggregation

[Arbor334]

Great project, I looked at your model.py and train.py files
Can you pinpoint the core code for frame selection and frame aggregation in the file?:happy:

[angelaaye]

Hi @Arbor334, frame selection is done here, which calls modeling/sampler.py to sample the frames.

Frame aggregation is done here in the similarity matrix computation step.

[Arbor334]

thanks!

[Arbor334]

gumbel_softmax_top_k(logits, top_k, tau, hard=False, reduce_sum=True, maskvector=True, logit_tau=1): Is this function from modeling/gumbel_softmax.py used to select top-k frames? and actions = gumbel_softmax_top_k(prob, int(self.top_k), tau, True, reduce_sum=False) What does the return value actions mean？ select_frames or not ? 😃