Inquiry About the Use of Shift Operation on is_eos_codes in Code

[qixuema]

Hello, Phil

I'm currently exploring your code and have a question regarding the implementation of is_eos_codes. Specifically, I'm curious about the rationale behind using the shift operation on is_eos_codes here.

shifted_is_eos_tokens = F.pad(is_eos_codes, (1, -1))
mask = shifted_is_eos_tokens.float().cumsum(dim = -1) >= 1
codes = codes.masked_fill(mask, self.pad_id)

From my understanding, the purpose of the shift operation is to feed the eos_token_id into the decode_from_codes_to_faces for processing. However, I am somewhat confused about the potential problems this might introduce. Given that the eos_token_id is 16384, and the length of the codebooks is also 16384, this might lead to an indexing issue during the subsequent execution of autoencoder.decode_from_codes_to_faces.

face_coords, face_mask = self.autoencoder.decode_from_codes_to_faces(codes)

Specifically, when performing the gather operation here, it might be impossible to find an element at index 16384.

Could you please clarify if my understanding is correct, or if there's something I'm missing here? Any insights or explanations on this would be greatly appreciated.

Thank you for your time and assistance.

Best regards,
Xueqi

[lucidrains]

yes you are right, that was my mistake

thank you!