Add padding before or after log_mel_spectrogram?

[guillaumekln]

Hi,

In the transcribe function, the padding is added to the Mel spectrogram:

whisper/whisper/transcribe.py

Line 177 in eff383b

segment = pad_or_trim(mel[:, seek:], N_FRAMES).to(model.device).to(dtype)

However, in the README the padding is added to the input audio:

whisper/README.md

Lines 119 to 122 in eff383b

	audio = whisper.pad_or_trim(audio)
	# make log-Mel spectrogram and move to the same device as the model
	mel = whisper.log_mel_spectrogram(audio).to(model.device)

In that case the padding positions in the Mel spectrogram are non zero.

Which approach was used in the training?

Thanks!

[guillaumekln]

As mentioned in #838 (comment), the padding should be added to the input audio.