Contiguous memory warning when running on GPU

[tshmak]

I tried running the example on GPU, and it gave me the following warning:

/mnt/nas2/tshmak/WORK/Projects/TTS/FanoFastSpeech2/conda_gpu2_fs2/lib/python3.10/site-packages/torch/nn/modules/module.py:1190: UserWarning: RNN module weights are not part of single contiguous chunk of memory. This means they need to be compacted at every call, possibly greatly increasing memory usage. To compact weights again call flatten_parameters(). (Triggered internally at ../aten/src/ATen/native/cudnn/RNN.cpp:968.)
  return forward_call(*input, **kwargs)
/mnt/nas2/tshmak/WORK/Projects/TTS/FanoFastSpeech2/conda_gpu2_fs2/lib/python3.10/site-packages/torch/nn/modules/module.py:1190: UserWarning: operator() profile_node %669 : int[] = prim::profile_ivalue(%667)
 does not have profile information (Triggered internally at ../torch/csrc/jit/codegen/cuda/graph_fuser.cpp:105.)

Codes I ran:

SAMPLING_RATE = 16000

import torch
torch.set_num_threads(1)

from IPython.display import Audio
from pprint import pprint
# download example
torch.hub.download_url_to_file('https://models.silero.ai/vad_models/en.wav', 'en_example.wav')

USE_ONNX = False # change this to True if you want to test onnx model
if USE_ONNX:
    !pip install -q onnxruntime
  
model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad',
                              model='silero_vad',
                              force_reload=True,
                              onnx=USE_ONNX)

(get_speech_timestamps,
 save_audio,
 read_audio,
 VADIterator,
 collect_chunks) = utils

model = model.cuda()

wav = read_audio('en_example.wav', sampling_rate=SAMPLING_RATE).cuda()
# get speech timestamps from full audio file
speech_timestamps = get_speech_timestamps(wav, model, sampling_rate=SAMPLING_RATE)
pprint(speech_timestamps)

I'm using Python 3.10 and pytorch 1.13

Many thanks for your help!

[snakers4]

Hi,

The model is not designed to run on GPU.
Does the warning only appear or does the model run abnormally in your environment?

[tshmak]

It's just a warning. It runs ok on GPU.

[tshmak]

Actually, I found that the warning is triggered even when I did not pass .cuda(). It seems to automatically detect that I have GPU and triggers the warning. To avoid the warning, I have to run with CUDA_VISIBLE_DEVICES=-1. I can't avoid the warning by passing .cpu() to the model either.

[snakers4]

This is weird. I saw this warning when I was training large RNN models on several GPUs.

[cassiotbatista]

Surprinsingly, it only works on GPU if the window size is 512. If one increases it to 1024 or 1536 samples on 16 kHz data, TorchScript gives a very verbose error, as provided in the log attached.

speech_timestamps = get_speech_timestamps(
    wav, model, sampling_rate=16000, window_size_samples=1024
)

/tmp/err1024.log:RuntimeError: Given groups=1, weight of size [1, 64, 1], expected input[64, 2, 1] to have 64 channels, but got 2 channels instead
/tmp/err1536.log:RuntimeError: Given groups=1, weight of size [1, 64, 1], expected input[64, 3, 1] to have 64 channels, but got 3 channels instead

err1024.log

Finally, a question: if the model is designed not to work on GPU, any thoughts on why it works on this specific case? Unpredictable behaviour perhaps?