PLEASE HELP BAO

[xstrauss]

When I run it,

Interpolation

%shell mkdir -p '/content/gdrive/My Drive/DAIN/frames-out'
%cd /content/DAIN

!python colab_interpolate.py --netName DAIN --time_step 49.94 --start_frame 1 --end_frame 450 --frame_input_dir '/content/gdrive/My Drive/DAIN/frames-in' --frame_output_dir '/content/gdrive/My Drive/DAIN/frames-out'

I'm encountering with this error;

/content/DAIN
revise the unique id to a random numer 76481
Namespace(SAVED_MODEL=None, alpha=[0.0, 1.0], arg='./model_weights/76481-Wed-May-20-16:48/args.txt', batch_size=1, channels=3, ctx_lr_coe=1.0, datasetName='Vimeo_90K_interp', datasetPath='', dataset_split=97, debug=False, depth_lr_coe=0.001, dtype=<class 'torch.cuda.FloatTensor'>, end_frame=450, epsilon=1e-06, factor=0.2, filter_lr_coe=1.0, filter_size=4, flow_lr_coe=0.01, force=False, frame_input_dir='/content/gdrive/My Drive/DAIN/frames-in', frame_output_dir='/content/gdrive/My Drive/DAIN/frames-out', log='./model_weights/76481-Wed-May-20-16:48/log.txt', lr=0.002, netName='DAIN', no_date=False, numEpoch=100, occ_lr_coe=1.0, patience=5, rectify_lr=0.001, save_path='./model_weights/76481-Wed-May-20-16:48', save_which=1, seed=1, start_frame=1, time_step=49.94, uid=None, use_cuda=True, use_cudnn=1, weight_decay=0, workers=8)
cudnn is used
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/upsampling.py:129: UserWarning: nn.UpsamplingNearest2d is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.{} is deprecated. Use nn.functional.interpolate instead.".format(self.name))
/usr/local/lib/python3.6/dist-packages/torch/nn/modules/upsampling.py:129: UserWarning: nn.Upsample is deprecated. Use nn.functional.interpolate instead.
warnings.warn("nn.{} is deprecated. Use nn.functional.interpolate instead.".format(self.name))
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:2423: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
Traceback (most recent call last):
File "colab_interpolate.py", line 112, in
y_s, offset, filter = model(torch.stack((X0, X1),dim = 0))
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 489, in call
result = self.forward(*input, **kwargs)
File "/content/DAIN/networks/DAIN.py", line 165, in forward
cur_offset_output = [cur_offset_outputs[0][0], cur_offset_outputs[1][0]]
IndexError: list index out of range

[AlphaGit]

@xstrauss Could you please share your input video (or if private, something about it, like frame size) and the colab configuration you're using to run it?

This might help us get to the specific case where this error happens.

[xstrauss]

I got this error with multiple videos. In fact, I always get this error. But tonight I will try this with an another video and get you notified.

BTW, you have social media, instagram for example? Maybe you can help me at there.

[AlphaGit]

Let’s keep the conversation over here and possibly it can help others if they have the same problems.

[kirbyhi5]

i have the same problem

/content/DAIN
revise the unique id to a random numer 7304
Namespace(SAVED_MODEL=None, alpha=[0.0, 1.0], arg='./model_weights/7304-Mon-Jun-01-21:54/args.txt', batch_size=1, channels=3, ctx_lr_coe=1.0, datasetName='Vimeo_90K_interp', datasetPath='', dataset_split=97, debug=False, depth_lr_coe=0.001, dtype=<class 'torch.cuda.FloatTensor'>, end_frame=6481, epsilon=1e-06, factor=0.2, filter_lr_coe=1.0, filter_size=4, flow_lr_coe=0.01, force=False, frame_input_dir='/content/DAIN/input_frames', frame_output_dir='/content/DAIN/output_frames', log='./model_weights/7304-Mon-Jun-01-21:54/log.txt', lr=0.002, netName='DAIN_slowmotion', no_date=False, numEpoch=100, occ_lr_coe=1.0, patience=5, rectify_lr=0.001, save_path='./model_weights/7304-Mon-Jun-01-21:54', save_which=1, seed=1, start_frame=1, time_step=0.5000771862179623, uid=None, use_cuda=True, use_cudnn=1, weight_decay=0, workers=8)
cudnn is used
Interpolate 0 frames
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:2506: UserWarning: Default upsampling behavior when mode=bilinear is changed to align_corners=False since 0.4.0. Please specify align_corners=True if the old behavior is desired. See the documentation of nn.Upsample for details.
"See the documentation of nn.Upsample for details.".format(mode))
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/usr/local/lib/python3.6/dist-packages/torch/nn/functional.py:2705: UserWarning: Default grid_sample and affine_grid behavior has changed to align_corners=False since 1.3.0. Please specify align_corners=True if the old behavior is desired. See the documentation of grid_sample for details.
warnings.warn("Default grid_sample and affine_grid behavior has changed "
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
/pytorch/torch/csrc/autograd/python_function.cpp:622: UserWarning: Legacy autograd function with non-static forward method is deprecated and will be removed in 1.3. Please use new-style autograd function with static forward method. (Example: https://pytorch.org/docs/stable/autograd.html#torch.autograd.Function)
Traceback (most recent call last):
File "colab_interpolate.py", line 112, in
y_s, offset, filter = model(torch.stack((X0, X1),dim = 0))
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/content/DAIN/networks/DAIN_slowmotion.py", line 202, in forward
return cur_outputs,cur_offset_output,cur_filter_output
UnboundLocalError: local variable 'cur_offset_output' referenced before assignment

this is 1920x1080 30fps mp4 file.
GPU: Tesla P100-PCIE-16GB, 418.67, 16280 MiB

[AlphaGit]

@kirbyhi5 You have a different error but it might be related.

How many frames did your video have?

[kirbyhi5]

@AlphaGit 6480 frames

[AlphaGit]

Hey @kirbyhi5 -- I believe you might be using an old version of the Colab file, because we had taken care of the legacy autograd warnings that we're seeing in your logs some time ago.

Why don't you give it a try with the newest version? I have just sent a change that should soon be available in the master branch. In the meantime, you can get it here: (link)

Try that out and let me know how it goes!

[kirbyhi5]

Hi @AlphaGit , I used .ipynb from README baowenbo/DAIN before. Now I get this error:

/content/DAIN
revise the unique id to a random numer 36812
Namespace(SAVED_MODEL=None, alpha=[0.0, 1.0], arg='./model_weights/36812-Fri-Jun-12-06-37/args.txt', batch_size=1, channels=3, ctx_lr_coe=1.0, datasetName='Vimeo_90K_interp', datasetPath='', dataset_split=97, debug=False, depth_lr_coe=0.001, dtype=<class 'torch.cuda.FloatTensor'>, end_frame=6481, epsilon=1e-06, factor=0.2, filter_lr_coe=1.0, filter_size=4, flow_lr_coe=0.01, force=False, frame_input_dir='/content/DAIN/input_frames', frame_output_dir='/content/DAIN/output_frames', log='./model_weights/36812-Fri-Jun-12-06-37/log.txt', lr=0.002, netName='DAIN_slowmotion', no_date=False, numEpoch=100, occ_lr_coe=1.0, patience=5, rectify_lr=0.001, save_path='./model_weights/36812-Fri-Jun-12-06-37', save_which=1, seed=1, start_frame=1, time_step=0.5000771862179623, uid=None, use_cuda=True, use_cudnn=1, weight_decay=0, workers=8)
cudnn is used
Interpolate 0 frames
Traceback (most recent call last):
File "colab_interpolate.py", line 112, in
y_s, offset, filter = model(torch.stack((X0, X1),dim = 0))
File "/usr/local/lib/python3.6/dist-packages/torch/nn/modules/module.py", line 532, in call
result = self.forward(*input, **kwargs)
File "/content/DAIN/networks/DAIN_slowmotion.py", line 202, in forward
return cur_outputs,cur_offset_output,cur_filter_output
UnboundLocalError: local variable 'cur_offset_output' referenced before assignment

[kirbyhi5]

I also reduced the video resolution to 720p