Unable to load frame

[Idavr]

Hi there!

When trying to train the model for the very first time after labeling a total of 200 frames, I get the error of being unable to lead specific frames:

KeyError: "Unable to load frame 44040 from MediaVideo(filename='C:/Users/idavalik/Videos/Exp15-1.mp4', grayscale=True, bgr=True, dataset='', input_format='')."
Run Path: C:/Users/idavalik\models\200622_110010.centered_instance.200

I don't know enough of how SLEAP runs, but I am assuming that the mismatched slashes in the Run Path may contribute to this error? However, I did not find anywhere in the terminal or under the installation guide in how to potentially rectify this (if that is even possible?).

Any help you can give is greatly appreciated, thank you!

[talmo]

Hi @Idavr,

The slashes will get handled automatically so it shouldn't matter unless the video file was moved. This typically happens when the video file is not reliably seekable.

You can try to do this for a quick test:

1. Close SLEAP and go to C:\Users\idavalik\Videos and rename Exp15-1.mp4 to Exp15-1.old.mp4
2. If you don't have ffmpeg, download it from here and extract the bin\ffmpeg.exe file to your C:\Users\idavalik\Videos folder.
3. Open up a terminal and run this to convert the video to a reliably seekable format:

cd C:\Users\idavalik\Videos
ffmpeg -i Exp15-1.old.mp4 -c:v libx264 -preset superfast -g 1 -framerate 25 -crf 15 -pix_fmt yuv420p Exp15-1.mp4

Then try running the training again. If you don't get the error about reading the same video again we'll know it's the format, otherwise let us know.

[Idavr]

Hi @talmo ,

Running through your steps I get the same error but for one of the other videos (10 in total were labeled). Does this mean I have to convert all the videos using ffmpeg like you described for the first one?

[talmo]

Great! Yes, it does seem like it'll be the case that you'll have to convert the rest. This is a requirement since we need to be able to access single frames in random order rather than one-by-one sequentially as some formats expect.

Depending on what software you use to acquire the videos, you might be able to set it up so that you won't have to do this in the future.

After you convert them, I'd recommend opening the SLEAP project again and double checking some of the labeled frames to make sure that the annotations still overlap with the animals correctly. It may happen that the wrong frame was displayed when you were labeling (usually within +-100 frames of the exact frame).

[Idavr]

Alright, that is an easy fix!

Thank you so much for your help, I'll give this a try and hopefully I'll have a working model running soon :)

[talmo]

Great! Feel free to open up a new issue if you're having issues with model training. Cheers :)

[Idavr]

Hi @talmo ,

Sorry to revive this thread with a different question but it is directly related to the solution for the last one. You recommended re-opening the SLEAP-project again to double check some of the labels, which I did and indeed the labels were mixed between all frames. Getting to work over the past couple of days I've gone through all the frames and re-labeled them, but have discovered that some of the labels don't "stick" to some of the frames, if I can explain it that way.

For example: I may have gone through and re-labeled (by moving the skeletons already there into their correct positions) 10-15 frames, but when going back to check on them (without closing or doing anything else with the project and the GUI) 1/3 to 1/4 of them have moved! I then re-label them, but the same thing keeps happening over and over and over again. Any ideas?

[ntabris]

Hi, @Idavr. Suppose you adjust the annotations on frame 123 so that they match the image, you go to some other frames, then you come back to frame 123 and the annotations no longer align with the image. Can you tell whether this is because the annotations are in a different position from where you put them or because the image is different than when you were on frame 123 before? If it's the image, then you're probably still not using a seekable video.

[Idavr]

Hi @ntabris . Thank you for getting back to me so quickly. It does indeed seem like the image has jumped one or two places back or forth, actually... My immediate thought is to make a new project with these new converted videos and start from scratch, or could re-loading the videos into the existing project (with the existing annotations) work as well?

[ntabris]

You might as well try reloading the videos in the existing project and if that doesn't work, make a new project with the converted videos.

[Idavr]

Thank you very much :)