Increase Video Processing from 1 Frame to Multi Frames in Parallel

[hollowaykeanho]

Description

Please provide a short description of what feature you're looking forward to
enhance below. Please include the story behind your idea as well to give a
better visualization of your idea.

Basically to speed things up.

Expected Behavior

Please specify the expected behavior of your requested enhancement. Some great
and helpful pointers are your expected interface (e.g. command patterns,
simple sketches of the user interface, etc).

Upscaler can work a lot faster via work.

Current Behavior

Please specify the current behavior (e.g. workaround, blockage, etc).

Upscaler only work on 1 frame at a time.

Attachment

Please drag and drop the necessary data files (e.g. screenshot, logs, etc)
below.

[hollowaykeanho]

Help needed as I do not have a better graphics card. Limited to only NVIDIA GeForce MX150 from a laptop to test its working capability on consumer products.

[hollowaykeanho]

Not possible with POSIX and Powershell. Neither of them provides some kind of sync mechanism like mutex lock or channels. Closing this issue as won't fix.

[hollowaykeanho]

Re-open as now we have a possibility to process frame upscaling concurrently using scoring system.

[hollowaykeanho]

@Joly0, @JeanShuralyov , can I acquire --parallel argument name for experimenting this feature ya?

like --parallel 2 for 2 images simultenously.

[Joly0]

Sounds good for me, i´d have to adjust the powershell side of this when you are done

[hollowaykeanho]

TQ & noted. I'll begin the enhancement work after clearing some works at hand.

[JeanShuralyov]

@hollowaykeanho , fine by me. Will you be using my fork repo for this enhancement?

[hollowaykeanho]

yes. Please keep it.

[hollowaykeanho]

@Joly0 , I might need help with your graphics card. =x

Appearently, I completed my prototyping codes but same like my previous attempt, I blew my vRAM when parallel is set to 2. (It demands 2400MB I believe).

I will complete the commit and push to Jean's forked repo for easier referencing.

[Joly0]

Sure, how can i help?

[hollowaykeanho]

Okay. I just pushed in parallel processing capability for init/unix.sh side of things. Diff: aa4fabb

You can try it out with --parallel 2 (I can't get through due to the hardware limitation). Once translated to the PowerShell side, this issue can be marked completed. =)

---

Heads up: the unix.sh hits the 1000 lines warning limit already, anything beyond will need a proper progrmaming language which also means: Issue #34.

Growing beyond 1000 lines for shell scripts are nightmare to maintain in near future.

[JeanShuralyov]

@hollowaykeanho , there is no way my Iris can operate --parallel 2 if your Nvidia failed. Passed. Otherwise, great job.

[hollowaykeanho]

noted. In that case I won't increase its parallelism for benchmark testing then since we do not have proper hardware to test it iteratively.

[hollowaykeanho]

@Joly0 , is it possible to translate the current outcome ya? It will be the last one since I'll drilling into Tencent NCNN framework for the next step.

[Joly0]

Hey, yes, i am on it getting this done in powershell, but i dont have that much time currently, so i cant tell, when i am finished. I will try my best to get this done asap.

[hollowaykeanho]

Haha.. no problem. It's open-source so no deadline. Take your time.

In that case I will begin the work on the source codes now. Let me know when you're ready for release.

[hollowaykeanho]

Just in case: please note that you're not required to create directory for each frames that I used.

I only did that because mkdir can behave safely in Linux side for concurrency application. You might need to find out is PowerShell has a similar command that has a safe concurrent mechanism.

[Joly0]

Btw, in powershell this is a hell of a ride, but i am getting quite close i guess, but i have to set powershell 7 as a requirement for this to work. Powershell 5 has a few techniques to make parallel execution working (jobs, runspaces, workflows), but none of them are easy to work with.
I could try to get it working with jobs or workflows, but its such a pain to work with functions and variables outside of a job or workflow scope.
Powershell 7 has the "foreach-object -parallel " function, which works atleast better. Still having issues with variables and functions, but its atleast not too difficult to work with.

So if its ok, i will get this working with powershell 7, otherwise if you want, i´ll adjust it to powershell 5

But just so you see it, its working

(running the benchmark here with --parallel 2)

I´ll let this run through and will try with some other files aswell and i have to get the control file working again, because it is borked now XD

Other than that, it looks like it is working

[hollowaykeanho]

Powershell 5 has a few techniques to make parallel execution working (jobs, runspaces, workflows), but none of them are easy to work with.

Hahaha.. completely agree with you. I'm currently working on a PowerShell + POSIX compliant Shell CI infra at https://github.com/ChewKeanHo/AutomataCI/tree/experimental. Currently working on the Docker image packaging job development.

Once I completed the AutomataCI to a stable release, I will port it here and there will be various released packages (for sure having .deb, .rpm, .flatpak, .tar.xz, and .zip.

You can make use of my newly developed shell-based libraries here (e.g. you might be interested in io): https://github.com/ChewKeanHo/AutomataCI/tree/experimental/automataCI/services

(running the benchmark here with --parallel 2)

Good luck! I couldn't run here.

I´ll let this run through and will try with some other files aswell and i have to get the control file working again, because it is borked now XD

Haha. Yeap. Some changes done because the execution is different now.

[Joly0]

Btw, just to give a quick update. I havent had any time in the past few months to look into this at all, which i am quite sorry for. But i think i´ll might have some time now to get this a all fixed, finish it up and push it out, so it can be merged :D

[corygalyna]

@Joly0 , I had cleaned up the project dashboard into a road-map kanban style already. This should further the repository development with a more directional approach.

Yawn. Enjoy. Oh by the way, @hollowaykeanho , did something mad in this project: https://github.com/ChewKeanHo/AutomataCI/tree/main/automataCI/services

I believe controlled multi-threading is available in there io/sync.*

[Joly0]

Joly0@9ce20e9

[hollowaykeanho]

I assume it was tested right? In that case, I will merge it in.

[Joly0]

Apparentely the retry function is not yet working correctly, but otherwise yes, this works. I can do some testing again later and can check how much of a difference this makes going from parallel 1 to 2 or 4.

Also while we are at it, it might be useful to have a shorter second benchmark/test video for people to check the performance of parallel easier. It can take quite a while for even the 400 frames benchmark video to finish. I think just a second of video at 25fps should be enough

[hollowaykeanho]

Also while we are at it, it might be useful to have a shorter second benchmark/test video for people to check the performance of parallel easier. It can take quite a while for even the 400 frames benchmark video to finish. I think just a second of video at 25fps should be enough

All right. I will look into it.

[hollowaykeanho]

I merged your patch into the production already: 3ad9974

The test video will need to generate a new one. I will generate a test result so that it's clearer.

UPDATE: your branch was forced-push for alignment. Please make sure you force update locally before any new development.