Is it possible to just get the dll for using in Unity?

[milos12345]

I don't see a way to build it on Windows and VS

[bmegli]

Building on Windows

There should be nothing stopping you.

The dependencies are standard library and ffmpeg.

If you don't know how to use cmake or don't want to use it:

• make sure ffmpeg is available for compiler and linker
• add hvd.c, hvd.h to VS
• link with ffmpeg
• build a dll as you would usually

In theory, it has been some time since I used Windows for development.

Using directly from Unity:

In theory it is possible but:

• the library makes heavy use of pointers (you generally don't want to copy any data when working with 60 video frames per second, you always aim for zero copy or as close as possible)
• using pointers from Unity is possible but clumsy
• you generally would not want to make any managed copies of image data (at 1080p, single decoded frame is around 8 MB, at 60 fps this gives around 0,5 GB/s which doesn't play nice with Garbage Collector)

[milos12345]

Thank you, I'll have to figure out how to do the first part. Your unity pointers make sense, I did not think of that so maybe it would not work as I expected

[bmegli]

@milos12345

For the second part (using directly from Unity) - I am trying to say that it will not be easy, and may not be the way to go. Still it is possible.

Using native code from managed code (Unity), with complex pointer logic, needs more experience and care than is required for writing native only or managed only code.

There are also other things to consider:

• HVD will block for the duration of decoding
• you should not wait for the duration of blocking code in Unity Update
• so decoding should probably happen in separate thread
• this thread will probably need synchronization with main Unity thread (mutex, semaphor, etc).

You may see example (but Linux only) of similar logic in unity-network-hardware-video-decoder

• all network and decoding logic moved to native side (no managed memory)
• the heavy stuff (network and decoding) runs in unmanaged thread
• there is synchronization (mutex) when updating the texture data
• a double buffer like mechanism was used (for decoded frames)
• a smart pointer like reference counting for frame data was used (ffmpeg frame referencing, for avoiding copies of decoded video data)
• the only thing left in Unity is texture update (with IntPtr, left in Unity for portability)
• the network protocol used is custom (this would be bad choice, the reason is that those are also my experiments with the protocol)

There was also recently somebody trying to port nhvd code to Windows (for unhvd)

[bmegli]

Hi @milos12345,

Have a look at ViveMediaDecoder by HTC

It is a Windows Unity video decoder.
From looking at code it is software decoder with hardware accelerated rendering (shaders).
Also, it seems to be working with RTSP over TCP/IP if you need to stream video.
There are a few examples in their repository.

Kind regards

[milos12345]

Hi @bmegli,
I have tried following your steps but it still exceeds my knowledge so I was unsuccessful. I have tried ViveMediaDecoder you suggested and seems to be fitting my needs.
Thank you!