Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Mirror.exe file transfer performance on remote network share #1246

Open
5 tasks done
xsidexpt opened this issue Sep 21, 2024 · 4 comments
Open
5 tasks done

Mirror.exe file transfer performance on remote network share #1246

xsidexpt opened this issue Sep 21, 2024 · 4 comments

Comments

@xsidexpt
Copy link

xsidexpt commented Sep 21, 2024

Feature request can skip this form. Bug report must complete it. Check List must be 100% match or it will be automatically closed without further discussion. Please remove this line.

Environment

  • Windows version: 11
  • Processor architecture: x64
  • Dokany version: 2.2.0
  • Library type (Dokany/FUSE): Dokany

Check List

  • I checked my issue doesn't exist yet
  • My issue is valid with mirror default sample and not specific to my user-mode driver implementation
  • I can always reproduce the issue with the provided description below.
  • I have updated Dokany to the latest version and have reboot my computer after.
  • I tested one of the last snapshot from appveyor CI

Description

I have this use case where I am using mirror.exe to mount some remote network shares as local drives on a Windows 11 VM.
The shares are connected to via WireGuard VPN and the performance when copying a file via the explorer (without Dokany) is the upload limit of the remote connection (around 40mb/s) but when doing the same operation through Dokany's "local" drive the performance maxes out at around 12mb/s.
However, if I start multiple concurrent copies, I can achieve similar speeds to what I could with a single file transfer without Dokany.
I was wondering if there is anything I can possibly do to improve the performance of this particular scenario.
The performance penalty of using Dokany with a local network share is pretty negligible on the same machine, but this remote drive ends up chocking it.

I appreciate the help!

@Liryna
Copy link
Member

Liryna commented Sep 28, 2024

Hi @xsidexpt ,

Have you tried with the network option /n ? Sometime Windows might increase the read size which will increase the speed copy for remote storage.

This is a case where caching with read ahead would be very helpful. #1161 #1047

@xsidexpt
Copy link
Author

xsidexpt commented Sep 30, 2024

Just did some testing and using the /n option results in no changes to performance, but you are probably on the right track with the read size theory, I just wished I knew how to better configure this to claim back the 40% of speed I am losing. There is also a very significant difference to the folder navigation speed when using the mirror vs the native folder, but if I could get the throughput higher I would be happy

@Liryna
Copy link
Member

Liryna commented Sep 30, 2024

There could be many factors that can slow down mirror. It is basic (no caching, etc.) since it is a sample but I wouldn't expect it to be 40% slower on a clean environment (without antivirus, etc.). I am surprised that folder navigation is noticeably slow.
The mirror really just open / read the file using standard API.

There might be something slowing down Dokany since it can achieve very high read / write speed. Antivirus are usually curious by apps accessing remote content that could be the reason but that's only a guess.

You might be able to see with Procmon if there is any filter drivers in the callstack that are accessing the content between dokan driver and explorer (or else).

@xsidexpt
Copy link
Author

xsidexpt commented Oct 1, 2024

Funny thing is when I access files that are on a different computer on the local network I can achieve the same speed with and without dokan (the 1Gbit of the Ethernet port) but not for these remote folders.
It caps for some reason at 12.8mb/s with dokan and can get very close to 30mb/s without.
I have already ruled out windows defender getting in the way, and I can't understand enough of what shows up in Procmon to be able to conclude anything significant.
My current train of thought is that has to be a delay of sorts happening because otherwise the folder navigation wouldn't be this much slower. We are talking multiple seconds to navigate around the folders, which is definitely not there when I access the same path with dokan in the middle.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants