Filediscovery: Path not found when space in path, and path ends with \ (Windows 10)

[GrumpyBear57]

Environment

• OS: Windows 10 1904
• AVD Version: 8188 (From AniDB wiki download link)
• Tested in Command Prompt and Windows Terminal
• Tests were performed on a SMB share, but I noted earlier that I had issues with a space in names when scanning from a local NTFS drive

Writeup

So this is rather odd, and I'm not entirely sure if the title makes sense, so I've included my test cases at the bottom of the issue. Please note that I've tried both ' and " to see if there's any difference on the failing paths^†. I've also included paths with no spaces in them, using various combinations of ', ", and trailing \, to ensure that it is actually the space causing the error.

I'd also like to note that when printing the Filediscovery error when a space is in the directory, it prints the trailing ' or ", whereas when there is no space, it removes the character and just prints the path. I don't know if this is related, or useful at all, but I've included additions test paths for a true non-existent directory (X:\Anime\NotThere\ in case it wasn't obvious -- all other folders used exist on the file system) which shows what the 'normal' output for the error shows. You can also note that the error for the directory that actually doesn't exist properly prints the trailing \, while it's missing from a directory with a space.

† It seems like using Command Prompt, it won't accept ' at all -- which could be a separate issue. I'm not sure if anything in cmd will accept paths surrounded by ', or if it just accepts ". I've tried both because I'm using Terminal, which accepts paths using either. Note the test using ' instead of " returns a " trailing the path in the error, which leads me to believe that Terminal is just converting the ' into a " before the executable receives it, but I'm not sure about that.

Sorry that this issue is kinda a massive info dump. I tried to include anything I might find useful as a developer, but if there's anything else you need/want to know or have me test, just let me know. I'll also try to take a look at the code later, to see if I can try to fix it myself.

Tests

Base Command (should it even matter): .\AVDump3CL.exe --auth={snip} --Ed2kLogPath=ed2k.txt -R {path}
Paths tested:

• 'X:\Anime\Steins Gate\' FAILS
• "X:\Anime\Steins Gate\" FAILS
• 'X:\Anime\Steins Gate' WORKS
• "X:\Anime\Steins Gate" WORKS
• 'X:\Anime\Kanon\' WORKS
• "X:\Anime\Kanon\" WORKS
• X:\Anime\Kanon\ WORKS
• 'X:\Anime\Kanon' WORKS
• "X:\Anime\Kanon" WORKS
• X:\Anime\Kanon WORKS

Error Output:

• 'X:\Anime\Steins Gate\' -- Filediscovery: Path not found: X:\Anime\Steins Gate"
• "X:\Anime\Steins Gate\" -- Filediscovery: Path not found: X:\Anime\Steins Gate"
• 'X:\Anime\NotThere\' -- Filediscovery: Path not found: X:\Anime\NotThere\
• 'X:\Anime\NotThere' -- Filediscovery: Path not found: X:\Anime\NotThere
• X:\Anime\NotThere\ -- Filediscovery: Path not found: X:\Anime\NotThere\
• X:\Anime\NotThere -- Filediscovery: Path not found: X:\Anime\NotThere

[DvdKhl]

Windows is a bit weird when it comes to console arguments.
If you add a trailing "\" and add quotes around it, the "\" will act as an escaping character for ".
It will work for example if you use this: "X:\Anime\Steins Gate\\"

Your case "X:\Anime\Kanon\" should not work either. It least it didn't when I tried it just now.

[GrumpyBear57]

Well that isn't annoying at all, seeing as Terminal adds the trailing \ automatically when tab completing a path... Maybe it could be trimmed off before the string gets parsed into a path?

It will work for example if you use this: "X:\Anime\Steins Gate\"

I'm not sure what you mean by this, as this is one of the examples I explicitly state does not work. Perhaps you meant to add an additional \ (making it "X:\Anime\Steins Gate\\") which I just tested and works properly.

Your case "X:\Anime\Kanon" should not work either. It least it didn't when I tried it just now.

So, assuming you're missing a trailing \ on the path again, I might have an explanation for this. Using Terminal, it correctly finds the path, however using Command Prompt, the trailing \ will escape the ", and break the path. Why it works like this in Terminal and not Command Prompt, beyond me. Why Terminal doesn't work like this when there happens to be a space in the path is even more beyond me. My best guess is Terminal is parsing the string into a path before it passes it to the program, and removing unnecessary bits, which makes the case with no space work, but the case with a space fail, since the space requires the path string to be enclosed in ".

EDIT: I see now why the quotes are missing a \, because Github Markdown is escaping it.

[DvdKhl]

Yes I also just now noticed that markdown didn't display it. I fixed that now.

I can't easily "fix" that since I'm using the preparsed arguments which splits the arguments into an array of strings, already containing the error.

I may switch to completely parsing the arguments myself on windows in the foreseeable future to avoid that issue.

[kheldorn]

Not sure what you guys are doing, but if I tab-complete a path to a folder in the Windows 10 cmd.exe window it does not end with a \. And I get regular double quotes around paths containing spaces.

Are you running it from a powershell console? That does end on \ and also utilizes single quotes around paths containing spaces. If that is the case, just use cmd.exe?

[GrumpyBear57]

@DvdKhl while I certainly won't tell you not to write your own complete parser for args, I don't think it should be necessary. I haven't looked at it too closely, but you should just be able to do a quick check or two on the items in the paths array before you create the FileStreamProvider with it.

@kheldorn, as mentioned in the original post, I'm using Terminal, which I much prefer because it allows multiple tabs rather than opening another window (among various other features). I recognize the fact that Command Prompt does not append the trailing \, but I don't particularly see having to remove a character as a reason to switch my entire command line environment. Not having to remove it would obviously be beneficial, hence the discussion about fixing it with the program.

[GrumpyBear57]

Ah, I think I see what you mean now about not being an easy fix, my previous suggestion would only work for a single path, but since adding two paths in the arguments is valid .\AVDump3CL.exe -R 'Y:\Anime\Steins Gate\' 'Y:\Anime\Steins Gate 0\' gets parsed to Y:\Anime\Steins Gate" Y:\Anime\Steins, Gate, and 0" as the args, which you can't prevent as is. Unfortunately it also looks like Environment.CommandLine returns an escaped string as well, so I'm not sure how one would even go about writing a workaround for when a " is preceded by a \ in the args.

EDIT: According to this stackoverflow answer you can call an external method to get the actual raw string, though I'm not sure how that would work on other platforms. I'm assuming this is what you'd have to do in order to write your own argument parser. Alternatively there's this library that claims to work, though it looks like that might only be for .NET Framework, instead of Core. You might be able to adapt it for Core by replacing the Environment.CommandLine with the result from the P/Invoke though.

[kheldorn]

E:\Programme\avdump_builds\AVD3 Build 8238>dotnet AVDump3CL.dll FROMFILE arguments.txt "t:\#to_sort\"
Testing anidb connection
Filediscovery: Path not found: t:\#to_sort"
Accepted files: 0

E:\Programme\avdump_builds\AVD3 Build 8238>dotnet AVDump3CL.dll FROMFILE arguments.txt "t:\#to_sort"
Testing anidb connection
Accepted files: 111

Only difference is the trailing \ at the end of the path. Doesn't seem fixed to me?