NtQueryDirectoryFile succeeded when it shouldn't

[apkipa]

Environment

• Windows version: Win10 22H2 19045.3208
• Processor architecture: x64
• Dokany version: 2.0.6.1000
• Library type (Dokany/FUSE): Dokany

Check List

• I checked my issue doesn't exist yet
• My issue is valid with mirror default sample and not specific to my user-mode driver implementation
• I can always reproduce the issue with the provided description below.
• I have updated Dokany to the latest version and have reboot my computer after.
• I tested one of the last snapshot from appveyor CI

Description

NtQueryDirectoryFile can be used to enumerate files or check the existence of a specific file, and it has a parameter RestartScan. However, NtQueryDirectoryFile can return faulty results when RestartScan=TRUE in Dokan. Following is test code (it enumerates all files first, then reuses the handle to restart with a non-existent filename as pattern):

#define NOMINMAX

#include <Windows.h>
#include <stdio.h>

typedef _Return_type_success_(return >= 0) LONG NTSTATUS;
#define NT_SUCCESS(Status)  (((NTSTATUS)(Status)) >= 0)

typedef struct _UNICODE_STRING {
    USHORT Length;
    USHORT MaximumLength;
#ifdef MIDL_PASS
    [size_is(MaximumLength / 2), length_is((Length) / 2)] USHORT * Buffer;
#else // MIDL_PASS
    _Field_size_bytes_part_(MaximumLength, Length) PWCH   Buffer;
#endif // MIDL_PASS
} UNICODE_STRING;
typedef UNICODE_STRING *PUNICODE_STRING;
typedef const UNICODE_STRING *PCUNICODE_STRING;

typedef enum _FILE_INFORMATION_CLASS {
    FileDirectoryInformation = 1,
    // ...
} FILE_INFORMATION_CLASS, *PFILE_INFORMATION_CLASS;

typedef struct _IO_STATUS_BLOCK {
    union {
        NTSTATUS Status;
        PVOID Pointer;
    } u;
    ULONG_PTR Information;
} IO_STATUS_BLOCK, *PIO_STATUS_BLOCK;

typedef VOID
(NTAPI *PIO_APC_ROUTINE)(
    IN PVOID ApcContext,
    IN PIO_STATUS_BLOCK IoStatusBlock,
    IN ULONG Reserved);

// NTSYSCALLAPI
NTSTATUS
(NTAPI
*NtQueryDirectoryFile)(
    _In_ HANDLE FileHandle,
    _In_opt_ HANDLE Event,
    _In_opt_ PIO_APC_ROUTINE ApcRoutine,
    _In_opt_ PVOID ApcContext,
    _Out_ PIO_STATUS_BLOCK IoStatusBlock,
    _Out_writes_bytes_(Length) PVOID FileInformation,
    _In_ ULONG Length,
    _In_ FILE_INFORMATION_CLASS FileInformationClass,
    _In_ BOOLEAN ReturnSingleEntry,
    _In_opt_ PUNICODE_STRING FileName,
    _In_ BOOLEAN RestartScan
);

#define STATUS_NO_MORE_FILES             ((NTSTATUS)0x80000006L)
#define STATUS_NO_SUCH_FILE 0xC000000F

void (NTAPI *RtlInitUnicodeString)(
    __inout   PUNICODE_STRING DestinationString,
    __in_opt  PCWSTR SourceString
);

void run(void) {
    NTSTATUS r;
    IO_STATUS_BLOCK iosb;
    char buf[0x250];
    HANDLE h;

    printf("Start\n");

    h = CreateFileW(L".", FILE_LIST_DIRECTORY,
        FILE_SHARE_READ | FILE_SHARE_WRITE | FILE_SHARE_DELETE,
        NULL, OPEN_EXISTING, FILE_FLAG_BACKUP_SEMANTICS, NULL);
    if (h == INVALID_HANDLE_VALUE) {
        return;
    }

    printf("Folder opened\n");

    UNICODE_STRING PatternString;
    RtlInitUnicodeString(&PatternString, L"NON_EXISTENT_FILE");

    r = NtQueryDirectoryFile(
        h, 0i64, 0i64, 0i64, &iosb, buf, sizeof buf,
        FileDirectoryInformation,
        0u,
        nullptr,
        0u
    );
    if (NT_SUCCESS(r)) {
        printf("Success\n");
        while (true) {
            r = NtQueryDirectoryFile(
                h, 0i64, 0i64, 0i64, &iosb, buf, sizeof buf,
                FileDirectoryInformation,
                0u,
                nullptr,
                0u
            );
            if (!NT_SUCCESS(r)) {
                if (r != STATUS_NO_MORE_FILES) {
                    printf("NtQueryDirectoryFile failed, code = 0x%08x\n", r);
                }
                else {
                    r = 0;
                    printf("Drain buffer\n", r);
                }
                break;
            }
        }
    }
    else {
        // STATUS_NO_SUCH_FILE
        printf("NtQueryDirectoryFile failed, code = 0x%08x\n", r);
    }
    r = NtQueryDirectoryFile(
        h, 0i64, 0i64, 0i64, &iosb, buf, sizeof buf,
        FileDirectoryInformation,
        1u,
        &PatternString,
        1u
    );
    printf("NtQueryDirectoryFile result code = 0x%08x\n", r);
    if (r != STATUS_NO_MORE_FILES) {
        printf("WARNING: NtQueryDirectoryFile should return STATUS_NO_MORE_FILES\n");
        if (NT_SUCCESS(r)) {
            printf("ERROR: NtQueryDirectoryFile MUST NOT succeed\n");
        }
    }
    else {
        printf("SUCCESS: Check passed\n");
    }

    CloseHandle(h);
}

void init(void) {
    auto ntdll = GetModuleHandleW(L"ntdll.dll");
#define LOAD_FUNC(mod, name)                        \
    name = reinterpret_cast<decltype(name)>(        \
        GetProcAddress(mod, # name))
    LOAD_FUNC(ntdll, RtlInitUnicodeString);
    LOAD_FUNC(ntdll, NtQueryDirectoryFile);
#undef LOAD_FUNC
}

int main(void) {
    init();
    run();
}

Run the code on different volumes (D: is native NTFS drive, F: is WinFsp memfs, M: is Dokany memfs/mirror):

D:\folder>main.exe
Start
Folder opened
Success
Drain buffer
NtQueryDirectoryFile result code = 0x80000006
SUCCESS: Check passed

D:\folder>M:

M:\folder>main.exe
Start
Folder opened
Success
Drain buffer
NtQueryDirectoryFile result code = 0x00000000
WARNING: NtQueryDirectoryFile should return STATUS_NO_MORE_FILES
ERROR: NtQueryDirectoryFile MUST NOT succeed

M:\folder>F:

F:\folder>main.exe
Start
Folder opened
Success
Drain buffer
NtQueryDirectoryFile result code = 0xc000000f
WARNING: NtQueryDirectoryFile should return STATUS_NO_MORE_FILES

In my own implementation of FindFilesWithPattern, I can see that a stale pattern is being used for the second scan, which seems to be the root cause. This issue prevents msvc from building some C++ projects on a Dokan drive.

Logs

Please attach in separate files: mirror output, library logs and kernel logs.
In case of BSOD, please attach minidump or dump analyze output.

[Liryna]

Hi @apkipa ,

That's an awesome finding! Thank you for sharing the issue and including a repro code.
This should be easy to fix whether in the kernel or in the library.
Have you already looked for a solution?

[apkipa]

Not really, as I am unable to set up a working test environment for now. My guess is that the problem lies around

dokany/sys/directory.c

Lines 121 to 126 in ce8b822

	initial = (BOOLEAN)(ccb->SearchPattern == NULL &&
	!(DokanCCBFlagsIsSet(ccb, DOKAN_DIR_MATCH_ALL)));
	// this is an initial query
	if (initial) {
	DOKAN_LOG_FINE_IRP(RequestContext, "Initial query");

and adding an extra check for SL_RESTART_SCAN should work. I'm not sure whether this is the proper fix (i.e. behaves exactly like NTFS), though.

[LTRData]

If we look at how this is implemented in cdfs.sys and fastfat.sys, there seems indeed to be a check for SL_RESTART_SCAN and a reset of the logic in that case:
CD: https://github.com/microsoft/Windows-driver-samples/blob/main/filesys/cdfs/dirctrl.c#L956
FAT: https://github.com/microsoft/Windows-driver-samples/blob/main/filesys/fastfat/dirctrl.c#L327