CLI patterns differ on Windows vs Linux

[janphkre]

Expected Behavior

The glob patterns as described in https://pinterest.github.io/ktlint/latest/install/cli/#globs should work the same way running ktlint on any underlying operating system and should match the same files given the same file structure.

Observed Behavior

At the moment (using 1.3.1) glob patterns do not match the same files on Windows as on Linux.
Here are two examples:
*.kt matches any file with the .kt file-ending in any subdirectory on Windows, however the same glob does not match any subdirectory on Linux.

**/test*/** seems to drop / ignore the second slash on Windows, since it matches a file at the location "src/example/TestFile.kt". On Linux this behaviour can not be observed.

Generally Linux seems to be in line with the git manpage and online glob testing sites.
I suspect that the single star wildcard is not being handled correctly on Windows machines.

Sadly, the ktlint command does not produce any meaning full tracing output, similar to the test's output mentione below (with trace logging enabled):

[DEBUG] c.p.ktlint.cli.internal.FileUtils - Start walkFileTree from directory: 'C:\project\project1'
[DEBUG] c.p.ktlint.cli.internal.FileUtils - Discovered 2 files to be processed in 1 ms
[INFO] c.p.k.cli.internal.FileUtilsTest - Getting files with [patterns = [**/test*/**]] and [rootdir = C:\project\project1] returns [files = [project1/src/mock/kotlin/TestOneSetup.kt, project1/src/testMock/kotlin/TwoTest.kt]]

Steps to Reproduce

Following test can be added to https://github.com/pinterest/ktlint/blob/master/ktlint-cli/src/test/kotlin/com/pinterest/ktlint/cli/internal/FileUtilsTest.kt

    @Test
    fun `Find files with correct glob for wildcards`() {
        val ktFileInProjectOutsideFlavoredDirectoryWithOverlappingName = "project1/src/mock/kotlin/TestOneSetup.kt"
        val ktFileInProjectFlavoredDirectoryWithoutOverlappingName = "project1/src/testMock/kotlin/TwoTest.kt"

        ktlintTestFileSystem.createFile(ktFileInProjectOutsideFlavoredDirectoryWithOverlappingName)
        ktlintTestFileSystem.createFile(ktFileInProjectFlavoredDirectoryWithoutOverlappingName)

        val foundFiles =
            getFiles(
                patterns = listOf("**/test*/**"),
                rootDir = ktlintTestFileSystem.resolve(ktFileInProjectRootDirectory).parent.toAbsolutePath(),
            )

        assertThat(foundFiles)
            .containsExactlyInAnyOrder(
                ktFileInProjectFlavoredDirectoryWithoutOverlappingName
            ).doesNotContain(
                ktFileInProjectOutsideFlavoredDirectoryWithOverlappingName
            )
    }

Your Environment

• Version of ktlint used: 1.3.1
• Issue occurs with or without any editorconfig being defined.
• Name and version (or code for custom task) of integration used (Gradle plugin, Maven plugin, command line, custom Gradle task): ktlint-cli
• Version of Gradle used (if applicable): none
• Operating System and version: observable on Windows 10 & Windows 11

[paul-dingemans]

Hi @janphkre
Tnx for your observation and the test case. This test case is succeeding on MacOs. Is it correct (e.g. to be expected) that this test will fail on Windows?

[janphkre]

Yes, that is correct. The test is failing on Windows:

java.lang.AssertionError: 
Expecting actual:
  ["project1/src/mock/kotlin/TestOneSetup.kt",
    "project1/src/testMock/kotlin/TwoTest.kt"]
to contain exactly in any order:
  ["project1/src/testMock/kotlin/TwoTest.kt"]
but the following elements were unexpected:
  ["project1/src/mock/kotlin/TestOneSetup.kt"]

[paul-dingemans]

**/test*/** seems to drop / ignore the second slash on Windows, since it matches a file at the location "src/example/TestFile.kt". On Linux this behaviour can not be observed.

The ** means zero or more subdirectories. Basically this means that pattern **/test*/** is identical to **/test* or **/test*/. As Windows has case insensitive paths, the pattern matches file TestOneSetup.kt.

Your test does not fail on Linux as Linux has case sensitive paths. If you would alter the pattern in your test to **/Test*/** it fails as well. To make the test results identical, you should also change the directory name testMock to TestMock.

Changing your test as follows, shows that the results on Windows and Linux both fail for same reason as you reported above:

    @Test
    fun `Issue 2781b - Find files with correct glob for wildcards`() {
        val ktFileInProjectOutsideFlavoredDirectoryWithOverlappingName = "project1/src/mock/kotlin/TestOneSetup.kt"
        // Path "testMock" changed to "TestMock" so that results on Windows and Linux are comparable
        val ktFileInProjectFlavoredDirectoryWithoutOverlappingName = "project1/src/TestMock/kotlin/TwoTest.kt"

        ktlintTestFileSystem.createFile(ktFileInProjectOutsideFlavoredDirectoryWithOverlappingName)
        ktlintTestFileSystem.createFile(ktFileInProjectFlavoredDirectoryWithoutOverlappingName)

        val foundFiles =
            getFiles(
                // Linux has case-sensitive path. This test should now fail on both Linux and Windows
                patterns = listOf("**/Test*/**"),
                rootDir = ktlintTestFileSystem.resolve(ktFileInProjectRootDirectory).parent.toAbsolutePath(),
            )

        assertThat(foundFiles)
            .containsExactlyInAnyOrder(
                ktFileInProjectFlavoredDirectoryWithoutOverlappingName,
            ).doesNotContain(
                ktFileInProjectOutsideFlavoredDirectoryWithOverlappingName,
            )
    }

Changing the glob to **/Test*/**/*.kt* makes the test pass on both Windows and Linux. It looks like that the confusion is caused by using ** as trailing part in pattern, and not taken into account the difference between case sensitivity of paths on Linux and Windows.

[janphkre]

After doing some more testing, I still believe that there is something not working as intended with the way two conesecutive aserisks are being handled in ktlint.

The difference in behaviour may come from the case sensitivity of the file system, but that is not the exact issue I was trying to point out here.

You correctly acknowledged the behaviour in your comment:

The ** means zero or more subdirectories.

This means, that anything before the ** is treated as a directory. This is not the case at the momenet and is not in line with the way git uses the gitignore file, even though the documentation is linking to that for the behaviour of the globs.
This is also being mentioned in the git documentationas to not have any confusion on how trailing ** should work.

A trailing "/**" matches everything inside. For example, "abc/**" matches all files inside directory "abc", relative to the location of the .gitignore file, with infinite depth.

So the issue I want to point out here is, that ktlint is somehow ommitting any slash before a **.
Altering the test to use the upeprcase word "Test" everywhere like you suggests, still produces a failure on both OSs, even though git would ignore these files with the same pattern, so even though the title of this issue may not be correct, I still see it as an issue that the test is not passing in the case you described. Changing the glob to **/Test*/**/* is also enough to satisfy the pattern matching as a workaround, though it should not be necessary, since the slash after Test* should not be ignored here.

[paul-dingemans]

So the issue I want to point out here is, that ktlint is somehow ommitting any slash before a **.
Altering the test to use the upeprcase word "Test" everywhere like you suggests, still produces a failure on both OSs, even though git would ignore these files with the same pattern, so even though the title of this issue may not be correct, I still see it as an issue that the test is not passing in the case you described.

Thanks for pointing this out. The way in which is traling ** is handled by ktlint is not in sync with .gitignore. See #2787 for fix.