Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test: do not use file names with illegal characters in tests #1421

Merged
merged 1 commit into from
Dec 7, 2022

Conversation

rdeltour
Copy link
Member

@rdeltour rdeltour commented Dec 5, 2022

File names with EPUB-illegal characters are also NTFS-invalid names, so they can cause issues (cannot be easily checked out by git on Windows or may cause runtime issues on NTFS).

This commit replaces tests using files with invalid characters by tests using files with a full stop as the last character. Other illegal characters are tested in the file name checker unit tests.

Fix #1420.

@rdeltour rdeltour added this to the v5.0.0 milestone Dec 5, 2022
@rdeltour rdeltour self-assigned this Dec 5, 2022
@mattgarrish
Copy link
Member

I can check this one out fine, but I don't get the expected error message when building:

Report forbidden characters in file names(EPUB 3 ù Open Container Format)  Time elapsed: 0.047 sec  <<< FAILURE!
java.lang.AssertionError: No message found with ID PKG-011
Expected: is not null
     but: was null
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessageOnce(AssertionSteps.java:70)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessage(AssertionSteps.java:58)
        at ?.error PKG-011 is reported(classpath:epub3/04-ocf/ocf.feature:65)

Report forbidden characters in file names even for non-publication resources(EPUB 3 ù Open Container Format)  Time elapsed: 0.052 sec  <<< FAILURE!
java.lang.AssertionError: No message found with ID PKG-011
Expected: is not null
     but: was null
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessageOnce(AssertionSteps.java:70)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessage(AssertionSteps.java:58)
        at ?.error PKG-011 is reported(classpath:epub3/04-ocf/ocf.feature:73)

@rdeltour
Copy link
Member Author

rdeltour commented Dec 5, 2022

I don't get the expected error message when building

that's weird, the tests pass for me and I don't see any OS-dependent thing.
Did you clean the build (mvn clean) before testing?

If not, can you add a @debug tag to the two relevant scenarios, re-run mvn clean test -Dtest=DebugCucumberTest, and paste the output here?

[edit: fix typo]

@mattgarrish
Copy link
Member

[edit: fix typo]

I checked the man page just in case... 😉

Here's the full output from the tests:

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.w3c.epubcheck.DebugCucumberTest

@debug @spec @xref:sec-container-filenames
Scenario: Report forbidden characters in file names           # epub3/04-ocf/ocf.feature:60
  Given EPUB test files located at '/epub3/04-ocf/files/'     # org.w3c.epubcheck.test.TestConfiguration.setBasepath(java.lang.String)
  And EPUBCheck with default settings                         # org.w3c.epubcheck.test.TestConfiguration.configureDefaults()
INFO null: [size] 20 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compressed size] 20 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compression method] Stored | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [SHA-256] e468e350d1143eb648f6c7bbd63111ec544a361ca74ecef256ac91f48b | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [size] 252 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compressed size] 185 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [SHA-256] 9fbdf6b6dc65fdf4fa5fe32aa6483a97ed153f03e6eac70a4e55a1e69f4f345 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [size] 330 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compressed size] 222 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [SHA-256] f7aaf6476f59c9196761517081152731b46ea0d5041384ddc9b155dfa7fd4b | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [size] 715 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compressed size] 373 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [SHA-256] 9c2e1ceda7bbdaea9d7fdef8cb687690574c65889f4422b7a6958a498d2dfcc | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [size] 0 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compressed size] 2 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [SHA-256] e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [size] 248 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compressed size] 171 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [SHA-256] 957946f4cbee2de5d554d61caa81399796c911708251fdfdb257d860a85157 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-error.epub
INFO null: [format name] application/epub+zip
INFO null: [format version] 3.3
INFO null: [title] Minimal EPUB 3.0
INFO null: [language] en
INFO null: [unique identifier] NOID
INFO null: [modification date] 2017-06-14T00:00:01Z
INFO null: [unique identifier] content_001 | in content_001.xhtml
INFO null: [unique identifier] nav | in nav.xhtml
INFO null: [unique identifier] video | in video.
INFO null: [Has ncx file] false
INFO null: [is spine item] true | in EPUB/content_001.xhtml
INFO null: [linear] true | in EPUB/content_001.xhtml
INFO null: [spine index] 0 | in EPUB/content_001.xhtml
INFO null: [items count] 3
INFO null: [declared mimetype] application/xhtml+xml | in EPUB/content_001.xhtml
INFO null: [declared mimetype] application/xhtml+xml | in EPUB/nav.xhtml
INFO null: [declared mimetype] video/mp4 | in EPUB/video.
INFO null: [characters count] 60 | in EPUB/content_001.xhtml
INFO null: [characters count] 56 | in EPUB/nav.xhtml
ERROR RSC-001: File "EPUB/video." could not be found. | in EPUB/video.
WARNING OPF-003: Item "EPUB/video" exists in the EPUB, but is not declared in the OPF manifest. | in /D:/repos/w3c/epubcheck/target/test-classes/epub3/04-ocf/files/ocf-filename-character-forbidden-error.epub
  When checking EPUB 'ocf-filename-character-forbidden-error' # org.w3c.epubcheck.test.ExecutionSteps.check(java.lang.String)
  Then error PKG-011 is reported                              # org.w3c.epubcheck.test.AssertionSteps.assertMessage(com.adobe.epubcheck.messages.Severity,com.adobe.epubcheck.messages.MessageId,int,java.lang.String)
      java.lang.AssertionError: No message found with ID PKG-011
Expected: is not null
     but: was null
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessageOnce(AssertionSteps.java:70)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessage(AssertionSteps.java:58)
        at ✽.error PKG-011 is reported(classpath:epub3/04-ocf/ocf.feature:65)

  And no other errors or warnings are reported                # org.w3c.epubcheck.test.AssertionSteps.assertNoErrorsOrWarning()

@debug @spec @xref:sec-container-filenames
Scenario: Report forbidden characters in file names even for non-publication resources # epub3/04-ocf/ocf.feature:69
  Given EPUB test files located at '/epub3/04-ocf/files/'                              # org.w3c.epubcheck.test.TestConfiguration.setBasepath(java.lang.String)
  And EPUBCheck with default settings                                                  # org.w3c.epubcheck.test.TestConfiguration.configureDefaults()
INFO null: [size] 20 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compressed size] 20 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compression method] Stored | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [SHA-256] e468e350d1143eb648f6c7bbd63111ec544a361ca74ecef256ac91f48b | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [size] 228 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compressed size] 172 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [SHA-256] e7f4a3d7beca3a7ab777dbdefb2666c205d7f5596ab2e98653fc59573f18d0 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [size] 330 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compressed size] 222 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [SHA-256] f7aaf6476f59c9196761517081152731b46ea0d5041384ddc9b155dfa7fd4b | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [size] 656 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compressed size] 353 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [SHA-256] 12acc870653bf7c6bdd6eb9e9088e8c2ea7a48e61816d0658c1db0525721312c | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [size] 248 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compressed size] 171 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [SHA-256] 957946f4cbee2de5d554d61caa81399796c911708251fdfdb257d860a85157 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [size] 0 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compressed size] 2 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [compression method] Deflated | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [SHA-256] e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855 | in D:\repos\w3c\epubcheck\target\test-classes\epub3\04-ocf\files\ocf-filename-character-forbidden-non-publication-resource-error.epub
INFO null: [format name] application/epub+zip
INFO null: [format version] 3.3
INFO null: [title] Minimal EPUB 3.0
INFO null: [language] en
INFO null: [unique identifier] NOID
INFO null: [modification date] 2017-06-14T00:00:01Z
INFO null: [unique identifier] content_001 | in content_001.xhtml
INFO null: [unique identifier] nav | in nav.xhtml
INFO null: [Has ncx file] false
INFO null: [is spine item] true | in EPUB/content_001.xhtml
INFO null: [linear] true | in EPUB/content_001.xhtml
INFO null: [spine index] 0 | in EPUB/content_001.xhtml
INFO null: [items count] 2
INFO null: [declared mimetype] application/xhtml+xml | in EPUB/content_001.xhtml
INFO null: [declared mimetype] application/xhtml+xml | in EPUB/nav.xhtml
INFO null: [characters count] 57 | in EPUB/content_001.xhtml
INFO null: [characters count] 56 | in EPUB/nav.xhtml
  When checking EPUB 'ocf-filename-character-forbidden-non-publication-resource-error' # org.w3c.epubcheck.test.ExecutionSteps.check(java.lang.String)
  Then error PKG-011 is reported                                                       # org.w3c.epubcheck.test.AssertionSteps.assertMessage(com.adobe.epubcheck.messages.Severity,com.adobe.epubcheck.messages.MessageId,int,java.lang.String)
      java.lang.AssertionError: No message found with ID PKG-011
Expected: is not null
     but: was null
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessageOnce(AssertionSteps.java:70)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessage(AssertionSteps.java:58)
        at ✽.error PKG-011 is reported(classpath:epub3/04-ocf/ocf.feature:73)

  And no other errors or warnings are reported                                         # org.w3c.epubcheck.test.AssertionSteps.assertNoErrorsOrWarning()
Tests run: 2, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 7.781 sec <<< FAILURE! - in org.w3c.epubcheck.DebugCucumberTest
Report forbidden characters in file names(EPUB 3 ù Open Container Format)  Time elapsed: 6.955 sec  <<< FAILURE!
java.lang.AssertionError: No message found with ID PKG-011
Expected: is not null
     but: was null
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessageOnce(AssertionSteps.java:70)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessage(AssertionSteps.java:58)
        at ?.error PKG-011 is reported(classpath:epub3/04-ocf/ocf.feature:65)

Report forbidden characters in file names even for non-publication resources(EPUB 3 ù Open Container Format)  Time elapsed: 0.082 sec  <<< FAILURE!
java.lang.AssertionError: No message found with ID PKG-011
Expected: is not null
     but: was null
        at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessageOnce(AssertionSteps.java:70)
        at org.w3c.epubcheck.test.AssertionSteps.assertMessage(AssertionSteps.java:58)
        at ?.error PKG-011 is reported(classpath:epub3/04-ocf/ocf.feature:73)


Results :

Failed tests:
  No message found with ID PKG-011
Expected: is not null
     but: was null
  No message found with ID PKG-011
Expected: is not null
     but: was null

Tests run: 2, Failures: 2, Errors: 0, Skipped: 0

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  25.382 s
[INFO] Finished at: 2022-12-05T12:18:27-04:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-surefire-plugin:2.18.1:test (default-test) on project epubcheck: There are test failures.
[ERROR]
[ERROR] Please refer to D:\repos\w3c\epubcheck\target\surefire-reports for the individual test results.
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

@rdeltour
Copy link
Member Author

rdeltour commented Dec 5, 2022

ERROR RSC-001: File "EPUB/video." could not be found. | in EPUB/video.
WARNING OPF-003: Item "EPUB/video" exists in the EPUB, but is not declared in the OPF manifest.

Interesting, it seems the file ending with full-stop was not checked out as-is, somehow the final full-stop character was removed… (by Windows git?).

@rdeltour
Copy link
Member Author

rdeltour commented Dec 5, 2022

Did git give you any warning about that when checking out this branch?

@rdeltour
Copy link
Member Author

rdeltour commented Dec 5, 2022

Also, what do you see in your file explorer? (I'm trying to figure out if the full-stop character was removed at check-out time or if it's a runtime issue)

@rdeltour
Copy link
Member Author

rdeltour commented Dec 5, 2022

(by the way let me know if you don't have more time to spend on this, and I'll dust off my Windows VM 😊)

@rdeltour
Copy link
Member Author

rdeltour commented Dec 5, 2022

Ah, just seeing that the Windows naming conventions say:

Do not end a file or directory name with a space or a period. Although the underlying file system may support such names, the Windows shell and user interface does not.

Mmm. I'm considering adding a step instruction to disable these tests on Windows. (Given the runtime OS is not Windows, When …)

@mattgarrish
Copy link
Member

mattgarrish commented Dec 5, 2022

Sorry, didn't see the new emails come in while I was going through the rest of the PRs.

No, there weren't any error messages from git when I checked it out. The weird thing is it doesn't recognize any files as missing after checkout, but it picks up the two files you mention as unversioned. This is immediately after deleting the branch and checking it out again, so it's nothing from the epubcheck build.

@mattgarrish
Copy link
Member

I'm considering adding a step instruction to disable these tests on Windows.

Ya, I'd go with whatever is easiest. This is a pretty minor piece in the overall scheme of things.

@rdeltour
Copy link
Member Author

rdeltour commented Dec 5, 2022

there weren't any error messages from git when I checked it out. The weird thing is it doesn't recognize any files as missing after checkout, but it picks up the two files you mention as unversioned

yeah, in fact this PR doesn't fix anything: files ending with a period are rejected like the ones with invalid characters before… it's just that you were not warned because you had disabled the NTFS protection setting earlier while we were troubleshooting the issue 😉 so back to square one.

I'm going to try if a packaged EPUB works (i.e. can be unpacked without creating runtime trouble).

File names with EPUB-illegal characters are also NTFS-invalid names,
so they can cause issues (cannot be easily checked out by git on Windows
or may cause runtime issues on NTFS).

This commit replaces the expanded EPUBs in invalid characters by tests
by packaged EPUBs.

Fix #1420.
@rdeltour rdeltour force-pushed the fix/test-illegal-ocf-filenames branch from f83e394 to 2809c23 Compare December 6, 2022 08:29
@rdeltour
Copy link
Member Author

rdeltour commented Dec 6, 2022

@mattgarrish I ended up using packaged EPUBs. It works fine on my Windows 10 box:

  • I could check out successfully, even with git config core.protectNTFS true
  • the tests pass

Base automatically changed from release/v5.0.0-beta-3 to release/v5.0.0 December 6, 2022 14:57
@mattgarrish
Copy link
Member

Ya, this obviously works for me now, too.

@rdeltour rdeltour merged commit 84dd8e4 into release/v5.0.0 Dec 7, 2022
@rdeltour rdeltour deleted the fix/test-illegal-ocf-filenames branch December 7, 2022 15:19
@rdeltour rdeltour modified the milestones: v5.0.0, v5.0.0-rc Dec 8, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants