[Bug] Unit test failures on main branch

[fice-t]

If these aren't showing up for others, then this may well be a Windows-specific issue.

Using ./gradlew testDebugUnitTest --tests org.mozilla.fenix.downloads*:
From SUITE: org.mozilla.fenix.downloads.DynamicDownloadDialogTest:

  TEST: WHEN calling getCannotOpenFileErrorMessage THEN should return the error message for the download file type
    W/PackageParser: Unknown element under <manifest>: queries at .\fenix\app\build\intermediates\apk_for_local_test\debugUnitTest\apk-for-local-test.ap_ Binary XML file line #40
  FAILURE

org.junit.ComparisonFailure: expected:<...o app found to open [gif] files> but was:<...o app found to open [] files>
        at org.junit.Assert.assertEquals(Assert.java:115)
        at org.junit.Assert.assertEquals(Assert.java:144)
        at org.mozilla.fenix.downloads.DynamicDownloadDialogTest.WHEN calling getCannotOpenFileErrorMessage THEN should return the error message for the download file type(DynamicDownloadDialogTest.kt:35)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.robolectric.RobolectricTestRunner$HelperTestRunner$1.evaluate(RobolectricTestRunner.java:591)
        at org.robolectric.internal.SandboxTestRunner$2.lambda$evaluate$0(SandboxTestRunner.java:274)
        at org.robolectric.internal.bytecode.Sandbox.lambda$runOnMainThread$0(Sandbox.java:88)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)

Using ./gradlew testDebugUnitTest --tests org.mozilla.fenix.wallpaper*:

From SUITE: org.mozilla.fenix.wallpapers.WallpaperDownloaderTest:

  TEST: GIVEN that thumbnail request is successful WHEN downloading THEN file is created in expected location
  FAILURE

java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest$GIVEN that thumbnail request is successful WHEN downloading THEN file is created in expected location$1.invokeSuspend(WallpaperDownloaderTest.kt:86)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest$GIVEN that thumbnail request is successful WHEN downloading THEN file is created in expected location$1.invoke(WallpaperDownloaderTest.kt)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest$GIVEN that thumbnail request is successful WHEN downloading THEN file is created in expected location$1.invoke(WallpaperDownloaderTest.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTestCoroutine$2.invokeSuspend(TestBuilders.kt:212)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTestCoroutine$2.invoke(TestBuilders.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTestCoroutine$2.invoke(TestBuilders.kt)
        at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
        at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:112)
        at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:126)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTestCoroutine(TestBuilders.kt:211)
        at kotlinx.coroutines.test.TestBuildersKt.runTestCoroutine(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invokeSuspend(TestBuilders.kt:167)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invoke(TestBuilders.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invoke(TestBuilders.kt)
        at kotlinx.coroutines.test.TestBuildersJvmKt$createTestResult$1.invokeSuspend(TestBuildersJvm.kt:13)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersJvmKt.createTestResult(TestBuildersJvm.kt:12)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest(TestBuilders.kt:166)
        at kotlinx.coroutines.test.TestBuildersKt.runTest(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest(TestBuilders.kt:154)
        at kotlinx.coroutines.test.TestBuildersKt.runTest(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest$default(TestBuilders.kt:147)
        at kotlinx.coroutines.test.TestBuildersKt.runTest$default(Unknown Source)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest.GIVEN that thumbnail request is successful WHEN downloading THEN file is created in expected location(WallpaperDownloaderTest.kt:75)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
        at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
        at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
        at com.sun.proxy.$Proxy5.processTestClass(Unknown Source)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
        at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)

  TEST: GIVEN that asset request is successful WHEN downloading assets THEN both files are created in expected location
  FAILURE

java.lang.AssertionError
        at org.junit.Assert.fail(Assert.java:86)
        at org.junit.Assert.assertTrue(Assert.java:41)
        at org.junit.Assert.assertTrue(Assert.java:52)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest$GIVEN that asset request is successful WHEN downloading assets THEN both files are created in expected location$1.invokeSuspend(WallpaperDownloaderTest.kt:70)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest$GIVEN that asset request is successful WHEN downloading assets THEN both files are created in expected location$1.invoke(WallpaperDownloaderTest.kt)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest$GIVEN that asset request is successful WHEN downloading assets THEN both files are created in expected location$1.invoke(WallpaperDownloaderTest.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTestCoroutine$2.invokeSuspend(TestBuilders.kt:212)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTestCoroutine$2.invoke(TestBuilders.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTestCoroutine$2.invoke(TestBuilders.kt)
        at kotlinx.coroutines.intrinsics.UndispatchedKt.startCoroutineUndispatched(Undispatched.kt:55)
        at kotlinx.coroutines.CoroutineStart.invoke(CoroutineStart.kt:112)
        at kotlinx.coroutines.AbstractCoroutine.start(AbstractCoroutine.kt:126)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTestCoroutine(TestBuilders.kt:211)
        at kotlinx.coroutines.test.TestBuildersKt.runTestCoroutine(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invokeSuspend(TestBuilders.kt:167)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invoke(TestBuilders.kt)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt$runTest$1$1.invoke(TestBuilders.kt)
        at kotlinx.coroutines.test.TestBuildersJvmKt$createTestResult$1.invokeSuspend(TestBuildersJvm.kt:13)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:33)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:106)
        at kotlinx.coroutines.EventLoopImplBase.processNextEvent(EventLoop.common.kt:284)
        at kotlinx.coroutines.BlockingCoroutine.joinBlocking(Builders.kt:85)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking(Builders.kt:59)
        at kotlinx.coroutines.BuildersKt.runBlocking(Unknown Source)
        at kotlinx.coroutines.BuildersKt__BuildersKt.runBlocking$default(Builders.kt:38)
        at kotlinx.coroutines.BuildersKt.runBlocking$default(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersJvmKt.createTestResult(TestBuildersJvm.kt:12)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest(TestBuilders.kt:166)
        at kotlinx.coroutines.test.TestBuildersKt.runTest(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest(TestBuilders.kt:154)
        at kotlinx.coroutines.test.TestBuildersKt.runTest(Unknown Source)
        at kotlinx.coroutines.test.TestBuildersKt__TestBuildersKt.runTest$default(TestBuilders.kt:147)
        at kotlinx.coroutines.test.TestBuildersKt.runTest$default(Unknown Source)
        at org.mozilla.fenix.wallpapers.WallpaperDownloaderTest.GIVEN that asset request is successful WHEN downloading assets THEN both files are created in expected location(WallpaperDownloaderTest.kt:55)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
        at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
        at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
        at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
        at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
        at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48)
        at org.junit.rules.RunRules.evaluate(RunRules.java:20)
        at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
        at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
        at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
        at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
        at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
        at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
        at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
        at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.runTestClass(JUnitTestClassExecutor.java:110)
        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:58)
        at org.gradle.api.internal.tasks.testing.junit.JUnitTestClassExecutor.execute(JUnitTestClassExecutor.java:38)
        at org.gradle.api.internal.tasks.testing.junit.AbstractJUnitTestClassProcessor.processTestClass(AbstractJUnitTestClassProcessor.java:62)
        at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.processTestClass(SuiteTestClassProcessor.java:51)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.base/java.lang.reflect.Method.invoke(Method.java:566)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
        at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
        at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
        at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
        at com.sun.proxy.$Proxy5.processTestClass(Unknown Source)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker$2.run(TestWorker.java:176)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
        at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
        at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:133)
        at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
        at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)

┆Issue is synchronized with this Jira Task

[fice-t]

Right, these are Windows-specific path separator issues. The wallpaper tests are fixed with:

diff --git a/app/src/main/java/org/mozilla/fenix/wallpapers/WallpaperDownloader.kt b/app/src/main/java/org/mozilla/fenix/wallpapers/WallpaperDownloader.kt
index d0ecc4252..c5b298997 100644
--- a/app/src/main/java/org/mozilla/fenix/wallpapers/WallpaperDownloader.kt
+++ b/app/src/main/java/org/mozilla/fenix/wallpapers/WallpaperDownloader.kt
@@ -80,7 +80,7 @@ class WallpaperDownloader(
             if (!response.isSuccess) {
                 throw IllegalStateException()
             }
-            File(localFile.path.substringBeforeLast("/")).mkdirs()
+            localFile.parentFile!!.mkdirs()
             response.body.useStream { input ->
                 input.copyTo(localFile.outputStream())
             }

The first is due to Android's MimeTypeMap.getFileExtensionFromUrl not accepting backslashes. Easy workaround is to just replace all backslashes with with forward slashes in the test before calling getCannotOpenFileErrorMessage.

[Mugurell]

Right, these are Windows-specific path separator issues.
The first is due to Android's MimeTypeMap.getFileExtensionFromUrl not accepting backslashes. Easy workaround is to just replace all backslashes with with forward slashes in the test before calling getCannotOpenFileErrorMessage.

To ensure the slashes are compatible with all platforms we should use File.separator instead of the slash character.

[fice-t]

@Mugurell I made a PR for the wallpaper tests.

To ensure the slashes are compatible with all platforms we should use File.separator instead of the slash character.

In this case, the error actually arises because A-C's DownloadState uses File.separator instead of assuming / because getCannotOpenFileErrorMessage calls Android's MimeTypeMap.getFileExtensionFromUrl, which doesn't respect File.separator.

Possible solutions:

1. Use a different method to get the file extension from the URL. Kotlin has an extension method File.extension that looks suitable.
2. Use the workaround above and simply replace backslashes with forward slashes in DynamicDownloadDialogTest. This would mean possible unexpected behaviour when running Fenix/AC on a Windows OS directly, but that does seem quite unlikely outside of tests.