[Mutation Testing] Pitest Gradle Plugin #12208
Draft
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Release Notes (Mockito): https://github.com/mockito/ mockito/releases/tag/v5.12.0 Release Notes (Mockito Kotlin): https://github.com/mockito/ mockito-kotlin/releases/tag/5.4.0 ------------------------------------------------------------------------ Note that the 'org.mockito:mockito-inline' dependency is now switched with 'org.mockito:mockito-core' and this is because the inline bytecode functionality that 'mockito-inline' provided is now the default and part of 'mockito-core'. Mockito version '5.0.0' introduced this change. ------------------------------------------------------------------------ FYI: This is done in order to make sure that Pitest is working with the most recent version of Mockito when running tests. This is because, sometimes, even though normally tests run green, Pitest says that the suite isn't green. And this happens because Pitest, to make things fast, does some tricky stuff to split your tests into small independent units. For more info see: https://pitest.org/faq/
This change is required as part of the Mockito update, which is causing the test to compile with the below error message: Error Message: "Cannot inline bytecode built with JVM target 11 into bytecode that is being built with JVM target 1.8. Please specify proper '-jvm-target' option"
This is done because otherwise the Pitest is failing when running some tests with the below example failure message: Example Failure Message: "java.lang.RuntimeException: Method i in android.util.Log not mocked. See https://developer.android.com/r/ studio-ui/build/not-mocked for details." For more info see: https://developer.android.com/training/testing/ local-tests#mocking-dependencies
For the Pitest plugin to work with Gradle, the project needs to use the 'gradle-pitest-plugin'. However, this Pitest Gradle plugin is not working with Android project. As such, a fork of that was created by Karlo, an Android developer working with 'Droids On Roids', a mobile app development company. Main Gradle Plugin: https://github.com/szpak/gradle-pitest-plugin Fork Gradle Plugin: https://github.com/koral--/gradle-pitest-plugin ------------------------------------------------------------------------ PS: Arcmutate and its accompanying Kotlin plugin can further enhance the mutation testing experience on Kotlin projects, but this requires a licence first. The good news is that using this license is free for open source projects. Arcmutate Kotlin Support: https://docs.arcmutate.com/docs/ kotlin.html#installation
This is done because otherwise mutation testing will fail with the below build failure message: Build Failure Message: "(9) tests did not pass without mutation when calculating line coverage" From those 9 tests, 5 are related to the 'OrderDetailsTransactionLauncherTest' test suite, while the rest 4 are related to the 'OrderListTransactionLauncherTest' test suite. The problem is that Mockiot cannot mock these classes, as it cannot mock final classes with the following settings: - explicit serialization (e.g. withSettings().serializable()) - extra interfaces (e.g. withSettings().extraInterfaces(...)) ------------------------------------------------------------------------ For more info see the full(ish) stacktrace below: Mockito cannot mock this class: class com.automattic.android.tracks. crashlogging.performance.PerformanceTransactionRepository. Can not mock final classes with the following settings : - explicit serialization (e.g. withSettings().serializable()) - extra interfaces (e.g. withSettings().extraInterfaces(...)) ... Caused by: org.mockito.exceptions.base.MockitoException: Could not modify all classes [class com.automattic.android.tracks. crashlogging.performance.PerformanceTransactionRepository] ... Caused by: java.lang.IllegalStateException: Byte Buddy could not instrument all classes within the mock's type hierarchy ... This problem should never occur for javac-compiled classes. This problem has been observed for classes that are: - Compiled by older versions of scalac - Classes that are part of the Android distribution ... Caused by: java.lang.TypeNotPresentException: Type io.sentry .ITransaction not present ... Caused by: java.lang.ClassNotFoundException: io.sentry.ITransaction
ParaskP7
added
status: do not merge
Generated by 🚫 Danger |
📲 You can test the changes from this Pull Request in WooCommerce-Wear Android by scanning the QR code below to install the corresponding build.
|
|
📲 You can test the changes from this Pull Request in WooCommerce Android by scanning the QR code below to install the corresponding build.
|
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## trunk #12208 +/- ##
============================================
- Coverage 39.97% 39.84% -0.13%
+ Complexity 5579 5555 -24
============================================
Files 1214 1214
Lines 69276 69274 -2
Branches 9702 9702
============================================
- Hits 27690 27604 -86
- Misses 39004 39093 +89
+ Partials 2582 2577 -5 ☔ View full report in Codecov by Sentry. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
TODO
Testing information
TODO
RELEASE-NOTES.txtif necessary. Use the "[Internal]" label for non-user-facing changes.