Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix part of #5343: Introduce Utilities to execute code coverage for a specific bazel test target #5423

Merged
merged 42 commits into from
Jun 24, 2024
Merged
Show file tree
Hide file tree
Changes from 38 commits
Commits
Show all changes
42 commits
Select commit Hold shift + click to select a range
6ec35a7
Fix lint new line and kdoc checks
Rd4dev Jun 4, 2024
1ec224a
Execute Bazel Command to run coverage on the test target
Rd4dev Jun 5, 2024
ee92168
Parse the coverage execution result to acquire the generated coverage…
Rd4dev Jun 5, 2024
075c77c
Implemented functionality to TestBazelWorkspace utility to add source…
Rd4dev Jun 7, 2024
d0519d6
Sample Test data and Listing all contents
Rd4dev Jun 9, 2024
b15b7bb
Added Test for BazelClient - for sample target and non target executions
Rd4dev Jun 9, 2024
f441d11
Added Coverage Runner functionality tests
Rd4dev Jun 10, 2024
cc86309
Added tests for the RunCoverageForTestTarget utility
Rd4dev Jun 11, 2024
4a6073b
Replaced Java Test Sources with Kotlin Sources
Rd4dev Jun 13, 2024
13534a1
Fix Static Checks / Lint warnings
Rd4dev Jun 13, 2024
fc52d82
Fix Buildifier Lint checks
Rd4dev Jun 13, 2024
0226baa
Added tests for TestBazelWorkspaceTest to check addition of source an…
Rd4dev Jun 13, 2024
2fe7485
Fix Lint warnings with TestBazelWorkspaceTest
Rd4dev Jun 13, 2024
bd683b5
Fix Regex Pattern checks
Rd4dev Jun 13, 2024
d410d17
Prototyped parsing and reading the extracted file data as byte array
Rd4dev Jun 15, 2024
ece147a
Updated the runCoverageForTestTarget to parse the data file and read …
Rd4dev Jun 15, 2024
b101ec8
Updated Bazel Client Tests with updated execution to return byte array
Rd4dev Jun 15, 2024
5db0650
Updated BazelClientTest for ParseCoverageDataFile and readDatFileAsBi…
Rd4dev Jun 17, 2024
87be114
Updated CoverageRunnerTest to check with updated getCoverage result r…
Rd4dev Jun 17, 2024
456e81b
Updated RunCoverageForTestTargetTest to check the returned coverage data
Rd4dev Jun 17, 2024
2c65524
Update kdoc to match the function paramaters
Rd4dev Jun 17, 2024
62f9e1d
Removing suspend while keeping the async operations
Rd4dev Jun 17, 2024
5e6d2ef
Updated RunCoverageForTestTarget to take in custom process timeout as…
Rd4dev Jun 17, 2024
31b6401
Initialized bazelClient using lazy field
Rd4dev Jun 17, 2024
81d6855
Adjusted visibility and removed unnecessary documentation for private…
Rd4dev Jun 17, 2024
6cb1da7
Revoking visibility of bazelClient functions as that is called outsid…
Rd4dev Jun 17, 2024
b41fb01
Fix Lint Check issues
Rd4dev Jun 17, 2024
e66ac12
Merge branch 'develop' of https://github.com/Rd4dev/oppia-android int…
Rd4dev Jun 17, 2024
c646840
Merge branch 'develop' of https://github.com/Rd4dev/oppia-android int…
Rd4dev Jun 18, 2024
248a20a
Merge branch 'develop' of https://github.com/Rd4dev/oppia-android int…
Rd4dev Jun 19, 2024
ee078e2
Refactored BazelClient coverage execution to return data as list of s…
Rd4dev Jun 21, 2024
f15e087
Fix Lint checks
Rd4dev Jun 21, 2024
506916d
Removed ending periods in KDoc Strings
Rd4dev Jun 21, 2024
c68487d
Merge branch 'develop' of https://github.com/Rd4dev/oppia-android int…
Rd4dev Jun 21, 2024
ba1c66a
Fix KDoc String punctuations, more clearer variable names and simplif…
Rd4dev Jun 22, 2024
438476c
Removed unused Paths import
Rd4dev Jun 22, 2024
ac30ca7
Merge branch 'develop' of https://github.com/Rd4dev/oppia-android int…
Rd4dev Jun 22, 2024
c7c2f2f
Fixed indentation and line spaces for better readability
Rd4dev Jun 22, 2024
c834aa7
Removed the RunCoverageForTestTarget as the functionality will be re-…
Rd4dev Jun 23, 2024
c654eed
Removed the RunCoverageForTestTarget Build inclusions
Rd4dev Jun 23, 2024
73f553d
Merge branch 'develop' into code_coverage_bazel_command_execution
BenHenning Jun 24, 2024
0f63469
Merge branch 'develop' into code_coverage_bazel_command_execution
BenHenning Jun 24, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions scripts/BUILD.bazel
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,15 @@ kt_jvm_binary(
],
)

kt_jvm_binary(
name = "run_coverage_for_test_target",
testonly = True,
main_class = "org.oppia.android.scripts.coverage.RunCoverageForTestTargetKt",
runtime_deps = [
"//scripts/src/java/org/oppia/android/scripts/coverage:run_coverage_for_test_target_lib",
],
)

# Note that this is intentionally not test-only since it's used by the app build pipeline. Also,
# this apparently needs to be a java_binary to set up runfiles correctly when executed within a
# Starlark rule as a tool.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,40 @@ class BazelClient(private val rootDirectory: File, private val commandExecutor:
return correctedTargets
}

/**
* Runs code coverage for the specified Bazel test target.
*
* Null return typically occurs when the coverage command fails to generate the 'coverage.dat' file
* This can happen due to: Test failures or misconfigurations that prevent the coverage data
* from being generated properly.
*
* @param bazelTestTarget Bazel test target for which code coverage will be run
* @return the generated coverage data as a list of strings
* or null if the coverage data file could not be parsed
*/
fun runCoverageForTestTarget(bazelTestTarget: String): List<String>? {
val coverageCommandOutputLines = executeBazelCommand(
"coverage",
bazelTestTarget
)
return parseCoverageDataFilePath(coverageCommandOutputLines)?.let { path ->
File(path).readLines()
}
}

private fun parseCoverageDataFilePath(coverageCommandOutputLines: List<String>): String? {
val regex = ".*coverage\\.dat$".toRegex()
for (line in coverageCommandOutputLines) {
val match = regex.find(line)
val extractedPath = match?.value?.substringAfterLast(",")?.trim()
if (extractedPath != null) {
println("Parsed Coverage Data File: $extractedPath")
return extractedPath
}
}
return null
}

/**
* Returns the results of a query command with a potentially large list of [values] that will be
* split up into multiple commands to avoid overflow the system's maximum argument limit.
Expand Down
32 changes: 32 additions & 0 deletions scripts/src/java/org/oppia/android/scripts/coverage/BUILD.bazel
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
"""
Libraries corresponding to developer scripts that obtain coverage data for test targets.
"""

load("@io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_library")

kt_jvm_library(
name = "run_coverage_for_test_target_lib",
testonly = True,
srcs = [
"RunCoverageForTestTarget.kt",
],
visibility = ["//scripts:oppia_script_binary_visibility"],
deps = [
"//scripts/src/java/org/oppia/android/scripts/common:bazel_client",
"//scripts/src/java/org/oppia/android/scripts/common:git_client",
"//scripts/src/java/org/oppia/android/scripts/coverage:coverage_runner",
],
)

kt_jvm_library(
name = "coverage_runner",
testonly = True,
srcs = [
"CoverageRunner.kt",
],
visibility = ["//scripts:oppia_script_binary_visibility"],
deps = [
"//scripts/src/java/org/oppia/android/scripts/common:bazel_client",
"//scripts/src/java/org/oppia/android/scripts/common:git_client",
],
)
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
package org.oppia.android.scripts.coverage

import kotlinx.coroutines.CoroutineScope
import kotlinx.coroutines.Deferred
import kotlinx.coroutines.async
import org.oppia.android.scripts.common.BazelClient
import org.oppia.android.scripts.common.CommandExecutor
import org.oppia.android.scripts.common.ScriptBackgroundCoroutineDispatcher
import java.io.File

/**
* Class responsible for running coverage analysis asynchronously.
*
* @param repoRoot the root directory of the repository
* @param scriptBgDispatcher the [ScriptBackgroundCoroutineDispatcher] to be used for running the coverage command
* @param commandExecutor executes the specified command in the specified working directory
*/
class CoverageRunner(
private val repoRoot: File,
private val scriptBgDispatcher: ScriptBackgroundCoroutineDispatcher,
private val commandExecutor: CommandExecutor
) {
private val bazelClient by lazy { BazelClient(repoRoot, commandExecutor) }

/**
* Runs coverage analysis asynchronously for the Bazel test target.
*
* @param bazelTestTarget Bazel test target to analyze coverage
* @return a deferred value that contains the coverage data
*/
fun runWithCoverageAsync(
bazelTestTarget: String
): Deferred<List<String>?> {
return CoroutineScope(scriptBgDispatcher).async {
retrieveCoverageResult(bazelTestTarget)
}
}

private fun retrieveCoverageResult(
bazelTestTarget: String
): List<String>? {
return bazelClient.runCoverageForTestTarget(bazelTestTarget)
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
package org.oppia.android.scripts.coverage

import kotlinx.coroutines.runBlocking
import org.oppia.android.scripts.common.CommandExecutor
import org.oppia.android.scripts.common.CommandExecutorImpl
import org.oppia.android.scripts.common.ScriptBackgroundCoroutineDispatcher
import java.io.File
import java.util.concurrent.TimeUnit

/**
* Entry point function for running coverage analysis for a single test target.
*
* Usage:
* bazel run //scripts:run_coverage_for_test_target -- <path_to_root> <//:test_targetname>
*
* Arguments:
* - path_to_root: directory path to the root of the Oppia Android repository
* - test_targetname: bazel target name of the test
*
* Example:
* bazel run //scripts:run_coverage_for_test_target -- $(pwd)
* //utility/src/test/java/org/oppia/android/util/parser/math:MathModelTest
* Example with custom process timeout:
* bazel run //scripts:run_coverage_for_test_target -- $(pwd)
* //utility/src/test/java/org/oppia/android/util/parser/math:MathModelTest processTimeout=10
*
*/
fun main(vararg args: String) {
val repoRoot = File(args[0]).absoluteFile.normalize()
val targetPath = args[1]

ScriptBackgroundCoroutineDispatcher().use { scriptBgDispatcher ->
val processTimeout: Long = args.find { it.startsWith("processTimeout=") }
?.substringAfter("=")
?.toLongOrNull() ?: 5

val commandExecutor: CommandExecutor = CommandExecutorImpl(
scriptBgDispatcher, processTimeout = processTimeout, processTimeoutUnit = TimeUnit.MINUTES
)

RunCoverageForTestTarget(
repoRoot,
targetPath,
commandExecutor,
scriptBgDispatcher
).runCoverage()
}
}

/**
* Class responsible for analyzing target files for coverage and generating reports.
*
* @param repoRoot the root directory of the repository
* @param targetPath Bazel test target to analyze coverage
* @param commandExecutor executes the specified command in the specified working directory
* @param scriptBgDispatcher the [ScriptBackgroundCoroutineDispatcher] to be used for running the coverage command
*/
class RunCoverageForTestTarget(
private val repoRoot: File,
private val targetPath: String,
private val commandExecutor: CommandExecutor,
private val scriptBgDispatcher: ScriptBackgroundCoroutineDispatcher
Rd4dev marked this conversation as resolved.
Show resolved Hide resolved
) {

/**
* Analyzes target file for coverage, generates chosen reports accordingly.
*/
fun runCoverage(): List<String>? {
return runWithCoverageAnalysis()
}

/**
* Runs coverage analysis on the specified target file asynchronously.
*
* @return the generated coverage data
*/
fun runWithCoverageAnalysis(): List<String>? {
return runBlocking {
val result =
CoverageRunner(repoRoot, scriptBgDispatcher, commandExecutor)
.runWithCoverageAsync(targetPath)
.await()
result
}
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,141 @@ class TestBazelWorkspace(private val temporaryRootFolder: TemporaryFolder) {
assertThat(bazelRcFile.exists()).isTrue()
}

/**
* Adds a source file and test file with the specified name and content,
* and updates the corresponding build configuration.
*
* @param filename the name of the source file (without the .kt extension)
* @param sourceContent the content of the source file
* @param testContent the content of the test file
* @param subpackage the subpackage under which the source and test files should be added
*/
fun addSourceAndTestFileWithContent(
filename: String,
sourceContent: String,
testContent: String,
subpackage: String
) {
val sourceSubpackage = "$subpackage/main/java/com/example"
addSourceContentAndBuildFile(
filename,
sourceContent,
sourceSubpackage
)

val testSubpackage = "$subpackage/test/java/com/example"
val testFileName = "${filename}Test"
addTestContentAndBuildFile(
filename,
testFileName,
testContent,
sourceSubpackage,
testSubpackage
)
}

/**
* Adds a source file with the specified name and content to the specified subpackage,
* and updates the corresponding build configuration.
*
* @param filename the name of the source file (without the .kt extension)
* @param sourceContent the content of the source file
* @param sourceSubpackage the subpackage under which the source file should be added
* @return the target name of the added source file
*/
fun addSourceContentAndBuildFile(
filename: String,
sourceContent: String,
sourceSubpackage: String
) {
initEmptyWorkspace()
ensureWorkspaceIsConfiguredForKotlin()
setUpWorkspaceForRulesJvmExternal(
listOf("junit:junit:4.12")
)

// Create the source subpackage directory if it doesn't exist
if (!File(temporaryRootFolder.root, sourceSubpackage.replace(".", "/")).exists()) {
temporaryRootFolder.newFolder(*(sourceSubpackage.split(".")).toTypedArray())
}

// Create the source file
val sourceFile = temporaryRootFolder.newFile(
"${sourceSubpackage.replace(".", "/")}/$filename.kt"
)
sourceFile.writeText(sourceContent)

// Create or update the BUILD file for the source file
val buildFileRelativePath = "${sourceSubpackage.replace(".", "/")}/BUILD.bazel"
val buildFile = File(temporaryRootFolder.root, buildFileRelativePath)
if (!buildFile.exists()) {
temporaryRootFolder.newFile(buildFileRelativePath)
}
prepareBuildFileForLibraries(buildFile)

buildFile.appendText(
"""
kt_jvm_library(
name = "${filename.lowercase()}",
srcs = ["$filename.kt"],
visibility = ["//visibility:public"]
)
""".trimIndent() + "\n"
)
}

/**
* Adds a test file with the specified name and content to the specified subpackage,
* and updates the corresponding build configuration.
*
* @param filename the name of the source file (without the .kt extension)
* @param testName the name of the test file (without the .kt extension)
* @param testContent the content of the test file
* @param testSubpackage the subpackage for the test file
*/
fun addTestContentAndBuildFile(
filename: String,
testName: String,
testContent: String,
sourceSubpackage: String,
testSubpackage: String
) {
initEmptyWorkspace()

// Create the test subpackage directory for the test file if it doesn't exist
if (!File(temporaryRootFolder.root, testSubpackage.replace(".", "/")).exists()) {
temporaryRootFolder.newFolder(*(testSubpackage.split(".")).toTypedArray())
}

// Create the test file
val testFile = temporaryRootFolder.newFile("${testSubpackage.replace(".", "/")}/$testName.kt")
testFile.writeText(testContent)

// Create or update the BUILD file for the test file
val testBuildFileRelativePath = "${testSubpackage.replace(".", "/")}/BUILD.bazel"
val testBuildFile = File(temporaryRootFolder.root, testBuildFileRelativePath)
if (!testBuildFile.exists()) {
temporaryRootFolder.newFile(testBuildFileRelativePath)
}
prepareBuildFileForTests(testBuildFile)

// Add the test file to the BUILD file with appropriate dependencies
testBuildFile.appendText(
"""
kt_jvm_test(
name = "test",
srcs = ["$testName.kt"],
deps = [
"//$sourceSubpackage:${filename.lowercase()}",
"@maven//:junit_junit",
],
visibility = ["//visibility:public"],
test_class = "com.example.$testName",
)
""".trimIndent() + "\n"
)
}

/**
* Generates and adds a new kt_jvm_test target with the target name [testName] and test file
* [testFile]. This can be used to add multiple tests to the same build file, and will
Expand Down
Loading
Loading