Added DependencyInstaller to copy dependencies from s3 to maven local #318

VachaShah · 2021-08-25T17:41:39Z

Description

This PR introduces a DependencyInstaller, requirements in #276. This will be used by integration tests, performance tests and bwc tests when run in the pipeline. It covers the following scenarios:

Install maven dependencies from S3 to maven local
Download build dependencies from S3 to a custom local path (to be used for scenarios like this)
Clean up the downloaded contents

Note: I have merged the changes from #367 to use the S3 utility functions.

Issues Resolved

#276

Check List

Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

dblock

Lots of comments.

Still don't understand how this is used and where, not sure why we would commit files that aren't used.

dblock · 2021-08-25T20:52:11Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        self.version = version
+        self.arch = arch
+
+    def get_dependency_path(self):


Move it into the constructor as self.dependency_path =

dblock · 2021-08-25T20:52:51Z

bundle-workflow/src/test_workflow/dependency_installer.py

+    def get_dependency_path(self):
+        return f"org/opensearch/{self.dependency_name}/{self.version}/"
+
+    def get_maven_local_path(self):


Same, move into the constructor to expose as a property. If you want to abstract the method make it private like __get_aven_local_path(self).

dblock · 2021-08-25T20:53:23Z

bundle-workflow/src/test_workflow/dependency_installer.py

+
+    # TODO: This is currently a stubbed function which returns files from the current directory,
+    # to be replaced after it is implemented
+    def download_from_s3(self):


I think it should just be called download, more future proof

dblock · 2021-08-25T20:56:01Z

bundle-workflow/src/test_workflow/dependency_installer.py

+            if os.path.isfile(os.path.join(os.path.dirname(os.path.abspath(__file__)), file_name))
+        ]
+
+    def copy_to_maven_local(self, dependency_from_s3, maven_local_path):


This should probably me a private method. Does it do anything other than copy a file to ~/.m2?

dblock · 2021-08-25T20:56:20Z

bundle-workflow/src/test_workflow/dependency_installer.py

+            local_file_path = os.path.join(
+                os.path.dirname(os.path.abspath(__file__)), file_name
+            )
+            if os.path.isfile(local_file_path):


This ignores files that don't exist. Something seems off, when is this expected?

Added the else condition.

dblock · 2021-08-25T20:57:03Z

bundle-workflow/src/test_workflow/dependency_installer.py

+    def install(self):
+        # s3_path = f"/builds/{self.version}/{self.build_id}/{self.arch}/maven/{self.get_dependency_path()}"
+        maven_local_path = self.get_maven_local_path()
+        if not os.path.exists(maven_local_path):


This creates a maven path, but then copies in another function. Extract this whole copying business into a MavenLocal class that ensures that local paths exist and such.

Makes sense! Extracted the copying functionality to a separate class.

dblock · 2021-08-25T20:57:25Z

bundle-workflow/tests/test_workflow/test_dependency_installer.py

+
+
+class DependencyInstallerTests(unittest.TestCase):
+    dependency_installer = DependencyInstaller("7", "job-scheduler", "1.1.0.0", "arm64")


Put this into setUp(self), self.dependency_installer =

dblock · 2021-08-25T20:58:46Z

bundle-workflow/tests/test_workflow/test_dependency_installer.py

+from src.test_workflow.dependency_installer import DependencyInstaller
+
+
+class DependencyInstallerTests(unittest.TestCase):


If you extract Maven implementation then you don't need to be checking what gets copied and what is not, just that Maven().copy is called with the right parameters. Then you can test the Maven class separately in a custom temporary folder.

VachaShah · 2021-08-25T21:02:50Z

Lots of comments.

Still don't understand how this is used and where, not sure why we would commit files that aren't used.

This is to be used in PR #287 Line 47, sorry I should have mentioned it in the description.

dblock · 2021-08-25T21:04:17Z

Lots of comments.
Still don't understand how this is used and where, not sure why we would commit files that aren't used.

This is to be used in PR #287 Line 47, sorry I should have mentioned it in the description.

Sounds like we can wait for that to merge before this one?

VachaShah · 2021-08-25T21:10:35Z

Lots of comments.
Still don't understand how this is used and where, not sure why we would commit files that aren't used.

This is to be used in PR #287 Line 47, sorry I should have mentioned it in the description.

Sounds like we can wait for that to merge before this one?

Sure, then I can edit the usage in that PR to use the DependencyInstaller along with introducing DependencyInstaller in this PR. I will address the other comments till then.

peternied · 2021-08-26T19:48:50Z

@VachaShah I'm waiting on reviewing until after you've addressed db's comments, I'll be keeping an eye out for updates.

dblock

I added more comments. A copy from S3 to maven local should just do that.

dblock · 2021-08-26T21:53:51Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        file_handler.copy(self.download(), self.maven_local_path)
+
+
+class MavenLocalFileHandler:


Since it's an implementation detail of DependencyInstaller, make it a subclass of that? (tab it in to the right :))

dblock · 2021-08-26T21:54:25Z

bundle-workflow/src/test_workflow/dependency_installer.py

+
+    def copy(self, dependency_from_s3, maven_local_path):
+        if not os.path.exists(maven_local_path):
+            os.makedirs(maven_local_path)


Takes exist_ok=True to avoid the if above.

dblock · 2021-08-26T21:56:04Z

bundle-workflow/src/test_workflow/dependency_installer.py

+            if os.path.isfile(local_file_path):
+                shutil.copy(local_file_path, maven_local_path)
+            else:
+                raise ValueError(f"No file found at path: {local_file_path}")


I think if you use copyfile it will properly raise errors and you don't need any of the special if/handling of whether the file exists or not. Which begs the question, aren't we copying everything from dependency_from_s3? Then this is just a copy of a folder into another folder, which is a single copy call.

Yeah its just a single copy call, removed the check.

dblock · 2021-08-26T21:59:43Z

bundle-workflow/tests/test_workflow/helper/test_utils.py

+            if os.path.isfile(os.path.join(test_dir, file_name))
+        ]
+
+    def clean_maven_local_path(maven_local_path):


I am scared of code that does rmtree. Why would I need this?

In production I'd expect to have a clean maven local when executing on a new node. In test I definitely do not want the test job to blow away my local maven!

Makes sense, removed the clean up path, I had added it earlier to clean up between tests but its not needed anymore.

dblock · 2021-08-26T22:00:56Z

bundle-workflow/tests/test_workflow/test_dependency_installer.py

+
+    def test_install(self):
+        maven_local_path = self.dependency_installer.maven_local_path
+        TestUtils.clean_maven_local_path(maven_local_path)


See my comment above. I certainly don't expect running tests to delete all my files in ~/.m2!

setiah · 2021-08-27T21:39:50Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        self.version = version
+        self.arch = arch
+        self.dependency_path = f"org/opensearch/{self.dependency_name}/{self.version}/"
+        self.maven_local_path = os.path.join(


multiple concurrent builds/tests on jenkins can corrupt this path ~/.m2/repository and cause test failures. It should be a custom path based on execution id and should also have cleanup method exposed to remove the contents from the location after build execution.

@setiah Are we allowing jenkins test workers to be reused? AFAIK we were not allowing worker reuse to avoid handling problems like you are describing

i'm not sure what you mean by workers reuse. What I'm saying here is the workers would sync the dependencies in ~/.m2/repository of jenkins host. If there are two concurrent test jobs running, one for 1.1 another for 1.0, since they filesystem is same, they might end up overwriting each others dependencies.

Added build-id in the path to keep the executions independent.

setiah · 2021-08-27T23:08:38Z

bundle-workflow/src/test_workflow/dependency_installer.py

+    Provides functionality to copy the maven dependencies from S3 to maven local.
+    """
+
+    def __init__(self, build_id, dependency_name, version, arch):


instead of "build_id, version, arch" -> use build_manfiest as input, which has all these and more. Also, you can skip dependency_name in __init__. DependencyInstaller should provide a method to sync - all maven dependencies, or - a list of dependencies.

Do you mean that if the user passes in the list of dependencies, then get the files for those dependencies and if nothing is passed in, get the files for all dependencies?

Yeah 2 functions essentially -

for syncing a list of maven dependencies say for a specific plugin.

for syncing all maven dependencies from build/ folder.

can be treated optional for now. 1 is must have

Added functionality for 1.

peternied · 2021-08-30T16:47:41Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        self.version = version
+        self.arch = arch
+        self.dependency_path = f"org/opensearch/{self.dependency_name}/{self.version}/"
+        self.maven_local_path = os.path.join(


@setiah Are we allowing jenkins test workers to be reused? AFAIK we were not allowing worker reuse to avoid handling problems like you are describing

peternied · 2021-08-30T16:49:36Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        ]
+
+    def install(self):
+        # s3_path = f"/builds/{self.version}/{self.build_id}/{self.arch}/maven/{self.dependency_path()}"


Remove commented out code

This is for the TODO where the files will be downloaded from S3 once the functionality is ready in PR #316.

peternied · 2021-08-30T16:51:51Z

bundle-workflow/src/test_workflow/dependency_installer.py

+            for file_name in os.listdir(os.path.dirname(os.path.abspath(__file__)))
+            if os.path.isfile(
+                os.path.join(os.path.dirname(os.path.abspath(__file__)), file_name)


Indentation is off making this functionality unpredictable. Based on the comment this isn't final code, can we remove/reduce to a minimal case to keep the checked in code of a high bar?

The pipenv run black . reformatted it this way since the flake8 was failing with the if section being right indented. Definitely agree on the high bar! For the final code, the currently stubbed method for downloading the dependencies from s3 needs to be replaced once that functionality is merged and the tests corresponding to that stubbed method would change as well. I raised this PR early to get it out there and get feedback on the other parts and then can get a final review once the stubbed method is replaced.

peternied · 2021-08-30T17:00:23Z

bundle-workflow/tests/test_workflow/helper/test_utils.py

+
+
+class TestUtils:
+    def get_test_dependencies():


Could you declare this function inside of the test class, if you foresee its reuse elsewhere please add comments or additional configuration

Removed the common test utils and added it in the test class itself.

codecov-commenter · 2021-08-31T22:48:37Z

Codecov Report

Merging #318 (5649d4f) into main (49ee171) will increase coverage by 0.95%.
The diff coverage is 91.17%.

@@            Coverage Diff             @@
##             main     #318      +/-   ##
==========================================
+ Coverage   59.04%   60.00%   +0.95%     
==========================================
  Files          38       39       +1     
  Lines        1111     1145      +34     
==========================================
+ Hits          656      687      +31     
- Misses        455      458       +3

Impacted Files	Coverage Δ
...workflow/src/test_workflow/dependency_installer.py	`91.17% <91.17%> (ø)`

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 49ee171...5649d4f. Read the comment docs.

dblock

This is good! I think there are some should/must have's below but it's close.

dblock · 2021-08-31T22:55:40Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+            )
+
+
+class MavenLocalFileHandlerTests(unittest.TestCase):


Can you nest tests the same way as classes to match? NBD

dblock · 2021-08-31T22:56:54Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+        self.maven_local_file_handler = DependencyInstaller.MavenLocalFileHandler()
+
+    def test_copy(self):
+        maven_local_path = tempfile.mkdtemp()


Use with TemporaryFile as or you need to get rid of maven_local_path in teardownd.

dblock · 2021-08-31T22:57:18Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+        test_files = get_test_dependencies()
+        self.maven_local_file_handler.copy(test_files, maven_local_path)
+        self.assertCountEqual(test_files, os.listdir(maven_local_path))
+        self.assertListEqual(test_files, os.listdir(maven_local_path))


Probably needs a sort to work reliably.

dblock · 2021-08-31T22:57:56Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+        maven_local_path = tempfile.mkdtemp()
+        test_files = get_test_dependencies()
+        self.maven_local_file_handler.copy(test_files, maven_local_path)
+        self.assertCountEqual(test_files, os.listdir(maven_local_path))


Ensure test_files isn't blank or this test passes through.

dblock · 2021-08-31T22:58:43Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+        self.assertListEqual(test_files, os.listdir(maven_local_path))
+
+
+def get_test_dependencies():


Move this to be a static and private __get_test_dependencies() method in the test class instead of global.

dblock · 2021-08-31T22:59:39Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+                "../tests_assemble_workflow/data/opensearch-build-1.1.0.yml",
+            )
+        ) as f:
+            self.manifest = BuildManifest.from_file(f)


BuildManifest now has from_path, use that so you don't need to open.

dblock · 2021-09-01T12:59:45Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+        def test_copy(self):
+            with tempfile.TemporaryFile() as maven_local_path:
+                test_files = DependencyInstallerTests.__get_test_dependencies()
+                self.assertTrue(test_files)


This only makes sure "something" is returned. Check that len(), the number of files returned is > 0.

setiah · 2021-09-03T17:27:01Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        self.s3_maven_location = (
+            f"/builds/{self.version}/{self.build_id}/{self.arch}/maven"
+        )
+        self.s3_build_location = (
+            f"/builds/{self.version}/{self.build_id}/{self.arch}/plugins"
+        )


nit: this location provider logic is common across build and test workflows and should be read from a single place. Check if there's already a utility/or we can make it. It's okay if you create an issue and take it separately outside this PR.

bundle-workflow/Pipfile

dblock

I have some should have's, but won't hold the PR for them.

dblock · 2021-09-08T12:21:49Z

bundle-workflow/src/test_workflow/dependency_installer.py

+            elif os.path.isdir(local_path):
+                shutil.rmtree(local_path)
+        except OSError as e:
+            print(f"Failed to clean {local_path}. Reason: {e}")


Use logging.error.

dblock · 2021-09-08T12:24:04Z

bundle-workflow/src/test_workflow/dependency_installer.py

+            s3_path = f"{dependency}-{version}.zip"
+            s3_bucket.download_file(s3_path, custom_local_path)
+
+    def cleanup(self, local_path):


This deletes a folder, which is ... dangerous if we get it wrong. For maven dependencies I don't think we will ever want to be deleting them after downloading explicitly because they will be intermixed with other dependencies that may have been potentially dropped into ~m2. We already are doing cleanup in #403. I think you should delete it from the PR and close any issues that say otherwise.

@dblock - valid point. But I think we still need to cleanup dependencies which are synced as part of one test run, else we will unnecessarily bloat the local maven over a period of time. IMO we should switch back to the custom maven local path, based on a random tmp location, where we download dependencies and cleanup after the test run. This keeps the concurrent test runs isolated. We planned on doing it earlier but then switched to having docker provide that isolation by mounting on a randomized tmp path. I think we need it in the code as developers will run this code on local setup as well where docker mounting won't provide that sort of isolation. @VachaShah and I discussed about it yesterday.

bloat the local maven over a period of time.

Jenkins Agent nodes will only exist for a single workflow then are torn down, how fast is this bloat? Enough that we need to clean it out to complete a workflow?

For the record CI will be providing throw away containers with their own file system for these jobs.

Removed the cleanup logic.

dblock · 2021-09-08T12:24:57Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+
+    @patch("test_workflow.dependency_installer.S3Bucket")
+    def test_install_build_dependencies(self, mock_s3_bucket):
+        s3_bucket = mock_s3_bucket.return_value


= S3Bucket() - because it's mocked you will get the mock object

I tried it but it tries to call the actual constructor instead of mocked.

dblock · 2021-09-08T12:25:17Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+
+    @patch("test_workflow.dependency_installer.S3Bucket")
+    def test_install_maven_dependencies(self, mock_s3_bucket):
+        s3_bucket = mock_s3_bucket.return_value


= S3Bucket()

dblock · 2021-09-08T12:25:59Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+    @patch("test_workflow.dependency_installer.S3Bucket")
+    def test_install_maven_dependencies(self, mock_s3_bucket):
+        s3_bucket = mock_s3_bucket.return_value
+        self.dependency_installer = DependencyInstaller(self.manifest.build)


This is a test, don't assign anything to self. other than in setUp. Looks like dependency_installer is needed in every test, so move this into setUp.

dblock · 2021-09-08T12:26:46Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+        )
+        self.dependency_installer.install_maven_dependencies(dependencies)
+        self.assertEqual(s3_bucket.download_folder.call_count, 2)
+        maven_local_paths = []


Extract into a method in DependencyInstaller that gives you a maven local path for a dependency. You have similar code in DependencyInstaller itself.

dblock · 2021-09-08T12:28:20Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+            [
+                call(
+                    "opensearch-job-scheduler/1.1.0.0",
+                    maven_local_paths[0],


This will become something like dependency_installer.maven_local_path('opensearch-job-scheduler')

setiah · 2021-09-08T16:45:25Z

bundle-workflow/src/test_workflow/dependency_installer.py

+                os.path.expanduser("~"),
+                f".m2/repository/org/opensearch/{dependency}/{version}/",
+            )
+            s3_bucket.download_folder(s3_path, maven_local_path)


nit: this is s3_relative_path

It should be s3_maven_location for the dependency which is a relative path inside the bucket. An example - builds/1.1.0/24/x64/maven/org/opensearch/

setiah · 2021-09-08T16:59:32Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        :param dependency_dict: list of dependency names with version for which the maven artifacts need to be downloaded.
+        Example: {'opensearch-job-scheduler':'1.1.0.0', 'opensearch-core':'1.1.0'}
+        """
+        s3_bucket = S3Bucket(self.s3_maven_location)


This looks incorrect. S3Bucket needs to be initialized with bucket_name

setiah · 2021-09-08T17:00:21Z

bundle-workflow/src/test_workflow/dependency_installer.py

+        Example: {'opensearch-job-scheduler':'1.1.0.0'}
+        :param custom_local_path: the path where the downloaded dependencies need to copied.
+        """
+        s3_bucket = S3Bucket(self.s3_build_location)


same as above

setiah · 2021-09-08T17:11:10Z

bundle-workflow/tests/tests_test_workflow/test_dependency_installer.py

+        )
+        self.manifest = BuildManifest.from_path(self.manifest_filename)
+
+    @patch("test_workflow.dependency_installer.S3Bucket")


you can replace this with S3Bucket that's checked into main

This is the mocked version of the S3Bucket checked in main.

Signed-off-by: Vacha <[email protected]>

VachaShah · 2021-09-08T21:46:20Z

Hi @dblock and @setiah, I have addressed the comments. LMK if this looks good to be merged.

setiah

Thanks @VachaShah. LGTM!

VachaShah marked this pull request as draft August 25, 2021 17:43

VachaShah force-pushed the dependency-installer branch from a3d73d5 to 5a9e878 Compare August 25, 2021 18:17

VachaShah requested review from saratvemulapalli and dblock August 25, 2021 20:33

VachaShah marked this pull request as ready for review August 25, 2021 20:34

dblock requested changes Aug 25, 2021

View reviewed changes

peternied self-requested a review August 26, 2021 19:48

VachaShah force-pushed the dependency-installer branch from 72f7f3f to 40ad330 Compare August 26, 2021 21:51

dblock requested changes Aug 26, 2021

View reviewed changes

VachaShah force-pushed the dependency-installer branch from 40ad330 to 85739e2 Compare August 26, 2021 23:19

VachaShah requested a review from dblock August 26, 2021 23:38

dblock previously approved these changes Aug 27, 2021

View reviewed changes

setiah reviewed Aug 27, 2021

View reviewed changes

peternied reviewed Aug 30, 2021

View reviewed changes

VachaShah dismissed dblock’s stale review via 68d813b August 31, 2021 22:47

VachaShah force-pushed the dependency-installer branch from 85739e2 to 68d813b Compare August 31, 2021 22:47

VachaShah requested a review from setiah August 31, 2021 22:53

dblock requested changes Aug 31, 2021

View reviewed changes

VachaShah requested a review from dblock September 1, 2021 03:48

dblock previously approved these changes Sep 1, 2021

View reviewed changes

VachaShah dismissed dblock’s stale review via ce55c4b September 1, 2021 16:22

VachaShah force-pushed the dependency-installer branch from 01cb80e to ce55c4b Compare September 1, 2021 16:22

VachaShah requested a review from dblock September 1, 2021 16:22

dblock previously approved these changes Sep 1, 2021

View reviewed changes

setiah reviewed Sep 3, 2021

View reviewed changes

owaiskazi19 reviewed Sep 3, 2021

View reviewed changes

bundle-workflow/Pipfile Outdated Show resolved Hide resolved

VachaShah force-pushed the dependency-installer branch from a8a58c2 to ce55c4b Compare September 7, 2021 21:19

VachaShah marked this pull request as draft September 7, 2021 21:23

VachaShah force-pushed the dependency-installer branch from ce55c4b to 7553b81 Compare September 7, 2021 23:46

VachaShah marked this pull request as ready for review September 7, 2021 23:49

VachaShah requested review from setiah and dblock September 7, 2021 23:49

dblock previously approved these changes Sep 8, 2021

View reviewed changes

setiah reviewed Sep 8, 2021

View reviewed changes

VachaShah added 6 commits September 8, 2021 13:04

Added DependencyInstaller to copy dependencies from s3 to maven local

c2881e9

Signed-off-by: Vacha <[email protected]>

Addressing comments and refactoring

126702f

Signed-off-by: Vacha <[email protected]>

Refactoring DependencyInstaller

a6e16d1

Signed-off-by: Vacha <[email protected]>

Addressing comments for tests

a357fd9

Signed-off-by: Vacha <[email protected]>

Implementing functionality to install build dependencies and to cleanup

177774a

Signed-off-by: Vacha <[email protected]>

Addressing comments

5649d4f

Signed-off-by: Vacha <[email protected]>

VachaShah dismissed dblock’s stale review via 5649d4f September 8, 2021 20:57

VachaShah force-pushed the dependency-installer branch from 7553b81 to 5649d4f Compare September 8, 2021 20:57

Removing the cleanup logic

cb22248

Signed-off-by: Vacha <[email protected]>

VachaShah requested review from setiah and dblock September 8, 2021 21:43

dblock approved these changes Sep 8, 2021

View reviewed changes

peternied approved these changes Sep 9, 2021

View reviewed changes

setiah approved these changes Sep 9, 2021

View reviewed changes

VachaShah merged commit 6cf57c7 into opensearch-project:main Sep 9, 2021

VachaShah deleted the dependency-installer branch February 26, 2023 02:56



		class DependencyInstallerTests(unittest.TestCase):
		dependency_installer = DependencyInstaller("7", "job-scheduler", "1.1.0.0", "arm64")

		from src.test_workflow.dependency_installer import DependencyInstaller


		class DependencyInstallerTests(unittest.TestCase):

		file_handler.copy(self.download(), self.maven_local_path)


		class MavenLocalFileHandler:

		self.assertListEqual(test_files, os.listdir(maven_local_path))


		def get_test_dependencies():

Added DependencyInstaller to copy dependencies from s3 to maven local #318

Added DependencyInstaller to copy dependencies from s3 to maven local #318

Conversation

VachaShah commented Aug 25, 2021 • edited Loading

Description

Issues Resolved

Check List

dblock left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VachaShah Aug 26, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VachaShah commented Aug 25, 2021

dblock commented Aug 25, 2021

VachaShah commented Aug 25, 2021

peternied commented Aug 26, 2021

dblock left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

setiah Aug 27, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

setiah Aug 27, 2021 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

codecov-commenter commented Aug 31, 2021 • edited Loading

Codecov Report

dblock left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dblock left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

VachaShah commented Aug 25, 2021 •

edited

Loading

VachaShah Aug 26, 2021 •

edited

Loading

setiah Aug 27, 2021 •

edited

Loading

setiah Aug 27, 2021 •

edited

Loading

codecov-commenter commented Aug 31, 2021 •

edited

Loading