Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Custom Annotation to Parameterize Tests with more Complex Scenarios #20484

Merged
Merged
Show file tree
Hide file tree
Changes from 18 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
c9569a3
Initial design for supporting tests parameterized by ServiceVersion
alzimmermsft Apr 9, 2021
e541ed9
Merge branch 'master' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Apr 9, 2021
b8617f8
Fix linting and added some testing
alzimmermsft Apr 9, 2021
570d689
Merge branch 'master' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Apr 16, 2021
8e84319
Tighten requirements on what is allowed for the source supplier method
alzimmermsft Apr 16, 2021
66bb035
Merge branch 'master' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Apr 16, 2021
6dc6355
Add the ability to use all service versions with a flag
alzimmermsft Apr 16, 2021
b887f6e
Merge branch 'master' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Apr 22, 2021
f343ac5
Fix failing tests
alzimmermsft Apr 22, 2021
18d6969
Merge in upstream and fix conflicts
alzimmermsft Apr 27, 2021
1e0e436
Merge branch 'master' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft May 11, 2021
29382f2
Add class level annotation for service version testing, simplify test…
alzimmermsft Jun 18, 2021
d9e800a
Merge branch 'main' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Jun 18, 2021
98cf6d6
Update dependency version
alzimmermsft Jun 18, 2021
1619253
Remove references to azure-core-test implementation
alzimmermsft Jun 18, 2021
d0a9fbc
Merge branch 'main' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Jun 18, 2021
8218424
Restructuring and added new extension to prevent execution from faili…
alzimmermsft Jun 19, 2021
3c7b353
Fixed test pointing to wrong method
alzimmermsft Jun 21, 2021
2124e91
Merge branch 'main' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Jun 22, 2021
1c76f9c
Merge branch 'main' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Jun 22, 2021
5de7e74
Support maximumServiceVersions and add PlaybackOnly annotation
alzimmermsft Jun 22, 2021
544444f
Changed class annotation to support inheritence and use min and max f…
alzimmermsft Jun 23, 2021
12b74f6
Merge in main and fix linting issue
alzimmermsft Jul 16, 2021
d053b1e
Merge branch 'main' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Jul 20, 2021
74537d2
Update based on feedback
alzimmermsft Jul 20, 2021
e4bcc28
Merge branch 'main' into AzTest_SupportTestingMultipleApiVersions
alzimmermsft Jul 20, 2021
eef39fe
Revert changes made to azure-search-documents
alzimmermsft Jul 20, 2021
11121f2
Fix broken Javadocs
alzimmermsft Jul 20, 2021
557d538
Fix InvalidPathException from preventing DefaultLogger construction
alzimmermsft Jul 21, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,6 @@
import com.azure.core.exception.ClientAuthenticationException;
import com.azure.core.http.HttpClient;
import com.azure.core.test.TestMode;
import com.azure.core.test.implementation.ImplUtils;
import com.azure.core.util.Context;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
Expand Down Expand Up @@ -51,9 +50,9 @@ private ContainerRegistryClient getContainerRegistryClient(HttpClient httpClient

@BeforeEach
void beforeEach() {
TestUtils.importImage(ImplUtils.getTestMode(), HELLO_WORLD_REPOSITORY_NAME, Arrays.asList("latest", "v1", "v2", "v3", "v4"));
TestUtils.importImage(getTestMode(), HELLO_WORLD_REPOSITORY_NAME, Arrays.asList("latest", "v1", "v2", "v3", "v4"));
TestUtils.importImage(
ImplUtils.getTestMode(),
getTestMode(),
ALPINE_REPOSITORY_NAME,
Arrays.asList(
LATEST_TAG_NAME,
Expand All @@ -72,7 +71,7 @@ public void listRepositoryNames(HttpClient httpClient) {
StepVerifier.create(registryAsyncClient.listRepositoryNames())
.recordWith(ArrayList::new)
.thenConsumeWhile(x -> true)
.expectRecordedMatches(repositories -> validateRepositories(repositories))
.expectRecordedMatches(this::validateRepositories)
alzimmermsft marked this conversation as resolved.
Show resolved Hide resolved
.verifyComplete();

List<String> repositories = registryClient.listRepositoryNames().stream().collect(Collectors.toList());
Expand All @@ -88,11 +87,12 @@ public void listRepositoryNamesWithPageSize(HttpClient httpClient) {
StepVerifier.create(registryAsyncClient.listRepositoryNames().byPage(PAGESIZE_1))
.recordWith(ArrayList::new)
.thenConsumeWhile(x -> true)
.expectRecordedMatches(pagedResList -> validateRepositoriesByPage(pagedResList))
.expectRecordedMatches(this::validateRepositoriesByPage)
.verifyComplete();

ArrayList<String> repositories = new ArrayList<>();
registryClient.listRepositoryNames().iterableByPage(PAGESIZE_1).forEach(res -> res.getValue().forEach(repo -> repositories.add(repo)));
registryClient.listRepositoryNames().iterableByPage(PAGESIZE_1)
.forEach(res -> repositories.addAll(res.getValue()));
validateRepositories(repositories);
}

Expand All @@ -103,7 +103,8 @@ public void listRepositoryNamesWithInvalidPageSize(HttpClient httpClient) {
registryClient = getContainerRegistryClient(httpClient);

ArrayList<String> repositories = new ArrayList<>();
assertThrows(IllegalArgumentException.class, () -> registryClient.listRepositoryNames().iterableByPage(-1).forEach(res -> res.getValue().forEach(repo -> repositories.add(repo))));
assertThrows(IllegalArgumentException.class, () -> registryClient.listRepositoryNames().iterableByPage(-1)
.forEach(res -> repositories.addAll(res.getValue())));

StepVerifier.create(registryAsyncClient.listRepositoryNames().byPage(-1))
.verifyError(IllegalArgumentException.class);
Expand Down Expand Up @@ -132,7 +133,7 @@ public void getContainerRepository(HttpClient httpClient) {
ContainerRepositoryAsync repositoryAsync = registryAsyncClient.getRepository(HELLO_WORLD_REPOSITORY_NAME);
assertNotNull(repositoryAsync);
StepVerifier.create(repositoryAsync.getProperties())
.assertNext(res -> validateProperties(res))
.assertNext(this::validateProperties)
.verifyComplete();

ContainerRepository repository = registryClient.getRepository(HELLO_WORLD_REPOSITORY_NAME);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,8 +8,8 @@
import com.azure.core.http.netty.NettyAsyncHttpClientBuilder;
import com.azure.core.http.rest.Response;
import com.azure.core.test.TestMode;
import com.azure.core.test.implementation.ImplUtils;
import com.azure.core.util.Context;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.parallel.Execution;
Expand All @@ -18,15 +18,14 @@
import reactor.test.StepVerifier;

import java.time.Duration;
import java.util.Arrays;
import java.util.Collections;

import static com.azure.containers.containerregistry.TestUtils.HELLO_WORLD_REPOSITORY_NAME;
import static com.azure.containers.containerregistry.TestUtils.HELLO_WORLD_SEATTLE_REPOSITORY_NAME;
import static com.azure.containers.containerregistry.TestUtils.HTTP_STATUS_CODE_202;
import static com.azure.containers.containerregistry.TestUtils.SLEEP_TIME_IN_MILLISECONDS;
import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

@Execution(ExecutionMode.SAME_THREAD)
public class ContainerRegistryClientTest extends ContainerRegistryClientsTestBase {
Expand All @@ -47,7 +46,7 @@ private ContainerRegistryAsyncClient getContainerRegistryAsyncClient(HttpClient

@BeforeEach
void beforeEach() {
TestUtils.importImage(ImplUtils.getTestMode(), repositoryName, Arrays.asList("latest"));
TestUtils.importImage(getTestMode(), repositoryName, Collections.singletonList("latest"));
if (getTestMode() == TestMode.PLAYBACK) {
httpClient = interceptorManager.getPlaybackClient();
} else {
Expand All @@ -68,12 +67,12 @@ public void deleteRepositoryByRegistryWithResponseAsyncClient() {
.then(registryAsyncClient.getRepository(repositoryName).getProperties())
.flatMap(res -> Mono.just(false))
.onErrorResume(res -> registryAsyncClient.getRepository(repositoryName)
.delete()
.then(Mono.just(true))
.delete()
.then(Mono.just(true))
.onErrorResume(err -> Mono.just(false)));

StepVerifier.create(deleteRepositoryTest)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand All @@ -89,7 +88,7 @@ public void deleteRepositoryByRegistryAsyncClient() {
.onErrorResume(err -> Mono.just(false)));

StepVerifier.create(deleteRepositoryTest)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand All @@ -105,7 +104,7 @@ public void deleteRepositoryWithResponseAsyncClient() {
.onErrorResume(err -> Mono.just(false)));

StepVerifier.create(deleteRepositoryTest)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand All @@ -121,7 +120,7 @@ public void deleteRepositoryAsyncClient() {
.onErrorResume(err -> Mono.just(false)));

StepVerifier.create(deleteRepositoryTest)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,6 @@
package com.azure.containers.containerregistry;

import com.azure.core.http.HttpClient;
import com.azure.core.test.implementation.ImplUtils;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.MethodSource;
Expand All @@ -22,14 +21,14 @@
public class ContainerRepositoryAnonymousAccessTests extends ContainerRegistryClientsTestBase {
@BeforeEach
void beforeEach() {
TestUtils.importImageAsync(ImplUtils.getTestMode(), ANONYMOUS_REGISTRY_NAME, HELLO_WORLD_REPOSITORY_NAME, Arrays.asList("latest", "v1", "v2", "v3", "v4")).block();
TestUtils.importImageAsync(getTestMode(), ANONYMOUS_REGISTRY_NAME, HELLO_WORLD_REPOSITORY_NAME, Arrays.asList("latest", "v1", "v2", "v3", "v4")).block();
}

@ParameterizedTest(name = DISPLAY_NAME_WITH_ARGUMENTS)
@MethodSource("getHttpClients")
public void listAnonymousRepositories(HttpClient httpClient) {
ContainerRegistryClient client = getContainerRegistryBuilder(httpClient, null, ANONYMOUS_REGISTRY_ENDPOINT).buildClient();
List<String> repositories = client.listRepositoryNames().stream().collect(Collectors.toList());
assertTrue(repositories.stream().anyMatch(a -> HELLO_WORLD_REPOSITORY_NAME.equals(a)));
assertTrue(repositories.stream().anyMatch(HELLO_WORLD_REPOSITORY_NAME::equals));
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
import com.azure.core.http.HttpClient;
import com.azure.core.http.netty.NettyAsyncHttpClientBuilder;
import com.azure.core.test.TestMode;
import com.azure.core.test.implementation.ImplUtils;
import com.azure.core.util.Context;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.BeforeEach;
Expand Down Expand Up @@ -41,7 +40,7 @@ public class ContainerRepositoryAsyncIntegrationTests extends ContainerRegistryC

@BeforeEach
void beforeEach() {
TestUtils.importImage(ImplUtils.getTestMode(), HELLO_WORLD_REPOSITORY_NAME, Arrays.asList("latest", "v1", "v2", "v3", "v4"));
TestUtils.importImage(getTestMode(), HELLO_WORLD_REPOSITORY_NAME, Arrays.asList("latest", "v1", "v2", "v3", "v4"));
}

@AfterEach
Expand Down Expand Up @@ -89,15 +88,11 @@ public void getPropertiesWithResponse(HttpClient httpClient) {
client = getContainerRepository(httpClient);

StepVerifier.create(asyncClient.getPropertiesWithResponse())
.assertNext(res -> {
validateProperties(res);
})
.assertNext(this::validateProperties)
.verifyComplete();

StepVerifier.create(asyncClient.getProperties())
.assertNext(res -> {
validateProperties(res);
})
.assertNext(this::validateProperties)
.verifyComplete();

validateProperties(client.getProperties());
Expand Down Expand Up @@ -148,7 +143,7 @@ public void listArtifactsWithPageSize(HttpClient httpClient) {
StepVerifier.create(asyncClient.listManifestProperties().byPage(PAGESIZE_2))
.recordWith(ArrayList::new)
.thenConsumeWhile(x -> true)
.expectRecordedMatches(pagedResList -> validateListArtifactsByPage(pagedResList)).verifyComplete();
.expectRecordedMatches(this::validateListArtifactsByPage).verifyComplete();

validateListArtifactsByPage(client.listManifestProperties().streamByPage(PAGESIZE_2).collect(Collectors.toList()));
}
Expand Down Expand Up @@ -190,7 +185,7 @@ public void listArtifactsWithPageSizeNoOrderBy(HttpClient httpClient) {
StepVerifier.create(asyncClient.listManifestProperties(ArtifactManifestOrderBy.NONE).byPage(PAGESIZE_2))
.recordWith(ArrayList::new)
.thenConsumeWhile(x -> true)
.expectRecordedMatches(pagedResList -> validateListArtifactsByPage(pagedResList))
.expectRecordedMatches(this::validateListArtifactsByPage)
.verifyComplete();

validateListArtifactsByPage(client.listManifestProperties(ArtifactManifestOrderBy.NONE, Context.NONE).streamByPage(PAGESIZE_2).collect(Collectors.toList()));
Expand All @@ -209,7 +204,7 @@ public void updateProperties(HttpClient httpClient) {
.verifyComplete();

StepVerifier.create(asyncClient.updateProperties(repoWriteableProperties))
.assertNext(res -> validateRepoContentProperties(res))
.assertNext(this::validateRepoContentProperties)
.verifyComplete();

validateRepoContentProperties(client.updateProperties(repoWriteableProperties));
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@
import com.azure.core.http.HttpClient;
import com.azure.core.http.rest.Response;
import com.azure.core.test.TestMode;
import com.azure.core.test.implementation.ImplUtils;
import com.azure.core.util.Context;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.params.ParameterizedTest;
Expand Down Expand Up @@ -47,7 +46,7 @@ public class RegistryArtifactAsyncIntegrationTests extends ContainerRegistryClie
@BeforeEach
void beforeEach() {
TestUtils.importImage(
ImplUtils.getTestMode(),
getTestMode(),
HELLO_WORLD_REPOSITORY_NAME,
Arrays.asList(
LATEST_TAG_NAME,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@
import com.azure.core.test.TestMode;
import com.azure.core.util.Context;
import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.parallel.Execution;
Expand All @@ -27,7 +28,6 @@
import static com.azure.containers.containerregistry.TestUtils.V3_TAG_NAME;
import static com.azure.containers.containerregistry.TestUtils.V4_TAG_NAME;
import static org.junit.jupiter.api.Assertions.assertThrows;
import static org.junit.jupiter.api.Assertions.assertTrue;

@Execution(ExecutionMode.SAME_THREAD)
public class RegistryArtifactTests extends ContainerRegistryClientsTestBase {
Expand Down Expand Up @@ -103,7 +103,7 @@ public void delete() {
.onErrorResume(err -> Mono.just(false))));

StepVerifier.create(delete)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand All @@ -122,7 +122,7 @@ public void deleteWithResponse() {
.onErrorResume(err -> Mono.just(false))));

StepVerifier.create(delete)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand All @@ -141,7 +141,7 @@ public void deleteTag() {
.onErrorResume(err -> Mono.just(false))));

StepVerifier.create(delete)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand All @@ -159,7 +159,7 @@ public void deleteTagWithResponse() {
.onErrorResume(err -> Mono.just(false))));

StepVerifier.create(delete)
.assertNext(res -> assertTrue(res))
.assertNext(Assertions::assertTrue)
.verifyComplete();
}

Expand Down Expand Up @@ -195,7 +195,7 @@ public void updateManifestProperties() {
digest = V1_TAG_NAME;

StepVerifier.create(asyncClient.updateManifestProperties(manifestWriteableProperties))
.assertNext(res -> validateManifestContentProperties(res))
.assertNext(this::validateManifestContentProperties)
.verifyComplete();

StepVerifier.create(asyncClient.updateManifestPropertiesWithResponse(manifestWriteableProperties))
Expand All @@ -217,7 +217,7 @@ public void updateTagProperties() {
reupdateTagProperties = true;

StepVerifier.create(asyncClient.updateTagProperties(V2_TAG_NAME, tagWriteableProperties))
.assertNext(res -> validateTagContentProperties(res))
.assertNext(this::validateTagContentProperties)
.verifyComplete();

StepVerifier.create(asyncClient.updateTagPropertiesWithResponse(V2_TAG_NAME, tagWriteableProperties))
Expand Down
19 changes: 14 additions & 5 deletions sdk/core/azure-core-test/pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -51,16 +51,17 @@
<artifactId>junit-jupiter-api</artifactId>
<version>5.7.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-api;external_dependency} -->
</dependency>

<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-engine</artifactId>
<version>5.7.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-engine;external_dependency} -->
<scope>test</scope>
<artifactId>junit-jupiter-params</artifactId>
<version>5.7.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-params;external_dependency} -->
</dependency>

<dependency>
<groupId>org.junit.jupiter</groupId>
<artifactId>junit-jupiter-params</artifactId>
<version>5.7.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-params;external_dependency} -->
<artifactId>junit-jupiter-engine</artifactId>
<version>5.7.2</version> <!-- {x-version-update;org.junit.jupiter:junit-jupiter-engine;external_dependency} -->
<scope>test</scope>
alzimmermsft marked this conversation as resolved.
Show resolved Hide resolved
</dependency>
<dependency>
Expand All @@ -69,6 +70,12 @@
<version>2.24.1</version> <!-- {x-version-update;com.github.tomakehurst:wiremock-standalone;external_dependency} -->
<scope>test</scope>
</dependency>
<dependency>
<groupId>org.mockito</groupId>
<artifactId>mockito-core</artifactId>
<version>3.9.0</version> <!-- {x-version-update;org.mockito:mockito-core;external_dependency} -->
<scope>test</scope>
</dependency>
alzimmermsft marked this conversation as resolved.
Show resolved Hide resolved
</dependencies>

<build>
Expand Down Expand Up @@ -110,6 +117,7 @@
<include>io.projectreactor:reactor-test:[3.4.6]</include> <!-- {x-include-update;io.projectreactor:reactor-test;external_dependency} -->
<!-- special allowance for azure-core-test as it is not a shipping library: -->
<include>org.junit.jupiter:junit-jupiter-api:[5.7.2]</include> <!-- {x-include-update;org.junit.jupiter:junit-jupiter-api;external_dependency} -->
<include>org.junit.jupiter:junit-jupiter-params:[5.7.2]</include> <!-- {x-include-update;org.junit.jupiter:junit-jupiter-params;external_dependency} -->
alzimmermsft marked this conversation as resolved.
Show resolved Hide resolved
</includes>
</bannedDependencies>
</rules>
Expand Down Expand Up @@ -148,6 +156,7 @@

--add-reads com.azure.core.test=ALL-UNNAMED
--add-opens com.azure.core.test/com.azure.core.test.models=ALL-UNNAMED
--add-opens com.azure.core.test/com.azure.core.test.implementation=org.junit.platform.commons
</argLine>
</configuration>
</plugin>
Expand Down
Loading