Adds performance tests for Event Hubs. #21822

conniey · 2021-05-25T17:48:47Z

Adds performance tests for our legacy client and our current client.

Related #20841
Fixes #20791

…ers.

YijunXieMS

LGTM. Left some comments but not required for the 1st usable version.

YijunXieMS · 2021-05-27T06:39:01Z

...eventhubs-track1-perf/src/main/java/com/azure/messaging/eventhubs/perf/EventHubsOptions.java

+        @Override
+        public TransportType convert(String s) {
+            if (s == null) {
+                throw new ParameterException(getErrorString("null", "AmqpTransportType"));


How about default value AMQP?

The default is to use AmqpTransportType.AMQP. It only tries to parse this if you pass in the parameter --transportType. And if you didn't pass a value, I think that should be a user error because it's not what you expected.

YijunXieMS · 2021-05-27T07:34:10Z

...enthubs-track2-perf/src/main/java/com/azure/messaging/eventhubs/perf/EventProcessorTest.java

+    }
+
+    private String getResults(int index, EventsCounter eventsCounter) {
+        final double seconds = eventsCounter.elapsedTime() * 0.000000001;


/1_000_000_000.0 might be easier to count the number of zeros

Ahh yeah. I always thought multiplication was faster for processors than division. But maybe this is moot now.

...enthubs-track2-perf/src/main/java/com/azure/messaging/eventhubs/perf/EventProcessorTest.java

...eventhubs-track2-perf/src/main/java/com/azure/messaging/eventhubs/perf/EventHubsOptions.java

YijunXieMS · 2021-05-27T07:57:40Z

...bs-track2-perf/src/main/java/com/azure/messaging/eventhubs/perf/EventHubsReceiveOptions.java

+ * @see ReceiveEventsTest
+ * @see EventProcessorTest
+ */
+public class EventHubsReceiveOptions extends EventHubsOptions {


We may add prefetch in the options so we can test it. It's one of the most important factors for receiving (including event processor) performance.
But it requires us to change the SDK code to use the prefetch for request size instead of just the first request.

YijunXieMS · 2021-05-27T08:10:54Z

...hubs-track1-perf/src/main/java/com/azure/messaging/eventhubs/perf/EventProcessorOptions.java

+ * @see EventProcessorTest
+ */
+public class EventProcessorOptions extends EventHubsReceiveOptions {
+    @Parameter(names = {"-scs", "--storageConnectionString"}, description = "Connection string for Storage account.",


Do we allow a test that use sample checkpoint store without a storage container?

The problem is that if we want something comparable, we need to do the same implementation for Track 1... I had tried it for a few hours but it was too confusing to continue.

YijunXieMS · 2021-05-27T08:12:22Z

...hubs-track1-perf/src/main/java/com/azure/messaging/eventhubs/perf/EventProcessorOptions.java

+    private String storageEndpoint;
+
+    @Parameter(names = {"-e", "--eventsToSend"}, description = "Number of events to send per partition.")
+    private int eventsToSend = 100000;


May add event payload size as a parameter

Base class PerfStressOptions expose a .getSize() (-s, --size) parameter.

We use this in ServiceTest line 62.

…into performance-tests

Machinelearningservices microsoft.machine learning services 2022 12 01 preview (Azure#21761) * Adds base for updating Microsoft.MachineLearningServices from version preview/2022-10-01-preview to version 2022-12-01-preview * Updates readme * Updates API version in new specs and examples * Add Dec API Registries Swagger (Azure#21419) * add december registries swagger + examples * add status code 202 in examples * fix 202 examples * fixes * fixes * fix * add 202 back in for put/patch Co-authored-by: Komal Yadav <[email protected]> * remove location (Azure#21430) Co-authored-by: Komal Yadav <[email protected]> * remove readonly flag on schedules property for CI (Azure#21653) Co-authored-by: Naman Agarwal <[email protected]> * add missing workspace properties (Azure#21725) * December preview updating mfe.json specs (Azure#21510) * December preview updating mfe.json specs * MFE Dec 2022 Preview API - Adding logbase * MFE 2022-12-01-preview swagger spec model validation fix * MFE 2022-12-01-preview swagger spec model validation fix, add missing location * MFE 2022-12-01-preview swagger spec model validation - typo fix * MFE 2022-12-01-preview swagger spec model validation - fix api version in automljob example * MFE 2022-12-01-preview swagger spec model validation - fix for multiselectenabled error * MFE 2022-12-01-preview swagger spec model validation - fix for multiselectenabled error * Fix for 1006 - RemovedDefinition (RecurrenceTrigger,CronTrigger) (Azure#21822) * fix ReadonlyPropertyChanged of MLC (Azure#21814) Co-authored-by: Bingchen Li <[email protected]> * fixed custom-words conflict (Azure#21829) * fix custom-words conflict merge (Azure#21830) * example fix (INVALID_REQUEST_PARAMETER) (Azure#21832) Co-authored-by: Ivaliy Ivanov <[email protected]> * example fix, use correct api preview version - (INVALID_REQUEST_PARAMETER) (Azure#21833) Co-authored-by: Ivaliy Ivanov <[email protected]> * Revert breaking change for MLC swagger 2022-12-01-preview (Azure#21885) Co-authored-by: Bingchen Li <[email protected]> * Revert Connection Category back to enum. (Azure#21939) * revert provisioning state change (Azure#21940) * remove body (Azure#21978) Co-authored-by: Komal Yadav <[email protected]> * Addressed comments, added x-ms-long-running-operation to a patch call (Azure#22005) * Addressed comments, added x-ms-long-running-operation to a patch call * fix examples for patch - remove body * fixed formatting * Ivalbert fix patch2 (Azure#22006) * Addressed comments, added x-ms-long-running-operation to a patch call * fix examples for patch - remove body * fixed formatting * fixed formatting * Updated custom words (Azure#22262) * Fixed prettier errors (Azure#22237) * fixed examples for LRO_RESPONSE_HEADER check (Azure#22293) * fixed examples for LRO_RESPONSE_HEADER check (Azure#22294) * Example fix - OBJECT_MISSING_REQUIRED_PROPERTY - Missing required property: triggerType (Azure#22317) --------- Co-authored-by: Komal Yadav <[email protected]> Co-authored-by: Komal Yadav <[email protected]> Co-authored-by: Naman Agarwal <[email protected]> Co-authored-by: Naman Agarwal <[email protected]> Co-authored-by: ZhidaLiu <[email protected]> Co-authored-by: libc16 <[email protected]> Co-authored-by: Bingchen Li <[email protected]> Co-authored-by: Ivaliy Ivanov <[email protected]>

conniey added 30 commits May 25, 2021 09:48

Add initial pom.xml files.

64e9a33

Add changelog and readme.

8a08880

Add initial EventHubsOptions and SendEventDataTest

f48ae1b

Adding SendEventDataTest.

985813e

(WIP)

77253ca

Running version script.

0e6dcea

Adding tests for Event Processor and Send operations.

73e79f0

Update lease manager and checkpoint manager.

98bc1c6

Deleting models folder.

4a8bf2e

Only create async and sync client once. Remember to dispose of receiv…

b586e1a

…ers.

Unregistering processor every time.

59da224

Adding documentation.

0cdc85b

Updating ProcessorTest.

6afa8a1

Adding implementation for Track 2 perf tests.

cbd6183

Adding base classes for t2 tests.

772c043

Fixing public ctor build break.

4d9e8e3

Update tests to use run/setup/global-setup correctly.

21147ba

Use InMemoryCheckpointManager

a3ba66c

(WIP)

bbe502d

Update to print results.

be5c829

Creating options and setting to use Azure Storage.

7c53a58

Always shutdown.

fa7eb1c

Update documentation and add test classes.

2925035

Update options.

66dfa2f

Update documentation.

ad5207c

Update dependency.

64f6610

Adding content to event processor test.

e3d3d6a

Update type ServiceTest

8b7f156

Update output on processor and remove parameters.

962f03d

Add documentation.

5270f3e

conniey added 5 commits May 25, 2021 09:48

Updating to send events first time.

1481674

Add documentation and finish sending.

40d4ac9

Rename to EventsCounter.

ea02700

Update dispose method.

c0c9b24

Add back -se.

0f90cab

conniey added Event Hubs Client This issue points to a problem in the data-plane of the library. labels May 25, 2021

conniey self-assigned this May 25, 2021

conniey requested review from alzimmermsft, danieljurek, g2vinay, JimSuplizio, mitchdenny, samvaity, srnagar and YijunXieMS as code owners May 25, 2021 17:48

conniey added 6 commits May 25, 2021 15:06

Adding tests for EventHubsOption classes.

6e03ab7

Removing jackson-databind.

387dc61

Change to 1.8.

e849915

Adding documentation with tests and removing unused parameters.

cd50f73

Update documentation.

02c9385

Adding publish switch and trying to schedule on bounded elastic.

abff096

YijunXieMS approved these changes May 27, 2021

View reviewed changes

conniey added 2 commits May 27, 2021 10:47

Merge branch 'master' of https://github.com/Azure/azure-sdk-for-java …

d30e5fc

…into performance-tests

Update versions.

f061fd4

conniey enabled auto-merge (squash) May 27, 2021 17:56

conniey merged commit ce82ffe into Azure:master May 27, 2021

conniey deleted the performance-tests branch June 1, 2021 10:57

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Adds performance tests for Event Hubs. #21822

Adds performance tests for Event Hubs. #21822

conniey commented May 25, 2021

YijunXieMS left a comment

YijunXieMS May 27, 2021

conniey May 27, 2021

YijunXieMS May 27, 2021

conniey May 27, 2021

YijunXieMS May 27, 2021

YijunXieMS May 27, 2021

conniey May 27, 2021

YijunXieMS May 27, 2021

conniey May 27, 2021

conniey May 27, 2021

Adds performance tests for Event Hubs. #21822

Adds performance tests for Event Hubs. #21822

Conversation

conniey commented May 25, 2021

YijunXieMS left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment