Skip to content

Commit

Permalink
[Event Hubs] Convert tests to use identity auth (#46330)
Browse files Browse the repository at this point in the history 
* [Event Hubs] Convert tests to use identity auth

The focus of these changes is to convert the Event Hubs
live tests to use identity-based authorization by default.
Connection strings will continue to have test coverage and
appear in "Hello World" samples and in legacy samples for
the migration guide.

* Test fixes

* Fixing more typos

* Test debugging

* tuning test timing and renabling cancellation  check

* tuning test timing

* Extending test timing slightly

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Update sdk/eventhub/Azure.Messaging.EventHubs/tests/Snippets/Sample02_EventHubsClientsLiveTests.cs

Co-authored-by: Madalyn Redding Heaps <[email protected]>

* Regenerating snippets

---------

Co-authored-by: Madalyn Redding Heaps <[email protected]>
  • Loading branch information
@jsquire @m-redding
jsquire and m-redding authored Oct 8, 2024
1 parent ee9547d commit 7c8f16d
Show file tree
Hide file tree
Showing 50 changed files with 3,247 additions and 1,617 deletions.
65 changes: 54 additions & 11 deletions sdk/eventhub/Azure.Messaging.EventHubs.Processor/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -120,10 +120,12 @@ var processor = new EventProcessorClient(storageClient, consumerGroup, eventHubs
In order to use the `EventProcessorClient`, handlers for event processing and errors must be provided. These handlers are considered self-contained and developers are responsible for ensuring that exceptions within the handler code are accounted for.

```C# Snippet:EventHubs_Processor_ReadMe_ConfigureHandlers
var storageConnectionString = "<< CONNECTION STRING FOR THE STORAGE ACCOUNT >>";
var credential = new DefaultAzureCredential();

var storageAccountEndpoint = "<< Account Uri (likely similar to https://{your-account}.blob.core.windows.net) >>";
var blobContainerName = "<< NAME OF THE BLOB CONTAINER >>";

var eventHubsConnectionString = "<< CONNECTION STRING FOR THE EVENT HUBS NAMESPACE >>";
var fullyQualifiedNamespace = "<< NAMESPACE (likely similar to {your-namespace}.servicebus.windows.net) >>";
var eventHubName = "<< NAME OF THE EVENT HUB >>";
var consumerGroup = "<< NAME OF THE EVENT HUB CONSUMER GROUP >>";

Expand Down Expand Up @@ -157,8 +159,23 @@ async Task processErrorHandler(ProcessErrorEventArgs eventArgs)
}
}

var storageClient = new BlobContainerClient(storageConnectionString, blobContainerName);
var processor = new EventProcessorClient(storageClient, consumerGroup, eventHubsConnectionString, eventHubName);
var blobUriBuilder = new BlobUriBuilder(new Uri(storageAccountEndpoint))
{
BlobContainerName = blobContainerName
};

var storageClient = new BlobContainerClient(
blobUriBuilder.ToUri(),
credential);

var processor = new EventProcessorClient
(
storageClient,
consumerGroup,
fullyQualifiedNamespace,
eventHubName,
credential
);

processor.ProcessEventAsync += processEventHandler;
processor.ProcessErrorAsync += processErrorHandler;
Expand All @@ -172,18 +189,35 @@ The `EventProcessorClient` will perform its processing in the background once it
var cancellationSource = new CancellationTokenSource();
cancellationSource.CancelAfter(TimeSpan.FromSeconds(45));

var storageConnectionString = "<< CONNECTION STRING FOR THE STORAGE ACCOUNT >>";
var credential = new DefaultAzureCredential();

var storageAccountEndpoint = "<< Account Uri (likely similar to https://{your-account}.blob.core.windows.net) >>";
var blobContainerName = "<< NAME OF THE BLOB CONTAINER >>";

var eventHubsConnectionString = "<< CONNECTION STRING FOR THE EVENT HUBS NAMESPACE >>";
var fullyQualifiedNamespace = "<< NAMESPACE (likely similar to {your-namespace}.servicebus.windows.net) >>";
var eventHubName = "<< NAME OF THE EVENT HUB >>";
var consumerGroup = "<< NAME OF THE EVENT HUB CONSUMER GROUP >>";

Task processEventHandler(ProcessEventArgs eventArgs) => Task.CompletedTask;
Task processErrorHandler(ProcessErrorEventArgs eventArgs) => Task.CompletedTask;

var storageClient = new BlobContainerClient(storageConnectionString, blobContainerName);
var processor = new EventProcessorClient(storageClient, consumerGroup, eventHubsConnectionString, eventHubName);
var blobUriBuilder = new BlobUriBuilder(new Uri(storageAccountEndpoint))
{
BlobContainerName = blobContainerName
};

var storageClient = new BlobContainerClient(
blobUriBuilder.ToUri(),
credential);

var processor = new EventProcessorClient
(
storageClient,
consumerGroup,
fullyQualifiedNamespace,
eventHubName,
credential
);

processor.ProcessEventAsync += processEventHandler;
processor.ProcessErrorAsync += processErrorHandler;
Expand Down Expand Up @@ -225,13 +259,22 @@ To make use of an Active Directory principal with Azure Storage blob containers,

```C# Snippet:EventHubs_Processor_ReadMe_CreateWithIdentity
var credential = new DefaultAzureCredential();
var blobStorageUrl ="<< FULLY-QUALIFIED CONTAINER URL (like https://myaccount.blob.core.windows.net/mycontainer) >>";

var fullyQualifiedNamespace = "<< FULLY-QUALIFIED EVENT HUBS NAMESPACE (like something.servicebus.windows.net) >>";
var storageAccountEndpoint = "<< Account Uri (likely similar to https://{your-account}.blob.core.windows.net) >>";
var blobContainerName = "<< NAME OF THE BLOB CONTAINER >>";

var fullyQualifiedNamespace = "<< NAMESPACE (likely similar to {your-namespace}.servicebus.windows.net) >>";
var eventHubName = "<< NAME OF THE EVENT HUB >>";
var consumerGroup = "<< NAME OF THE EVENT HUB CONSUMER GROUP >>";

var storageClient = new BlobContainerClient(new Uri(blobStorageUrl), credential);
var blobUriBuilder = new BlobUriBuilder(new Uri(storageAccountEndpoint))
{
BlobContainerName = blobContainerName
};

var storageClient = new BlobContainerClient(
blobUriBuilder.ToUri(),
credential);

var processor = new EventProcessorClient
(
Expand Down
10 changes: 5 additions & 5 deletions sdk/eventhub/Azure.Messaging.EventHubs.Processor/samples/Sample01_HelloWorld.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,13 +12,13 @@ To begin, please ensure that you're familiar with the items discussed in the [Ge

## Client lifetime

Each of the Event Hubs client types is safe to cache and use for the lifetime of the application, which is best practice when the application publishes or reads events regularly or semi-regularly. The clients are responsible for efficient resource management, working to keep resource usage low during periods of inactivity and manage health during periods of higher use.
Each of the Event Hubs client types is safe to cache and use for the lifetime of the application, which is best practice when the application publishes or reads events regularly or semi-regularly. The clients are responsible for efficient resource management, working to keep resource usage low during periods of inactivity and manage health during periods of higher use.

For the `EventProcessorClient`, calling the `StopProcessingAsync` method when your application is closing will ensure that network resources and other unmanaged objects are cleaned up. Calling either the `CloseAsync` or `DisposeAsync` method on the `EventHubProducerClient` will perform the equivalent clean-up.

## Publish events

To publish events, we will make use of the `EventHubsProducerClient`. Because this is the only area of our sample that will be publishing events, we will close the client once publishing has completed. In the majority of real-world scenarios, closing the producer when the application exits is the preferred pattern.
To publish events, we will make use of the `EventHubsProducerClient`. Because this is the only area of our sample that will be publishing events, we will close the client once publishing has completed. In the majority of real-world scenarios, closing the producer when the application exits is the preferred pattern.

So that we have something to process, our example will publish a full batch of events. The `EventHubDataBatch` exists to ensure that a set of events can safely be published without exceeding the size allowed by the Event Hub. The `EventDataBatch` queries the service to understand the maximum size and is responsible for accurately measuring each event as it is added to the batch. When its `TryAdd` method returns `false`, the event is too large to fit into the batch.

Expand Down Expand Up @@ -74,9 +74,9 @@ finally

Now that the events have been published, we'll process them using the `EventProcessorClient`. It's important to note that because events are not removed when reading, you are likely to see events that had been previously published as well as those from the batch that we just sent, if you're using an existing Event Hub.

The `EventProcessorClient` is associated with a specific Event Hub and [consumer group](https://docs.microsoft.com/azure/event-hubs/event-hubs-features#consumer-groups). Conceptually, the consumer group is a label that identifies one or more event consumers as a set. Often, consumer groups are named after the responsibility of the consumer in an application, such as "Telemetry" or "OrderProcessing". When an Event Hub is created, a default consumer group is created for it. The default group, named "$Default", is what we'll be using for illustration.
The `EventProcessorClient` is associated with a specific Event Hub and [consumer group](https://learn.microsoft.com/azure/event-hubs/event-hubs-features#consumer-groups). Conceptually, the consumer group is a label that identifies one or more event consumers as a set. Often, consumer groups are named after the responsibility of the consumer in an application, such as "Telemetry" or "OrderProcessing". When an Event Hub is created, a default consumer group is created for it. The default group, named "$Default", is what we'll be using for illustration.

When events are published, they will continue to exist in the Event Hub and be available for consuming until they reach an age greater than the [retention period](https://docs.microsoft.com//azure/event-hubs/event-hubs-faq#what-is-the-maximum-retention-period-for-events). Once removed, the events are no longer available to be read and cannot be recovered. Though the Event Hubs service is free to remove events older than the retention period, it does not do so deterministically; there is no guarantee of when events will be removed.
When events are published, they will continue to exist in the Event Hub and be available for consuming until they reach an age greater than the [retention period](https://learn.microsoft.com//azure/event-hubs/event-hubs-faq#what-is-the-maximum-retention-period-for-events). Once removed, the events are no longer available to be read and cannot be recovered. Though the Event Hubs service is free to remove events older than the retention period, it does not do so deterministically; there is no guarantee of when events will be removed.

Each `EventProcessorClient` has its own full view of the events each partition of an Event Hub, meaning that events are available to all processors and are not removed from the partition when a processor reads them. This allows for one or more of the different Event Hub clients to read and process events from the partition at different speeds and beginning with different events without interfering with one another.

Expand Down Expand Up @@ -218,4 +218,4 @@ finally
processor.ProcessEventAsync -= processEventHandler;
processor.ProcessErrorAsync -= processErrorHandler;
}
```
```
Loading

0 comments on commit 7c8f16d

Please sign in to comment.