Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AzDatalake] Pipelines + cleanup #21298

Merged
merged 82 commits into from
Aug 10, 2023

Conversation

tasherif-msft
Copy link
Contributor

@tasherif-msft tasherif-msft commented Jul 31, 2023

TODO:
figure out unmarshaljson issue

jhendrixMSFT and others added 30 commits April 28, 2023 09:55
Enabled gocritic's evalOrder to catch dependencies on undefined behavior
on return statements.
Updated to latest version of golangci-lint.
Fixed issue in azblob flagged by latest linter.
* Adding header and value

* Wiring and tests

* format

* Fixing value

* change log
…e#20710)

Logging improvements:

* Updating the logging to print more tracing information (per-link) in prep for the bigger release coming up.
* Trimming out some of the verbose logging, seeing if I can get it a bit more reasonable.

Stress tests:

* Add a timestamp to the log name we generate and also default to append, not overwrite.
* Use 0.5 cores, 0.5GB as our baseline. Some pods use more and I'll tune them more later.
…re#20721)

This now works just like the message batch - you'll get an ErrMessageTooLarge
if you attempt to send a message that's too large for the link's configured
size.

NOTE: there's a patch to `internal/go-amqp/Sender.go` to match what's in go-amqp's
main so it returns a programmatically useful error when the message is too large.

Fixes Azure#20647
…re#20722)

When the remote entity is full we get a resource-limit-exceeded condition. This isn't something we should keep retrying on and it's best to just abort and let the user know immediately, rather than hoping it might eventually clear out.

This affected both Event Hubs and Service Bus.

Fixes Azure#20647
* Update changelog with latest features

Prepare for upcoming release.

* bump minor version
The checkpoint store wasn't guarding against multiple owners claiming for the first time - fixing this by using IfNoneMatch

Fixes Azure#20717
…zure#20437)

* [Release] sdk/resourcemanager/paloaltonetworksngfw/armpanngfw/0.1.0 generation from spec commit: 85fb4ac6f8bfefd179e6c2632976a154b5c9ff04

* client factory

* fix

* fix

* update
* add sdk/resourcemanager/postgresql/armpostgresql live test

* update assets.json

* set subscriptionId default value

* format
* add sdk/resourcemanager/eventhub/armeventhub live test

* update assets
* add sdk/resourcemanager/compute/armcompute live test

* skus filter

* fix subscriptionId default value

* fix

* gofmt

* update recording
* sdk/resourcemanager/network/armnetwork live test

* update subscriptionId default value

* update recording
* add sdk/resourcemanager/cosmos/armcosmos live test

* update assets.json

* update assets.json

* update assets.json

* update assets.json
…e#20737)

We shouldn't be resetting the etag to nil - it's what we use to enforce a "single winner" when doing ownership claims.

The bug here was two-fold: I had bad logic in my previous claim ownership, which I fixed in a previous PR, but we need to reflect that same constraint properly in our in-memory checkpoint store for these tests.
* Fixing connection string parse logic

* Update README
* fix flaky test

* charles suggestion
@tasherif-msft
Copy link
Contributor Author

/azp run go - azdatalake - ci

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@Azure Azure deleted a comment from azure-pipelines bot Aug 1, 2023
@tasherif-msft
Copy link
Contributor Author

/azp run go - azdatalake - ci

@azure-pipelines
Copy link

No commit pushedDate could be found for PR 21298 in repo Azure/azure-sdk-for-go

@tasherif-msft
Copy link
Contributor Author

/azp run go - azdatalake

@azure-pipelines
Copy link

Azure Pipelines successfully started running 1 pipeline(s).

@tasherif-msft
Copy link
Contributor Author

/check-enforcer override

@tasherif-msft tasherif-msft merged commit 8fd0022 into Azure:feature/azdatalake Aug 10, 2023
@tasherif-msft tasherif-msft deleted the ci-pipe branch August 10, 2023 01:07
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AzDatalake Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

Successfully merging this pull request may close these issues.