Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[run-tests] [fix][sec] Fix transitive critical CVEs in file-system tiered storage #56

Closed
wants to merge 56 commits into from

Conversation

nicoloboschi
Copy link
Owner

This PR is for running tests for upstream PR apache#19957.

mattisonchao and others added 30 commits March 14, 2023 16:48
…DataStore (apache#19730)

Master Issue: apache#16691

### Motivation

Raising a PR to implement apache#16691.

Based on the current design, only the leader needs to consume the topk bundle load data.

### Modifications
This PR 
- made only the leader consume TopBundlesLoadDataStore
- moved LeaderElectionService to ExtensibleLoadManager from ServiceUnitStateChannel.
…mpl that checks for existing schema and new schema types (apache#19753)
### Motivation

Upgrade bookkeeper to 4.15.4.
…9749)

PIP: apache#16691

### Motivation
Raising a PR to implement apache#16691.

We need to support metrics for unload/transfer operations in Load Manager Extension.

### Modifications
In this PR:
* Change the `findBundlesForUnloading` method return type from `UnloadDecision` to `Set<UnloadDecision>`.
* The `UnloadDecision` no longer contains all unload objects. Each unload object has its own reason.
* Add units test to verify the unload counter.
)

Master Issue: apache#16691

### Motivation

Raising a PR to implement apache#16691, this PR makes the split admin API functional.

### Modifications

* Make the split admin API functional when enabling the `ExtensibleLoadManager`.
* Support more split algorithm.
…ache#19670)

### Motivation

The `DirectProxyHandler` and the `ProxyConnection` are run in the same event loop to prevent context switching. As such, we do not need to schedule an event onto an event loop that is in fact the same event loop. Further, scheduling on that event loop could have resulted in uncaught failures because the method was run without any error handling.

Additionally, we can use `assert` to verify that we are in the event loop. Netty makes extensive use of this paradigm, as described in this PR apache#19653. The primary benefit here is that we skip some unnecessary volatile reads when running the code in production.

### Modifications

* Replace `checkState` with `assert` in `ProxyConnection` class
* Remove unnecessary event execution in callback when starting up the `DirectProxyHandler`.

### Verifying this change

This change is covered by the assertions that are added.

### Does this pull request potentially affect one of the following parts:

This is a minor improvement that should not break anything.

### Documentation

- [x] `doc-not-needed`

### Matching PR in forked repository

PR in forked repository: michaeljmarshall#33
…ior (apache#19845)

Relates to: apache#19455 apache#19830

### Motivation

When I added the requirement for the proxy to use a role in the `proxyRoles` set, I didn't add a test that checked the negative case. This new test was first added in apache#19830 with one small difference. The goal of this test is to ensure that authorization of the client role and the original role is handled correctly.

### Modifications

* Add new test class named `AuthorizationServiceTest`. We use `pass.proxy` and `fail.proxy` as proxy roles to simulate cases where the proxy's role passes and fails authorization, which is always possible.
* Add new mock authorization provider named `MockAuthorizationProvider`. The logic is to let any role that starts with `pass` be considered authorized.

### Verifying this change

This is a new test. It simply verifies the existing behavior to prevent future regressions.

### Documentation

- [x] `doc-not-needed`

### Matching PR in forked repository

PR in forked repository: Skipping forked test since the new tests pass locally and there are no other modifications.
…apache#19852)

### Motivation

The `onNegativeAcksSend` is only called by:
https://github.com/apache/pulsar/blob/80c5791b87482bee3392308ecef45f455f8de885/pulsar-client/src/main/java/org/apache/pulsar/client/impl/NegativeAcksTracker.java#L83

It will not be null. The statement `null if acknowledge fail.` is incorrect.

### Modifications

* Remove the incorrect statement and refine the doc.

Signed-off-by: Zike Yang <[email protected]>
Co-authored-by: Jun Ma <[email protected]>
heesung-sn and others added 26 commits March 21, 2023 22:33
…nti-affinity-group policy from topk load bundles (apache#19742)

### Motivation

Raising a PR to implement apache#16691.

Bundles with isolation policies or anti-affinity-group policies are not ideal targets to auto-unload as destination brokers are limited. 

### Modifications
This PR 
- Excluded bundles with isolation policy or anti-affinity-group policy from topk load bundles
- Introduced a config `loadBalancerSheddingBundlesWithPoliciesEnabled ` to control this behavior.
…eleted (apache#19825)

When deleting the zk node of the cursor, if the exception `MetadataStoreException.NotFoundException`  occurs, the deletion is considered successful.
…pic fails due to enabled geo-replication (apache#19879)

Motivation: As expected, If geo-replication is enabled, a topic cannot be deleted. However deleting that topic returns a 500, and no further info.
Modifications: Make response code to 400 instead of 500 when delete topic fails due to enabled geo-replication
…ioned-topic stat (apache#19942)

### Motivation

Pulsar will merge the variable `PartitionedTopicStatsImpl.replication[x].connected` by the way below when we call `pulsar-admin topics partitioned-stats`

``` java
this.connected = this.connected & other.connected
```

But the variable `connected` of `PartitionedTopicStatsImpl.replication` is initialized `false`, so the expression `this.connected & other.connected` will always be `false`.

Then we will always get the value `false` if we call `pulsar-admin topics partitioned-stats`.

### Modifications

make the variable `` of `PartitionedTopicStatsImpl` is initialized `true`
…pache#19851)

PIP: apache#16691

### Motivation
Raising a PR to implement apache#16691.

We need to support delete namespace bundle admin API.

### Modifications

* Support delete namespace bundle admin API.
* Add units test.
@nicoloboschi nicoloboschi deleted the upgrade-hadoop branch March 30, 2023 07:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.