Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Blue green #8

Open
wants to merge 687 commits into
base: master
Choose a base branch
from
Open

Blue green #8

wants to merge 687 commits into from

Conversation

vineeth1995
Copy link
Owner

Fixes #xyz

Master Issue: #xyz

Motivation

Modifications

Verifying this change

  • Make sure that the change passes the CI checks.

(Please pick either of the following options)

This change is a trivial rework / code cleanup without any test coverage.

(or)

This change is already covered by existing tests, such as (please describe tests).

(or)

This change added tests and can be verified as follows:

(example:)

  • Added integration tests for end-to-end deployment with large payloads (10MB)
  • Extended integration test for recovery after broker failure

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

  • Dependencies (add or upgrade a dependency)
  • The public API
  • The schema
  • The default values of configurations
  • The threading model
  • The binary protocol
  • The REST endpoints
  • The admin CLI options
  • Anything that affects deployment

Documentation

  • doc
  • doc-required
  • doc-not-needed
  • doc-complete

Matching PR in forked repository

PR in forked repository:

heesung-sn and others added 30 commits January 29, 2023 15:19
…dCursor(single subscription check) upon subscription (apache#19343)
Relates to: apache#17831 (comment)

### Motivation

When the `ProxyConnection` handles a `Connect` command, that is the time to go to `Connecting` state. There is no other time that makes sense to switch to connecting. The current logic will go to connecting in certain re-authentication scenarios, but those are incorrect. By moving the state change to earlier in the logic, we make the state transition clearer and prevent corrupted state.

### Modifications

* Remove `state = State.Connecting` from the `doAuthentication` method, which is called multiple times for various reasons
* Add `state = State.Connecting` to the start of the `handleConnect` method.

### Verifying this change

The existing tests will verify this change, and reading through the code makes it clear this is a correct change.

### Does this pull request potentially affect one of the following parts:

Not a breaking change.

### Documentation

- [x] `doc-not-needed` 

It would be nice to map out the state transitions for our connection classes. That is our of the scope of this small improvement.

### Matching PR in forked repository

PR in forked repository: michaeljmarshall#21
… txn race condition. (apache#19201)

Fixes apache#19200


### Motivation
transaction lasted for long time and will not be aborted, which cause TB's MaxReadPosition do not move and will not take snapshot. With an old snapshot, TB will read a lot of entry while doing recovery.
In worst cases, there are 30 minutes of unavailable time with Topics.

### Modifications


avoid concurrent execution.
### Motivation

If the transaction build fails, the `Success to new txn` still is printed. This is an incorrect log and will make users confused.

### Modifications

* Print success log only if the transaction build successfully.

Signed-off-by: Zike Yang <[email protected]>
Co-authored-by: Nicolò Boschi <[email protected]>
PIP: apache#12105 

### Motivation

Implement asynchronous auth for the proxy connection. This is one of the core PRs for implementing apache#12105. 

### Modifications

* Update `ProxyConnection` class to asynchronously handle the authentication result. The result is handled on the handler's event loop to ensure correctness.
* Update `ProxyAuthenticationTest` class to implement async auth methods and to make authentication asynchronous to test that code path.

### Verifying this change

There is an updated test, but it doesn't cover all code paths in this PR.

### Documentation

- [x] `doc-not-needed`

We do not need to document this portion of PIP 97.

### Matching PR in forked repository

PR in forked repository: michaeljmarshall#16
lhotari and others added 19 commits March 28, 2023 19:07
…ioned-topic stat (apache#19942)

### Motivation

Pulsar will merge the variable `PartitionedTopicStatsImpl.replication[x].connected` by the way below when we call `pulsar-admin topics partitioned-stats`

``` java
this.connected = this.connected & other.connected
```

But the variable `connected` of `PartitionedTopicStatsImpl.replication` is initialized `false`, so the expression `this.connected & other.connected` will always be `false`.

Then we will always get the value `false` if we call `pulsar-admin topics partitioned-stats`.

### Modifications

make the variable `` of `PartitionedTopicStatsImpl` is initialized `true`
…pache#19851)

PIP: apache#16691

### Motivation
Raising a PR to implement apache#16691.

We need to support delete namespace bundle admin API.

### Modifications

* Support delete namespace bundle admin API.
* Add units test.
Master Issue: Master Issue: apache#16691, apache#18099

### Motivation

Raising a PR to implement Master Issue: apache#16691, apache#18099

We want to reduce unload frequencies from flaky traffic.

### Modifications
This PR 
- Introduced a config `loadBalancerSheddingConditionHitCountThreshold` to further restrict shedding conditions based on the hit count.
- Normalized offload traffic
- Lowered the default `loadBalanceSheddingDelayInSeconds` value from 600 to 180, as 10 mins are too long. 3 mins can be long enough to catch the new load after unloads.
- Changed the config `loadBalancerBundleLoadReportPercentage` to `loadBalancerMaxNumberOfBundlesInBundleLoadReport` to make the topk bundle count absolute instead of relative.
- Renamed `loadBalancerNamespaceBundleSplitConditionThreshold` to `loadBalancerNamespaceBundleSplitConditionHitCountThreshold` to be consistent with `*ConditionHitCountThreshold`.
- Renamed `loadBalancerMaxNumberOfBrokerTransfersPerCycle ` to `loadBalancerMaxNumberOfBrokerSheddingPerCycle`.
- Added LoadDataStore cleanup logic in BSC monitor.
- Added `msgThroughputEMA` in BrokerLoadData to smooth the broker throughput info.
- Updated Topk bundles sorted in a ascending order (instead of descending)
- Update some info logs to only show in the debug mode.
- Added load data tombstone upon Own, Releasing, Splitting
- Added the bundle ownership(isOwned) check upon split and unload.
- Added swap unload logic
…9951)

Motivation
Kafka's schema has "Optional" flag that used there to validate data/allow nulls.
Pulsar's schema does not have such info which makes conversion to kafka schema lossy.

Modifications
Added a config parameter that lets one force primitive schemas into optional ones.
KV schema is always optional.

Default is false, to match existing behavior.
…pic with ProtoBuf schema (apache#19767)

### Motivation
1. There is a topic1 with a protobuf schema.
2. Create a producer1 with AutoProduceBytes schema.
3. The producer1 will be created failed because the way to get the schema of protobuf schema is not supported. ### ### 
### Modification
Because the Protobuf schema is implemented from the AvroBaseStructSchema. So we add a way to get Protobuf schema just like the AvroSchema.
…, when the updateStats method is executed (apache#19887)

Co-authored-by: lordcheng10 <[email protected]>
apache#19990)

### Motivation

While debugging an issue, I noticed that we call `super.exceptionCaught(ctx, cause);` in the `ProxyConnection` class. This leads to the following log line:

> An exceptionCaught() event was fired, and it reached at the tail of the pipeline. It usually means the last handler in the pipeline did not handle the exception. io.netty.channel.unix.Errors$NativeIoException: recvAddress(..) failed: Connection reset by peer

Because we always handle exceptions, there is no need to forward them to the next handler.

### Modifications

* Remove a single method call

### Verifying this change

This is a trivial change. Note that we do not call the super method in any other handler implementations in the project.

### Documentation

- [x] `doc-not-needed` <!-- Your PR changes do not impact docs -->

### Matching PR in forked repository

PR in forked repository: skipping PR for this trivial change
@vineeth1995 vineeth1995 force-pushed the blue-green branch 4 times, most recently from a7d95f3 to 6ce3799 Compare April 10, 2023 16:36
@github-actions
Copy link

The pr had no activity for 30 days, mark with Stale label.

@github-actions github-actions bot added the Stale label May 11, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.