Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add request parameter 'cluster_manager_timeout' and deprecate 'master_timeout' - in Cluster APIs #2658

Merged
merged 19 commits into from
Apr 4, 2022

Conversation

tlfeng
Copy link
Collaborator

@tlfeng tlfeng commented Mar 29, 2022

Description

Apply the change of CAT Nodes API in PR #2435 to applicable Cluster APIs.

  • Deprecate the request parameter master_timeout that used in Cluster APIs which have got the parameter.
  • Add alternative new request parameter cluster_manager_timeout.
  • Add unit tests.

List of the Cluster APIs that have got request parameter master_timeout:
Cluster health API https://opensearch.org/docs/opensearch/rest-api/cluster-health/
Cluster reroute API - POST /_cluster/reroute
Cluster state API - GET /_cluster/state/<metrics>/<target>
Cluster get settings API https://opensearch.org/docs/latest/opensearch/rest-api/cluster-settings/
Cluster update settings API https://opensearch.org/docs/latest/opensearch/rest-api/cluster-settings/
Pending cluster tasks API - GET /_cluster/pending_tasks

Issues Resolved

A part of issue #2511

Check List

  • New functionality includes testing.
    • All tests pass
  • New functionality has been documented.
    • New functionality has javadoc added
  • Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

@tlfeng tlfeng added enhancement Enhancement or improvement to existing feature or request v2.0.0 Version 2.0.0 backport 2.x Backport to 2.x branch backport 2.0 Backport to 2.0 branch labels Mar 29, 2022
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 5c770eb
Log 3878

Reports 3878

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 1c8a22f
Log 3885

Reports 3885

@tlfeng tlfeng force-pushed the manager-timeout-cluster-api branch from d63046c to 5b35951 Compare March 30, 2022 05:13
@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success a8b20ca
Log 3888

Reports 3888

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 2fb34ed
Log 3889

Reports 3889

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success d63046c0398ff054e00d66a9990f0bda9e19a70c
Log 3890

Reports 3890

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 5b35951
Log 3891

Reports 3891

@tlfeng tlfeng marked this pull request as ready for review March 30, 2022 06:42
@tlfeng tlfeng requested a review from a team as a code owner March 30, 2022 06:42
Copy link
Collaborator

@nknize nknize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lets remove the need to duplicate the parseDeprecatedMasterTimeoutParameter by refactoring as a public static method in BaseRestHandler. This way we only have to remove the method in one place when it comes time to remove it altogether.

* @deprecated As of 2.0, because promoting inclusive language.
*/
@Deprecated
private static void parseDeprecatedMasterTimeoutParameter(ClusterStateRequest clusterStateRequest, RestRequest request) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we refactor this as a public static method in BaseRestHandler.java and abstract it to take a base MasterNodeRequest instance so we don't have to duplicate in each of these files?

e.g.,

    /**
     * Parse the deprecated request parameter 'master_timeout', and add deprecated log if the parameter is used.
     * It also validates whether the value of 'master_timeout' is the same with 'cluster_manager_timeout'.
     * Remove the method along with MASTER_ROLE.
     * @deprecated As of 2.0, because promoting inclusive language.
     */
    @Deprecated
    public static void parseDeprecatedMasterTimeoutParameter(MasterNodeRequest mnr, RestRequest request) {
        final String deprecatedTimeoutParam = "master_timeout";
        if (request.hasParam(deprecatedTimeoutParam)) {
            deprecationLogger.deprecate("cluster_state_master_timeout_parameter", MASTER_TIMEOUT_DEPRECATED_MESSAGE);
            request.validateParamValuesAreEqual(deprecatedTimeoutParam, "cluster_manager_timeout");
            mnr.masterNodeTimeout(request.paramAsTime(deprecatedTimeoutParam, mnr.masterNodeTimeout()));
        }
    }

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi Nick, thanks a lot for your time in the review!
Sure, I will make the suggested change. I realized adding the duplicate method into every Action class is a pain, as well as removing in the future, and my idea was avoid adding the API-specific method into the abstract base class.
Since it's a temporary method and you agree with it, I will move it into BaseRestHandler class. 😁

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ty!

Copy link
Collaborator Author

@tlfeng tlfeng Mar 30, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nknize I put your suggested change into a separate PR #2670.
Please take a look at when you have time.

I plan to take that PR as a foundation and example for changing other REST APIs. After that PR merged, the codes of deprecating master_timeout parameter in the other REST APIs can be rebased and taken the benefit (such as PR #2557 & #2658).
I will update this PR after PR #2670 is merged.

Thank you for your good suggestion 😄, it can really reduce the lines of code changes and save my time!

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The duplidated parseDeprecatedMasterTimeoutParameter() method is put into abstract base class BaseRestHandler through PR #2670, and I have updated this PR. 😁

* @deprecated As of 2.0, because promoting inclusive language.
*/
@Deprecated
private static void parseDeprecatedMasterTimeoutParameter(ClusterHealthRequest clusterHealthRequest, RestRequest request) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in favor of base method

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved these issues in above. 😄

* @deprecated As of 2.0, because promoting inclusive language.
*/
@Deprecated
private static void parseDeprecatedMasterTimeoutParameter(ClusterRerouteRequest clusterRerouteRequest, RestRequest request) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in favor of base method

* @deprecated As of 2.0, because promoting inclusive language.
*/
@Deprecated
private static void parseDeprecatedMasterTimeoutParameter(ClusterStateRequest clusterStateRequest, RestRequest request) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in favor of base method

* @deprecated As of 2.0, because promoting inclusive language.
*/
@Deprecated
private static void parseDeprecatedMasterTimeoutParameter(
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in favor of base method

* @deprecated As of 2.0, because promoting inclusive language.
*/
@Deprecated
private static void parseDeprecatedMasterTimeoutParameter(PendingClusterTasksRequest pendingClusterTasksRequest, RestRequest request) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in favor of base method

* @deprecated As of 2.0, because promoting inclusive language.
*/
@Deprecated
private static void parseDeprecatedMasterTimeoutParameter(PendingClusterTasksRequest pendingClusterTasksRequest, RestRequest request) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove in favor of base method

Tianli Feng added 3 commits March 30, 2022 13:38
…r class to reduce duplication

Signed-off-by: Tianli Feng <[email protected]>
…r class to reduce duplication

Signed-off-by: Tianli Feng <[email protected]>
Signed-off-by: Tianli Feng <[email protected]>

# Conflicts:
#	server/src/test/java/org/opensearch/action/RenamedTimeoutRequestParameterTests.java
@tlfeng tlfeng force-pushed the manager-timeout-cluster-api branch from 30ed89a to 2399800 Compare April 2, 2022 00:02
Tianli Feng added 2 commits April 1, 2022 17:04
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 30ed89a7ca94bcc365e33241618e6792fa5aefef
Log 4050

Reports 4050

Signed-off-by: Tianli Feng <[email protected]>
@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 2399800
Log 4051

Reports 4051

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 33556a4
Log 4052

Reports 4052

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure f5d9fc7
Log 4053

Reports 4053

@tlfeng
Copy link
Collaborator Author

tlfeng commented Apr 2, 2022

In log 4053:

> Task :plugins:discovery-azure-classic:yamlRestTest
OpenJDK 64-Bit Server VM warning: Ignoring option --illegal-access=warn; support was removed in 17.0
org.gradle.internal.remote.internal.ConnectException: Could not connect to server [dc550475-9a07-4c5f-b754-25f11346beb2 port:34511, addresses:[/127.0.0.1]]. Tried addresses: [/127.0.0.1].
	at org.gradle.internal.remote.internal.inet.TcpOutgoingConnector.connect(TcpOutgoingConnector.java:67)
	at org.gradle.internal.remote.internal.hub.MessageHubBackedClient.getConnection(MessageHubBackedClient.java:36)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:123)
	at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:71)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
Caused by: java.net.ConnectException: Connection refused
	at java.base/sun.nio.ch.Net.pollConnect(Native Method)
	at java.base/sun.nio.ch.Net.pollConnectNow(Net.java:672)
	at java.base/sun.nio.ch.SocketChannelImpl.finishTimedConnect(SocketChannelImpl.java:1141)
	at java.base/sun.nio.ch.SocketChannelImpl.blockingConnect(SocketChannelImpl.java:1183)
	at java.base/sun.nio.ch.SocketAdaptor.connect(SocketAdaptor.java:98)
	at org.gradle.internal.remote.internal.inet.TcpOutgoingConnector.tryConnect(TcpOutgoingConnector.java:81)
	at org.gradle.internal.remote.internal.inet.TcpOutgoingConnector.connect(TcpOutgoingConnector.java:54)
	... 5 more

and

Tests with failures:
 - org.opensearch.client.ClusterClientIT.testClusterGetSettingsWithDefault
 - org.opensearch.client.ClusterClientIT.testClusterGetSettings
 - org.opensearch.client.ClusterClientIT.testClusterHealthYellowClusterLevel

The Client test failure will be resolved in #2702

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 4ce930a
Log 4055

Reports 4055

@tlfeng
Copy link
Collaborator Author

tlfeng commented Apr 2, 2022

In log 4055:

> Task :qa:remote-clusters:integTest

REPRODUCE WITH: ./gradlew ':qa:remote-clusters:integTest' --tests "org.opensearch.cluster.remote.test.RemoteClustersIT.testSniffModeConnectionFails" -Dtests.seed=2864D2487045E9F6 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=tr-TR -Dtests.timezone=Indian/Reunion -Druntime.java=17

org.opensearch.cluster.remote.test.RemoteClustersIT > testSniffModeConnectionFails FAILED
    org.opensearch.client.WarningFailureException: method [GET], host [http://localhost:49257], URI [/_cluster/health?master_timeout=30s&wait_for_nodes=1&level=cluster&timeout=30s&wait_for_status=yellow], status line [HTTP/1.1 200 OK]
    Warnings: [Deprecated parameter [master_timeout] used. To promote inclusive language, please use [cluster_manager_timeout] instead. It will be unsupported in a future major version.]

The Client test failure will be resolved in #2702

Copy link
Collaborator

@nknize nknize left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent! Thx for doing that. This LGTM!

@opensearch-ci-bot
Copy link
Collaborator

❌   Gradle Check failure 58b30d4
Log 4127

Reports 4127

@tlfeng
Copy link
Collaborator Author

tlfeng commented Apr 4, 2022

In log 4127:

> Task :qa:remote-clusters:integTest

REPRODUCE WITH: ./gradlew ':qa:remote-clusters:integTest' --tests "org.opensearch.cluster.remote.test.RemoteClustersIT.testHAProxyModeConnectionWorks" -Dtests.seed=3503A8ADB16E4721 -Dtests.security.manager=true -Dtests.jvm.argline="-XX:TieredStopAtLevel=1 -XX:ReservedCodeCacheSize=64m" -Dtests.locale=he -Dtests.timezone=GMT0 -Druntime.java=17
WARNING: A terminally deprecated method in java.lang.System has been called
WARNING: System::setSecurityManager has been called by org.gradle.api.internal.tasks.testing.worker.TestWorker (file:/home/ubuntu/.gradle/wrapper/dists/gradle-7.4.2-all/9uukhhbclvbegdvsww0j0cr3p/gradle-7.4.2/lib/plugins/gradle-testing-base-7.4.2.jar)
WARNING: Please consider reporting this to the maintainers of org.gradle.api.internal.tasks.testing.worker.TestWorker
WARNING: System::setSecurityManager will be removed in a future release

org.opensearch.cluster.remote.test.RemoteClustersIT > testHAProxyModeConnectionWorks FAILED
    java.lang.AssertionError
        at __randomizedtesting.SeedInfo.seed([3503A8ADB16E4721:32757D668AE8317C]:0)
        at org.junit.Assert.fail(Assert.java:87)
        at org.junit.Assert.assertTrue(Assert.java:42)
        at org.junit.Assert.assertTrue(Assert.java:53)
        at org.opensearch.cluster.remote.test.RemoteClustersIT.testHAProxyModeConnectionWorks(RemoteClustersIT.java:125)

It's reported in issue #1703
re-run: start gradle check.

@opensearch-ci-bot
Copy link
Collaborator

✅   Gradle Check success 58b30d4
Log 4134

Reports 4134

@tlfeng tlfeng merged commit 7d23b18 into opensearch-project:main Apr 4, 2022
@tlfeng tlfeng deleted the manager-timeout-cluster-api branch April 4, 2022 20:47
@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.x failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.x 2.x
# Navigate to the new working tree
cd .worktrees/backport-2.x
# Create a new branch
git switch --create backport/backport-2658-to-2.x
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7d23b180f77821266f0acd779a9db637739ee0b9
# Push it to GitHub
git push --set-upstream origin backport/backport-2658-to-2.x
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.x

Then, create a pull request where the base branch is 2.x and the compare/head branch is backport/backport-2658-to-2.x.

@opensearch-trigger-bot
Copy link
Contributor

The backport to 2.0 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-2.0 2.0
# Navigate to the new working tree
cd .worktrees/backport-2.0
# Create a new branch
git switch --create backport/backport-2658-to-2.0
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 7d23b180f77821266f0acd779a9db637739ee0b9
# Push it to GitHub
git push --set-upstream origin backport/backport-2658-to-2.0
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-2.0

Then, create a pull request where the base branch is 2.0 and the compare/head branch is backport/backport-2658-to-2.0.

tlfeng pushed a commit to tlfeng/OpenSearch that referenced this pull request Apr 4, 2022
…_timeout' - in Cluster APIs (opensearch-project#2658)

- Deprecate the request parameter `master_timeout` that used in Cluster APIs which have got the parameter.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <[email protected]>
tlfeng pushed a commit to tlfeng/OpenSearch that referenced this pull request Apr 4, 2022
…_timeout' - in Cluster APIs (opensearch-project#2658)

- Deprecate the request parameter `master_timeout` that used in Cluster APIs which have got the parameter.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <[email protected]>
tlfeng pushed a commit that referenced this pull request Apr 4, 2022
…_timeout' - in Cluster APIs (#2658) (#2755)

- Deprecate the request parameter `master_timeout` that used in Cluster APIs which have got the parameter.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <[email protected]>
tlfeng pushed a commit that referenced this pull request Apr 5, 2022
…_timeout' - in Cluster APIs (#2658) (#2754)

- Deprecate the request parameter `master_timeout` that used in Cluster APIs which have got the parameter.
- Add alternative new request parameter `cluster_manager_timeout`.
- Add unit tests.

Signed-off-by: Tianli Feng <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport 2.x Backport to 2.x branch backport 2.0 Backport to 2.0 branch enhancement Enhancement or improvement to existing feature or request v2.0.0 Version 2.0.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants