Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-20836][LAUNCHER] DRIVER_EXTRA_JAVA_OPTIONS is needed only when use… #18061

Conversation

wujianping10043419
Copy link

…rDeployMode is 'client'. If userDeployMode is not 'client', the launcher shouldn't care about DRIVER_EXTRA_JAVA_OPTIONS, nor should it exit abnormally because DRIVER_EXTRA_JAVA_OPTIONS contains Xmx configuration.

What changes were proposed in this pull request?

(Please fill in changes proposed in this fix)

How was this patch tested?

(Please explain how this patch was tested. E.g. unit tests, integration tests, manual tests)
(If this patch involves UI changes, please attach a screenshot; otherwise, remove this)

Please review http://spark.apache.org/contributing.html before opening a pull request.

…rDeployMode is 'client'. If userDeployMode is not 'client', the launcher shouldn't care about DRIVER_EXTRA_JAVA_OPTIONS, nor should it exit abnormally because DRIVER_EXTRA_JAVA_OPTIONS contains Xmx configuration.
@AmplabJenkins
Copy link

Can one of the admins verify this patch?

@srowen
Copy link
Member

srowen commented May 22, 2017

Although this could be moved, probably, it's not really worth bothering. It isn't actually redundant. If you specify driver memory settings in non-client mode, that should also be an error. Please close this.

@wujianping10043419
Copy link
Author

@srowen
The DRIVER_EXTRA_JAVA_OPTIONS is from a file. Maybe, Setting the parameter is an error. But Spark doesn't use it in non-client mode, nobody should pay for food who doesn't eat,so I think the code is not robust.

@srowen
Copy link
Member

srowen commented May 22, 2017

The check is truly trivial, and the check is not actually irrelevant to non-client mode.

@wujianping10043419
Copy link
Author

@srowen But the check result in the abnormal exit, because an irrelevant parameters.

@srowen
Copy link
Member

srowen commented May 23, 2017

This check is still correct for cluster mode. You should set memory with --driver-memory.

@srowen srowen mentioned this pull request Jun 7, 2017
@asfgit asfgit closed this in b771fed Jun 8, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants