Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use ctx.actions.args() in compile_scala #1124

Merged
merged 16 commits into from
Feb 23, 2021

Conversation

simuons
Copy link
Collaborator

@simuons simuons commented Oct 21, 2020

Description

Use ctx.actions.args() to pass parameters to persistent worker.

Motivation

As per recommendations in https://docs.bazel.build/versions/master/skylark/performance.html#use-ctxactionsargs-for-command-lines this should reduce memory consumption at analysis phase by:

  • deferring expansion of depsets
  • avoiding concatenation of list
  • avoiding construction of large strings for args file

@google-cla google-cla bot added the cla: yes label Oct 21, 2020
@simuons simuons marked this pull request as ready for review October 22, 2020 04:23
@simuons simuons requested a review from ittaiz as a code owner October 22, 2020 04:23
if (lSplit.length > 1) {
hm.put(lSplit[0], lSplit[1]);
}
HashMap<String, String> hm = new HashMap<>();
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there a reason to stay here with a HashMap instead of Map?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably not. Just left it as it was. So now I've changed var and return type to Map

enable_diagnostics_report = enable_diagnostics_report,
diagnostics_output = diagnosticsfile.path,
)
args = ctx.actions.args()
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can args collection be extracted into a separate function?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Probably it can. But if I just extract whole args to separate function it will contain same huge parameter list as compile_scala. If I extract multiple functions to build args I'll lose visibility how final args file would look like. I'll look what I can do about it.

@simuons
Copy link
Collaborator Author

simuons commented Oct 22, 2020

I'm sorry but I'm moving this PR back to draft mode. I noticed that argfile was passed as input to the ctx.actions.run and with this change it is not. I need to understand consequences of such change.

@simuons simuons marked this pull request as draft October 22, 2020 12:19
@ittaiz
Copy link
Member

ittaiz commented Oct 22, 2020 via email

@simuons simuons marked this pull request as ready for review February 22, 2021 13:23
@simuons
Copy link
Collaborator Author

simuons commented Feb 22, 2021

I'm sorry I had to do a rebase because it was handing for too long.

There are no new tests because this is a completely internal refactoring.

I believe that eventually we will need to move to ctx.actions.args in all other places as this is a recommended practice.

@ittaiz ittaiz merged commit 4130036 into bazelbuild:master Feb 23, 2021
@simuons simuons deleted the compile-scala-with-args branch February 24, 2021 06:37
wisechengyi pushed a commit to twitter-forks/rules_scala that referenced this pull request Mar 11, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants