Hide bwc build output on success #42102
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Previously we used LoggedExec for running the internal bwc builds. However, this had bad performance implications as all the output was buffered into memory, thus we changed back to normal Exec. This commit adds a `useFileBuffer` setting to LoggedExec which can be used for commands with large amounts of output, and switches the bwc builds to use this flag.
rjernst
added
>enhancement
:Delivery/Build
|
Pinging @elastic/es-core-infra |
|
@elasticmachine run elasticsearch-ci/1 |
|
@elasticmachine run elasticsearch-ci/bwc |
mark-vieira
approved these changes
May 13, 2019
| ByteArrayOutputStream output = new ByteArrayOutputStream(); | ||
| ByteArrayOutputStream error = new ByteArrayOutputStream(); | ||
| this.useFileBuffer = getProject().getObjects().property(Boolean.class); | ||
| this.useFileBuffer.set(false); // default to in memory |
There was a problem hiding this comment.
I believe the "convention" is to use convention() for setting default values on properties.
buildSrc/src/main/minimumRuntime/org/elasticsearch/gradle/LoggedExec.java
Outdated
Show resolved
Hide resolved
buildSrc/src/main/minimumRuntime/org/elasticsearch/gradle/LoggedExec.java
Outdated
Show resolved
Hide resolved
buildSrc/src/main/minimumRuntime/org/elasticsearch/gradle/LoggedExec.java
Outdated
Show resolved
Hide resolved
buildSrc/src/main/minimumRuntime/org/elasticsearch/gradle/LoggedExec.java
Outdated
Show resolved
Hide resolved
|
Thanks @mark-vieira. I really liked your idea for not using afterEvaluate. I pushed e4b8a9b, can you take another look? I believe I also addressed your other big points. |
mark-vieira
approved these changes
May 14, 2019
buildSrc/src/main/minimumRuntime/org/elasticsearch/gradle/LazyFileOutputStream.java
Show resolved
Hide resolved
| if (useFileBuffer) { | ||
| File spoolFile = new File(getProject().getBuildDir() + "/buffered-output/" + this.getName()); | ||
| out = new LazyFileOutputStream(spoolFile); | ||
| outputReader = () -> Files.readString(spoolFile.toPath()); |
There was a problem hiding this comment.
Another potential optimization (and not a blocker) would be to define instead an outputLogger instead of outputReader which also handled the logging bit. This way the file-based once could stream from the spool file to the console instead of having to read the entire log file into memory.
buildSrc/src/main/minimumRuntime/org/elasticsearch/gradle/LazyFileOutputStream.java
Show resolved
Hide resolved
|
Updates LGTM 👍 |
rjernst
added a commit
that referenced
this pull request
May 16, 2019
Previously we used LoggedExec for running the internal bwc builds. However, this had bad performance implications as all the output was buffered into memory, thus we changed back to normal Exec. This commit adds a `spoolOutput` setting to LoggedExec which can be used for commands with large amounts of output, and switches the bwc builds to use this flag.
gurkankaymak
pushed a commit
to gurkankaymak/elasticsearch
that referenced
this pull request
May 27, 2019
Previously we used LoggedExec for running the internal bwc builds. However, this had bad performance implications as all the output was buffered into memory, thus we changed back to normal Exec. This commit adds a `spoolOutput` setting to LoggedExec which can be used for commands with large amounts of output, and switches the bwc builds to use this flag.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Previously we used LoggedExec for running the internal bwc builds.
However, this had bad performance implications as all the output was
buffered into memory, thus we changed back to normal Exec. This commit
adds a
useFileBuffersetting to LoggedExec which can be used forcommands with large amounts of output, and switches the bwc builds to
use this flag.