Fix travis builds #84
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Lately, builds on travis have failed with sudo: pip3: command not found I'm unsure why that is, because /usr/bin/pip3 exists, /usr/bin/python3 exists etc. and the message is not clear about which command it's really missing. Since Python 3.4, pip is included as a callable module and surprisingly using python3 -m pip doesn't run into any command not found errors. Until we figure out what's really broken here, just use python3 -m pip.
apache-rat:check is a maven goal that can fail and writes rat.txt files containing more information about the failure. Show those failures.
Travis builds are failing the org.apache.rat:apache-rat-plugin:0.12:check goal with the following entry in integration-test/target/rat.txt: ***************************************************** Files with unapproved licenses: /home/travis/build/dashdb-spark/livy/integration-test/.pytest_cache/v/cache/nodeids ***************************************************** The pytest developers are aware of this, see pytest-dev/pytest#3286.
Codecov Report
@@ Coverage Diff @@
## master #84 +/- ##
============================================
+ Coverage 71.47% 71.55% +0.07%
- Complexity 793 795 +2
============================================
Files 97 97
Lines 5396 5396
Branches 798 798
============================================
+ Hits 3857 3861 +4
+ Misses 1015 1013 -2
+ Partials 524 522 -2
Continue to review full report at Codecov.
|
|
This might be due to recent changes of travis image/repo. Can you please investigate on why it happens recently? |
|
BTW, can you please create a related JIRA about this issue. |
|
Thanks for the fix. Merging to master and branch 0.5. |
asfgit
pushed a commit
that referenced
this pull request
Apr 9, 2018
Each individual commit has a more detailed description of what's being changed and why. At the moment, Travis builds don't work: * https://travis-ci.org/mineo/incubator-livy/builds/359324523 - the `sudo pip3 install --upgrade pip "setuptools < 36"` command fails with `sudo: pip3: command not found`. * fixing that, the `failing the org.apache.rat:apache-rat-plugin:0.12:check` maven goal fails because it sees `.pytest_cache` folders that it doesn't know about (pytest-dev/pytest#3286, failure in https://travis-ci.org/mineo/incubator-livy/jobs/359326261, check the raw log). This pull request works around the pip3 failures by just using pip as a callable module (possible since Python 3.4) and adding `.pytest_cache` to raw-excludes, as well as showing the contents of `rat.txt` files in the `after_failure` step. I concede that the pip change is more of a workaround, but I don't know how else to fix it and unblock testing. Running the tests on travis. Please review https://livy.incubator.apache.org/community/ before opening a pull request. Author: Wieland Hoffmann <[email protected]> Closes #84 from mineo/rat. (cherry picked from commit 06cfa7b) Signed-off-by: jerryshao <[email protected]>
tkakantousis
pushed a commit
to tkakantousis/incubator-livy
that referenced
this pull request
Oct 23, 2018
Each individual commit has a more detailed description of what's being changed and why. At the moment, Travis builds don't work: * https://travis-ci.org/mineo/incubator-livy/builds/359324523 - the `sudo pip3 install --upgrade pip "setuptools < 36"` command fails with `sudo: pip3: command not found`. * fixing that, the `failing the org.apache.rat:apache-rat-plugin:0.12:check` maven goal fails because it sees `.pytest_cache` folders that it doesn't know about (pytest-dev/pytest#3286, failure in https://travis-ci.org/mineo/incubator-livy/jobs/359326261, check the raw log). This pull request works around the pip3 failures by just using pip as a callable module (possible since Python 3.4) and adding `.pytest_cache` to raw-excludes, as well as showing the contents of `rat.txt` files in the `after_failure` step. I concede that the pip change is more of a workaround, but I don't know how else to fix it and unblock testing. Running the tests on travis. Please review https://livy.incubator.apache.org/community/ before opening a pull request. Author: Wieland Hoffmann <[email protected]> Closes apache#84 from mineo/rat. (cherry picked from commit 06cfa7b) Signed-off-by: jerryshao <[email protected]>
tkakantousis
added a commit
to logicalclocks/incubator-livy
that referenced
this pull request
Oct 24, 2018
* [MINOR] Addressed Docs dependency security vulnerability followup Followup to 26428c5 Author: Alex Bozarth <[email protected]> Closes apache#82 from ajbozarth/gem. (cherry picked from commit 6d2ffdd) Signed-off-by: Alex Bozarth <[email protected]> * [MINOR] Fix travis builds Each individual commit has a more detailed description of what's being changed and why. At the moment, Travis builds don't work: * https://travis-ci.org/mineo/incubator-livy/builds/359324523 - the `sudo pip3 install --upgrade pip "setuptools < 36"` command fails with `sudo: pip3: command not found`. * fixing that, the `failing the org.apache.rat:apache-rat-plugin:0.12:check` maven goal fails because it sees `.pytest_cache` folders that it doesn't know about (pytest-dev/pytest#3286, failure in https://travis-ci.org/mineo/incubator-livy/jobs/359326261, check the raw log). This pull request works around the pip3 failures by just using pip as a callable module (possible since Python 3.4) and adding `.pytest_cache` to raw-excludes, as well as showing the contents of `rat.txt` files in the `after_failure` step. I concede that the pip change is more of a workaround, but I don't know how else to fix it and unblock testing. Running the tests on travis. Please review https://livy.incubator.apache.org/community/ before opening a pull request. Author: Wieland Hoffmann <[email protected]> Closes apache#84 from mineo/rat. (cherry picked from commit 06cfa7b) Signed-off-by: jerryshao <[email protected]> * [LIVY-455][REPL] Fix json4s doesn't support java.math.BigDecimal issue ## What changes were proposed in this pull request? Livy's SQLInterpreter will throw exception when rows contain java.math.BigDecimal data. This is because current version of json4s doesn't treat java.math.BigDecimal type as primitive type. On the contrary, json4s supports Scala BigDecimal as primitive type. So the fix is to convert java BigDecimal to Scala BigDecimal. ## How was this patch tested? Unit test is added. Author: jerryshao <[email protected]> Closes apache#85 from jerryshao/LIVY-455. (cherry picked from commit 7e4bb3b) Signed-off-by: jerryshao <[email protected]> * [LIVY-457][REPL] Fix SQLContext is not initialized correctly issue ## What changes were proposed in this pull request? The signature of SQLContext's constructor is changed in Spark2, but we're still using the Spark1's signature, which will throw an exception when using this object. ## How was this patch tested? UT and local verification. Author: jerryshao <[email protected]> Closes apache#86 from jerryshao/LIVY-457. (cherry picked from commit cd8b112) Signed-off-by: jerryshao <[email protected]> * [LIVY-466][RSC] Fix RSCDriver exception during RPC shutdown ## What changes were proposed in this pull request? During RSCDriver's shutdown, it will first shutdown RPC server, and then all the RPC clients. When RPC client is closed, it will register a timeout to avoid orphaned RSCDriver, but this is not necessary during RSCDriver's shutdown, so here fixing this issue. The details can be seen in [JIRA](https://issues.apache.org/jira/browse/LIVY-466). ## How was this patch tested? Local verification. Author: jerryshao <[email protected]> Closes apache#90 from jerryshao/LIVY-466. (cherry picked from commit e3f45a0) Signed-off-by: jerryshao <[email protected]> * [LIVY-472][SERVER] Improve the logs for fail-to-create session ## What changes were proposed in this pull request? Livy currently doesn't give a very clear log about the fail-to-create session, it only says that session related app tag cannot be found in RM, but doesn't tell user how to search and get the true root cause. So here change the logs to make it more clear. ## How was this patch tested? Local verification. Author: jerryshao <[email protected]> Closes apache#96 from jerryshao/LIVY-472. (cherry picked from commit ca4cad2) Signed-off-by: jerryshao <[email protected]> * [Security] Update to support pyspark and sparkr changes in Spark 2.3.1 * [LIVY-498][REPL] Fix Windows CRLF line ending issue in SparkR interpreter ## What changes were proposed in this pull request? If the issued query contains CRLF EOL, it will be failed to execute on *nix machine. This happens when submitting queries from Windows machine and executing on Linux machine. So here propose to convert statement to match system's EOL. ## How was this patch tested? New UT added. Author: jerryshao <[email protected]> Closes apache#105 from jerryshao/LIVY-498. (cherry picked from commit 8027ca7) Signed-off-by: jerryshao <[email protected]> * upgrade to work with spark 2.3.2
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Each individual commit has a more detailed description of what's being changed and why.
What changes were proposed in this pull request?
At the moment, Travis builds don't work:
sudo pip3 install --upgrade pip "setuptools < 36"command fails withsudo: pip3: command not found.failing the org.apache.rat:apache-rat-plugin:0.12:checkmaven goal fails because it sees.pytest_cachefolders that it doesn't know about (.pytest_cache is showing up in projects git repos pytest-dev/pytest#3286, failure in https://travis-ci.org/mineo/incubator-livy/jobs/359326261, check the raw log).This pull request works around the pip3 failures by just using pip as a callable module (possible since Python 3.4) and adding
.pytest_cacheto raw-excludes, as well as showing the contents ofrat.txtfiles in theafter_failurestep. I concede that the pip change is more of a workaround, but I don't know how else to fix it and unblock testing.How was this patch tested?
Running the tests on travis.
Please review https://livy.incubator.apache.org/community/ before opening a pull request.