Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

SPARK-1497. Fix scalastyle warnings in YARN, Hive code #413

Closed
wants to merge 3 commits into from
Closed

SPARK-1497. Fix scalastyle warnings in YARN, Hive code #413

wants to merge 3 commits into from

Conversation

srowen
Copy link
Member

@srowen srowen commented Apr 15, 2014

(I wasn't sure how to automatically set SPARK_YARN=true and SPARK_HIVE=true when running scalastyle, but these are the errors that turn up.)

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished.

@AmplabJenkins
Copy link

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14144/

@srowen
Copy link
Member Author

srowen commented Apr 15, 2014

Jenkins, retest this please.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished.

@AmplabJenkins
Copy link

Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14145/

@srowen
Copy link
Member Author

srowen commented Apr 15, 2014

Different unrelated failures each time. One more roll of the dice.

Jenkins, retest this please.

@pwendell
Copy link
Contributor

Thanks for looking at this! To add this to the test harness you can augment dev/scalastyle with additional checks:

SPARK_YARN=true sbt/sbt yarn/scalastyle >> scalastyle.txt
SPARK_HADOOP_VERSION=2.2.0 SPARK_YARN=true sbt/sbt yarn/scalastyle >> scalastyle.txt
SPARK_HIVE=true sbt/sbt hive/scalastyle >> scalastyle.txt

@srowen
Copy link
Member Author

srowen commented Apr 15, 2014

Done, and good thing as it turned up a few more long lines to cut down.

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14155/

pwendell
pwendell reviewed Apr 16, 2014
View reviewed changes
dev/scalastyle
@@ -18,6 +18,10 @@
#

echo -e "q\n" | sbt/sbt clean scalastyle > scalastyle.txt
# Check style with YARN alpha built too
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any interest in doing the hive one here too? I edited my comment earlier to show how to do that.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Never mind - these are checked already.

@pwendell
Copy link
Contributor

Thanks - I've merged this!

@asfgit asfgit closed this in 77f8367 Apr 16, 2014
asfgit pushed a commit that referenced this pull request Apr 16, 2014
@srowen @pwendell
SPARK-1497. Fix scalastyle warnings in YARN, Hive code
c744d66 
(I wasn't sure how to automatically set `SPARK_YARN=true` and `SPARK_HIVE=true` when running scalastyle, but these are the errors that turn up.)

Author: Sean Owen <[email protected]>

Closes #413 from srowen/SPARK-1497 and squashes the following commits:

f0c9318 [Sean Owen] Fix more scalastyle warnings in yarn
80bf4c3 [Sean Owen] Add YARN alpha / YARN profile to scalastyle check
026319c [Sean Owen] Fix scalastyle warnings in YARN, Hive code
(cherry picked from commit 77f8367)

Signed-off-by: Patrick Wendell <[email protected]>
@srowen srowen deleted the SPARK-1497 branch April 22, 2014 12:22
pwendell added a commit to pwendell/spark that referenced this pull request May 12, 2014
@pwendell
Merge pull request apache#413 from rxin/scaladoc
68641bc 
Adjusted visibility of various components and documentation for 0.9.0 release.
pwendell pushed a commit to pwendell/spark that referenced this pull request May 12, 2014
@rxin
Merge pull request apache#425 from rxin/scaladoc
2ce23a5 
API doc update & make Broadcast public

In apache#413 Broadcast was mistakenly made private[spark]. I changed it to public again. Also exposing id in public given the R frontend requires that.

Copied some of the documentation from the programming guide to API Doc for Broadcast and Accumulator.

This should be cherry picked into branch-0.9 as well for 0.9.0 release.
pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
@srowen @pdeyhim
SPARK-1497. Fix scalastyle warnings in YARN, Hive code
84a34ed 
(I wasn't sure how to automatically set `SPARK_YARN=true` and `SPARK_HIVE=true` when running scalastyle, but these are the errors that turn up.)

Author: Sean Owen <[email protected]>

Closes apache#413 from srowen/SPARK-1497 and squashes the following commits:

f0c9318 [Sean Owen] Fix more scalastyle warnings in yarn
80bf4c3 [Sean Owen] Add YARN alpha / YARN profile to scalastyle check
026319c [Sean Owen] Fix scalastyle warnings in YARN, Hive code
andrewor14 pushed a commit to andrewor14/spark that referenced this pull request Jan 8, 2015
@rxin
Merge pull request apache#425 from rxin/scaladoc
119b6c5 
API doc update & make Broadcast public

In apache#413 Broadcast was mistakenly made private[spark]. I changed it to public again. Also exposing id in public given the R frontend requires that.

Copied some of the documentation from the programming guide to API Doc for Broadcast and Accumulator.

This should be cherry picked into branch-0.9 as well for 0.9.0 release.

(cherry picked from commit 2ce23a5)
Signed-off-by: Reynold Xin <[email protected]>
mccheah pushed a commit to mccheah/spark that referenced this pull request Nov 28, 2018
@robert3005
Document fork (apache#413)
6d02722
bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
@liusheng
Merge pull request apache#413 from jtopjian/revert-tf-schema-panic
e02e4bc 
Revert TF_SCHEMA_PANIC_ON_ERROR for Terraform OpenStack Provider
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@srowen @AmplabJenkins @pwendell