Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Loads test tables when running "sbt hive/console" without HIVE_DEV_HOME #417

Closed
wants to merge 1 commit into from
Closed

Loads test tables when running "sbt hive/console" without HIVE_DEV_HOME #417

wants to merge 1 commit into from

Conversation

liancheng
Copy link
Contributor

When running Hive tests, the working directory is $SPARK_HOME/sql/hive, while when running sbt hive/console, it becomes $SPARK_HOME, and test tables are not loaded if HIVE_DEV_HOME is not defined.

@liancheng
Copy link
Contributor Author

@marmbrus I think we can remove HIVE_DEV_HOME, what's your opinion?

@AmplabJenkins
Copy link

Merged build triggered.

@AmplabJenkins
Copy link

Merged build started.

@AmplabJenkins
Copy link

Merged build finished. All automated tests passed.

@AmplabJenkins
Copy link

All automated tests passed.
Refer to this link for build results: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/14150/

@marmbrus
Copy link
Contributor

This change seems reasonable. I think we should leave HIVE_DEV_HOME though. The point is to easily allow you to override the built in tests when we want to upgrade or run against a different version of hive.

@marmbrus
Copy link
Contributor

@pwendell, this can be merged. No reason not to include in 1.0 as well.

@adrian-wang
Copy link
Contributor

After I merge some commits recently into my previous cloned repo, I can no longer run hive/test by sbt and then sbt > test-only org.apache.spark.sql.hive.execution.xxSuite. All the suites in this package will also not be displayed in show defined-tests. Do you guys have the same issue there? I can only do this by sbt hive/test, but it takes too much time running tests I don't care about.

@liancheng
Copy link
Contributor Author

@adrian-wang You should run hive/test-only ... within the SBT repl :)

@adrian-wang
Copy link
Contributor

@liancheng Thank you for your solution, that works fine:)
But do you know why I was able to run that via test-only ... without hive/ before that? I think it is weird to run every suites omitting what's in package hive.

@liancheng
Copy link
Contributor Author

Not quite sure about what has been changed exactly right now. But HiveCompatibilitySuite is very time consuming, and causes Travis CI timing out (Travis kills builds that last longer than 50min). I think that's one of the main reason.

@adrian-wang
Copy link
Contributor

Anyway I can run that locally now, thank you very much!

@pwendell
Copy link
Contributor

Thanks I've merged this.

@asfgit asfgit closed this in fec462c Apr 16, 2014
asfgit pushed a commit that referenced this pull request Apr 16, 2014
@liancheng @pwendell
Loads test tables when running "sbt hive/console" without HIVE_DEV_HOME
9e908ab 
When running Hive tests, the working directory is `$SPARK_HOME/sql/hive`, while when running `sbt hive/console`, it becomes `$SPARK_HOME`, and test tables are not loaded if `HIVE_DEV_HOME` is not defined.

Author: Cheng Lian <[email protected]>

Closes #417 from liancheng/loadTestTables and squashes the following commits:

7cea8d6 [Cheng Lian] Loads test tables when running "sbt hive/console" without HIVE_DEV_HOME
(cherry picked from commit fec462c)

Signed-off-by: Patrick Wendell <[email protected]>
@marmbrus
Copy link
Contributor

BTW, its likely that the problem with SBT is that you don't have SPARK_HIVE=true set. All sorts of SBT things break with hive when that is not set (~ being the one that bit me) as the hive project is not added to the root project. (We may be able to fix this if someone wants to look it it).

Also, ask these sorts of questions on the dev list. Other people might want to know the answer :)

pdeyhim pushed a commit to pdeyhim/spark-1 that referenced this pull request Jun 25, 2014
@liancheng @pdeyhim
Loads test tables when running "sbt hive/console" without HIVE_DEV_HOME
23c3d8d 
When running Hive tests, the working directory is `$SPARK_HOME/sql/hive`, while when running `sbt hive/console`, it becomes `$SPARK_HOME`, and test tables are not loaded if `HIVE_DEV_HOME` is not defined.

Author: Cheng Lian <[email protected]>

Closes apache#417 from liancheng/loadTestTables and squashes the following commits:

7cea8d6 [Cheng Lian] Loads test tables when running "sbt hive/console" without HIVE_DEV_HOME
@liancheng liancheng deleted the loadTestTables branch July 3, 2014 21:28
bzhaoopenstack pushed a commit to bzhaoopenstack/spark that referenced this pull request Sep 11, 2019
@kiwik
Clean up secgroup_2 for fusioncloud job (apache#417)
2f905f3 
Sometimes secgroup_2 is residue after fusioncloud job
running, clean up it. And also clean up port resource.

Closes: theopenlab/openlab#170
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@liancheng @AmplabJenkins @marmbrus @adrian-wang @pwendell