Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-5323][SQL] Remove Row's Seq inheritance. #4115

Closed
wants to merge 8 commits into from
Closed

[SPARK-5323][SQL] Remove Row's Seq inheritance. #4115

wants to merge 8 commits into from

Conversation

rxin
Copy link
Contributor

@rxin rxin commented Jan 20, 2015

No description provided.

@rxin
Copy link
Contributor Author

rxin commented Jan 20, 2015

cc @yhuai @marmbrus @liancheng you should probably all take a look at this since it changes so many things.

Our test cases are really hard to change :(

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25797 has started for PR 4115 at commit 5c6fa9c.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25798 has started for PR 4115 at commit cd6d46a.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25797 has finished for PR 4115 at commit 5c6fa9c.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25797/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25798 has finished for PR 4115 at commit cd6d46a.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait Row extends Serializable

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25798/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25806 has started for PR 4115 at commit c21111a.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25806 has finished for PR 4115 at commit c21111a.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait Row extends Serializable

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25806/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25811 has started for PR 4115 at commit 9d48918.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25811 has finished for PR 4115 at commit 9d48918.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait Row extends Serializable

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25811/
Test FAILed.

liancheng
liancheng reviewed Jan 20, 2015
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala
if ((row1 eq null) && (row2 eq null)) {
"[ empty row ]"
} else if (row1 eq null) {
"[" + row2.mkString(",") + "]"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this: row2.mkString("[", ",", "]")?

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25832 has started for PR 4115 at commit 8a1256a.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25833 has started for PR 4115 at commit 7049186.

  • This patch merges cleanly.

marmbrus
marmbrus reviewed Jan 20, 2015
View reviewed changes
sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/WrapDynamic.scala
@@ -39,7 +39,8 @@ case class WrapDynamic(children: Seq[Attribute]) extends Expression {
override def eval(input: Row): DynamicRow = input match {
// Avoid copy for generic rows.
case g: GenericRow => new DynamicRow(children, g.values)
case otherRowType => new DynamicRow(children, otherRowType.toArray)
case otherRowType => new DynamicRow(children, otherRowType.toSeq.toArray)
// TODO: toSeq.toArray is very slow!
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We might even just get rid of WrapDynamic

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25832 has finished for PR 4115 at commit 8a1256a.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait Row extends Serializable

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25832/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25833 has finished for PR 4115 at commit 7049186.

  • This patch fails PySpark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):
    • trait Row extends Serializable

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25833/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25844 has started for PR 4115 at commit 01fe5f8.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25844 has finished for PR 4115 at commit 01fe5f8.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25844/
Test FAILed.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25845 has started for PR 4115 at commit 05310d7.

  • This patch does not merge cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25847 has started for PR 4115 at commit e33abd8.

  • This patch merges cleanly.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25847 has finished for PR 4115 at commit e33abd8.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25847/
Test PASSed.

@SparkQA
Copy link

SparkQA commented Jan 20, 2015

Test build #25845 has finished for PR 4115 at commit 05310d7.

  • This patch passes all tests.
  • This patch does not merge cleanly.
  • This patch adds no public classes.

@AmplabJenkins
Copy link

Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/25845/
Test PASSed.

@asfgit asfgit closed this in d181c2a Jan 20, 2015
@liancheng liancheng mentioned this pull request Jan 21, 2015
Closed
@rxin rxin deleted the row-seq branch January 21, 2015 19:41
asfgit pushed a commit that referenced this pull request Jan 21, 2015
@liancheng @rxin
[SQL] [Minor] Remove deprecated parquet tests
ba19689 
This PR removes the deprecated `ParquetQuerySuite`, renamed `ParquetQuerySuite2` to `ParquetQuerySuite`, and refactored changes introduced in #4115 to `ParquetFilterSuite` . It is a follow-up of #3644.

Notice that test cases in the old `ParquetQuerySuite` have already been well covered by other test suites introduced in #3644.

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/4116)
<!-- Reviewable:end -->

Author: Cheng Lian <[email protected]>

Closes #4116 from liancheng/remove-deprecated-parquet-tests and squashes the following commits:

f73b8f9 [Cheng Lian] Removes deprecated Parquet test suite
bomeng pushed a commit to Huawei-Spark/spark that referenced this pull request Jan 22, 2015
@rxin @bomeng
[SPARK-5323][SQL] Remove Row's Seq inheritance.
125e4a1 
Author: Reynold Xin <[email protected]>

Closes apache#4115 from rxin/row-seq and squashes the following commits:

e33abd8 [Reynold Xin] Fixed compilation error.
cceb650 [Reynold Xin] Python test fixes, and removal of WrapDynamic.
0334a52 [Reynold Xin] mkString.
9cdeb7d [Reynold Xin] Hive tests.
15681c2 [Reynold Xin] Fix more test cases.
ea9023a [Reynold Xin] Fixed a catalyst test.
c5e2cb5 [Reynold Xin] Minor patch up.
b9cab7c [Reynold Xin] [SPARK-5323][SQL] Remove Row's Seq inheritance.
bomeng pushed a commit to Huawei-Spark/spark that referenced this pull request Jan 22, 2015
@liancheng @bomeng
[SQL] [Minor] Remove deprecated parquet tests
b32d256 
This PR removes the deprecated `ParquetQuerySuite`, renamed `ParquetQuerySuite2` to `ParquetQuerySuite`, and refactored changes introduced in apache#4115 to `ParquetFilterSuite` . It is a follow-up of apache#3644.

Notice that test cases in the old `ParquetQuerySuite` have already been well covered by other test suites introduced in apache#3644.

<!-- Reviewable:start -->
[<img src="https://reviewable.io/review_button.png" height=40 alt="Review on Reviewable"/>](https://reviewable.io/reviews/apache/spark/4116)
<!-- Reviewable:end -->

Author: Cheng Lian <[email protected]>

Closes apache#4116 from liancheng/remove-deprecated-parquet-tests and squashes the following commits:

f73b8f9 [Cheng Lian] Removes deprecated Parquet test suite
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@rxin @SparkQA @AmplabJenkins @marmbrus @liancheng