Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-11673][SQL] Remove the normal Project physical operator (and keep TungstenProject) #9643

Closed
wants to merge 2 commits into from

Conversation

rxin
Copy link
Contributor

@rxin rxin commented Nov 11, 2015

Also make full outer join being able to produce UnsafeRows.

@@ -176,6 +176,22 @@ case class Invoke(
}
}

case class SerializeUsingKryo(child: Expression) extends UnaryExpression {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i need to revert this change

@rxin
Copy link
Contributor Author

rxin commented Nov 11, 2015

cc @davies

@SparkQA
Copy link

SparkQA commented Nov 11, 2015

Test build #45684 has finished for PR 9643 at commit da0657f.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds the following public classes (experimental):\n * case class SerializeUsingKryo(child: Expression) extends UnaryExpression\n

@rxin
Copy link
Contributor Author

rxin commented Nov 12, 2015

---OK let's work on this one after #9645 is merged.---

#6645 is now merged.

rxin added 2 commits November 11, 2015 19:34
…eep TungstenProject)

Also make full outer join being able to produce UnsafeRows.
@SparkQA
Copy link

SparkQA commented Nov 12, 2015

Test build #45706 has finished for PR 9643 at commit a049df9.

  • This patch fails Spark unit tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor

should we also remove the newProjection and newMutableProjection in SparkPlan?

@SparkQA
Copy link

SparkQA commented Nov 12, 2015

Test build #2048 has finished for PR 9643 at commit a049df9.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@rxin
Copy link
Contributor Author

rxin commented Nov 12, 2015

I think so, but usually I find it easier to remove stuff incrementally (due to breaking tests).

asfgit pushed a commit that referenced this pull request Nov 12, 2015
…eep TungstenProject)

Also make full outer join being able to produce UnsafeRows.

Author: Reynold Xin <[email protected]>

Closes #9643 from rxin/SPARK-11673.

(cherry picked from commit 30e7433)
Signed-off-by: Reynold Xin <[email protected]>
@rxin
Copy link
Contributor Author

rxin commented Nov 12, 2015

Oops - did anybody review this? I've already merged it. Can somebody take a closer look. Thanks.

cc @cloud-fan @davies @yhuai

@asfgit asfgit closed this in 30e7433 Nov 12, 2015
@yhuai
Copy link
Contributor

yhuai commented Nov 12, 2015

@cloud-fan I think we should keep newMutableProjection in SparkPlan. Or, do we have any other way to create a mutable projection?

@yhuai
Copy link
Contributor

yhuai commented Nov 12, 2015

looks good to me

@davies
Copy link
Contributor

davies commented Nov 12, 2015

LGTM

1 similar comment
@cloud-fan
Copy link
Contributor

LGTM

dskrvk pushed a commit to dskrvk/spark that referenced this pull request Nov 13, 2015
…eep TungstenProject)

Also make full outer join being able to produce UnsafeRows.

Author: Reynold Xin <[email protected]>

Closes apache#9643 from rxin/SPARK-11673.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants