Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[SPARK-23268][SQL][followup] Reorganize packages in data source V2 #20509

Closed
wants to merge 2 commits into from

Conversation

cloud-fan
Copy link
Contributor

What changes were proposed in this pull request?

This is a followup of #20435.

While reorganizing the packages for streaming data source v2, the top level stream read/write support interfaces should not be in the reader/writer package, but should be in the sources.v2 package, to follow the ReadSupport, WriteSupport, etc.

How was this patch tested?

N/A

@cloud-fan
Copy link
Contributor Author

@SparkQA
Copy link

SparkQA commented Feb 5, 2018

Test build #87065 has finished for PR 20509 at commit e3f007f.

  • This patch fails to build.
  • This patch merges cleanly.
  • This patch adds no public classes.

@gengliangwang
Copy link
Member

The proposal sounds good to me 👍

@SparkQA
Copy link

SparkQA commented Feb 5, 2018

Test build #87070 has finished for PR 20509 at commit 613d180.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

Copy link
Member

@dongjoon-hyun dongjoon-hyun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1, LGTM.

@gatorsmile
Copy link
Member

cc @tdas @marmbrus @rxin @zsxwing

@ueshin
Copy link
Member

ueshin commented Feb 7, 2018

nit: we can remove some imports due to moving into the same package, from ContinuousReadSupport.java, MicroBatchReadSupport.java, and StreamWriteSupport.java.

@SparkQA
Copy link

SparkQA commented Feb 7, 2018

Test build #87141 has finished for PR 20509 at commit 83e8197.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@SparkQA
Copy link

SparkQA commented Feb 8, 2018

Test build #87188 has finished for PR 20509 at commit ec64554.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@cloud-fan
Copy link
Contributor Author

thanks, merging to master/2.3!

asfgit pushed a commit that referenced this pull request Feb 8, 2018
## What changes were proposed in this pull request?

This is a followup of #20435.

While reorganizing the packages for streaming data source v2, the top level stream read/write support interfaces should not be in the reader/writer package, but should be in the `sources.v2` package, to follow the `ReadSupport`, `WriteSupport`, etc.

## How was this patch tested?

N/A

Author: Wenchen Fan <[email protected]>

Closes #20509 from cloud-fan/followup.

(cherry picked from commit a75f927)
Signed-off-by: Wenchen Fan <[email protected]>
@asfgit asfgit closed this in a75f927 Feb 8, 2018
robert3005 pushed a commit to palantir/spark that referenced this pull request Feb 12, 2018
## What changes were proposed in this pull request?

This is a followup of apache#20435.

While reorganizing the packages for streaming data source v2, the top level stream read/write support interfaces should not be in the reader/writer package, but should be in the `sources.v2` package, to follow the `ReadSupport`, `WriteSupport`, etc.

## How was this patch tested?

N/A

Author: Wenchen Fan <[email protected]>

Closes apache#20509 from cloud-fan/followup.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants