Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[SPARK-20506][DOCS] 2.2 migration guide #17996

Closed
wants to merge 7 commits into from
Closed

[SPARK-20506][DOCS] 2.2 migration guide #17996

wants to merge 7 commits into from

Conversation

MLnick
Copy link
Contributor

@MLnick MLnick commented May 16, 2017

Update ML guide for migration 2.1 -> 2.2 and the previous version migration guide section.

How was this patch tested?

Build doc locally.

@MLnick
Copy link
Contributor Author

MLnick commented May 16, 2017

Just doing a final check to find any other behavior changes or stuff that was deprecated in this release cycle.

@SparkQA
Copy link

SparkQA commented May 16, 2017

Test build #76961 has finished for PR 17996 at commit ac2e50d.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@MLnick
Copy link
Contributor Author

MLnick commented May 17, 2017

cc @yanboliang @srowen @jkbradley

yanboliang
yanboliang reviewed May 18, 2017
View reviewed changes
Copy link
Contributor

@yanboliang yanboliang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One minor question, otherwise, LGTM. Thanks!

docs/ml-guide.md

### Deprecations and changes of behavior

**Deprecations**

* [SPARK-18592](https://issues.apache.org/jira/browse/SPARK-18592):
Deprecate all Param setter methods except for input/output column Params for `DecisionTreeClassificationModel`, `GBTClassificationModel`, `RandomForestClassificationModel`, `DecisionTreeRegressionModel`, `GBTRegressionModel` and `RandomForestRegressionModel`
There are no deprecations.

**Changes of behavior**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we include #17233 in this section?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks - didn't catch that one

@MLnick
Copy link
Contributor Author

MLnick commented May 18, 2017

@felixcheung @yanboliang by the way I haven't added any SparkR stuff here as I'm not sure anything breaking, deprecated etc goes here or in the SparkR guide.

@yanboliang
Copy link
Contributor

yanboliang commented May 18, 2017 •
edited
Loading

@MLnick We have separate SparkR migration guide at http://spark.apache.org/docs/latest/sparkr.html#migration-guide . Thanks.

@SparkQA
Copy link

SparkQA commented May 18, 2017

Test build #77052 has finished for PR 17996 at commit e27d9e4.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

@felixcheung
Copy link
Member

looks good
We do have a separate section but it's mostly for breaking/behavior changes at the R layer, if there are changes to ml that affects R also would be great to include here.

and I think "what's new" is a great idea too

dongjoon-hyun
dongjoon-hyun reviewed May 19, 2017
View reviewed changes
docs/ml-guide.md Outdated
([SPARK-11968](https://issues.apache.org/jira/browse/SPARK-11968) and
[SPARK-20587](https://issues.apache.org/jira/browse/SPARK-20587))
* `Correlation` and `ChiSquareTest` stats functions for `DataFrames`
([SPARK-19635](https://issues.apache.org/jira/browse/SPARK-19635) and
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, @MLnick . Correlation issue is SPARK-19636.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah right thanks for catching that

@dongjoon-hyun
Copy link
Member

+1, LGTM except one minor comment about JIRA issue number.

@MLnick
Copy link
Contributor Author

MLnick commented May 19, 2017

Ok, I think this is good to go so will merge shortly.

@felixcheung for R, if you do know of anything else that should go here (rather than in the SparkR migration guide) let me know.

@SparkQA
Copy link

SparkQA commented May 19, 2017

Test build #77100 has finished for PR 17996 at commit fb9fb5b.

  • This patch passes all tests.
  • This patch merges cleanly.
  • This patch adds no public classes.

asfgit pushed a commit that referenced this pull request May 19, 2017
[SPARK-20506][DOCS] 2.2 migration guide
4fcd52b 
Update ML guide for migration `2.1` -> `2.2` and the previous version migration guide section.

## How was this patch tested?

Build doc locally.

Author: Nick Pentreath <[email protected]>

Closes #17996 from MLnick/SPARK-20506-2.2-migration-guide.

(cherry picked from commit b5d8d9b)
Signed-off-by: Nick Pentreath <[email protected]>
@MLnick
Copy link
Contributor Author

MLnick commented May 19, 2017

Merged to master/branch-2.2

@asfgit asfgit closed this in b5d8d9b May 19, 2017
@felixcheung
Copy link
Member

right, I reviewed them

  • this in ML could also affect R
  • this in R might be good to include in ML guide too

@MLnick MLnick deleted the SPARK-20506-2.2-migration-guide branch May 20, 2017 10:54
@MLnick
Copy link
Contributor Author

MLnick commented May 20, 2017

@felixcheung ok - the KMeans behavior change was in 2.1 actually (so not in this migration guide, in this PR it was moved to the "old versions migration guides" page).

For spark.lda issue - was that only for SparkR API? Or for all of LDA? If the latter then it could be included perhaps. If the former then I think just in the R section of the guide, no need for it here.

@MLnick MLnick mentioned this pull request May 20, 2017
Closed
@MLnick
Copy link
Contributor Author

MLnick commented May 20, 2017

I actually realized I forgot to add the HTML links to the relevant guide sections in the highlight list - opened a little PR at #18043 to fix that.

ghost pushed a commit to dbtsai/spark that referenced this pull request May 22, 2017
[SPARK-20506][DOCS] Add HTML links to highlight list in MLlib guide f…
be846db 
…or 2.2

Quick follow up to apache#17996 - forgot to add the HTML links to the relevant sections of the guide in the highlights list.

## How was this patch tested?

Built docs locally and tested links.

Author: Nick Pentreath <[email protected]>

Closes apache#18043 from MLnick/SPARK-20506-2.2-migration-guide-2.
asfgit pushed a commit that referenced this pull request May 22, 2017
[SPARK-20506][DOCS] Add HTML links to highlight list in MLlib guide f…
50dba30 
…or 2.2

Quick follow up to #17996 - forgot to add the HTML links to the relevant sections of the guide in the highlights list.

## How was this patch tested?

Built docs locally and tested links.

Author: Nick Pentreath <[email protected]>

Closes #18043 from MLnick/SPARK-20506-2.2-migration-guide-2.

(cherry picked from commit be846db)
Signed-off-by: Nick Pentreath <[email protected]>
liyichao pushed a commit to liyichao/spark that referenced this pull request May 24, 2017
@liyichao
[SPARK-20506][DOCS] 2.2 migration guide
83e0e96 
Update ML guide for migration `2.1` -> `2.2` and the previous version migration guide section.

## How was this patch tested?

Build doc locally.

Author: Nick Pentreath <[email protected]>

Closes apache#17996 from MLnick/SPARK-20506-2.2-migration-guide.
liyichao pushed a commit to liyichao/spark that referenced this pull request May 24, 2017
@liyichao
[SPARK-20506][DOCS] Add HTML links to highlight list in MLlib guide f…
7ca6de5 
…or 2.2

Quick follow up to apache#17996 - forgot to add the HTML links to the relevant sections of the guide in the highlights list.

## How was this patch tested?

Built docs locally and tested links.

Author: Nick Pentreath <[email protected]>

Closes apache#18043 from MLnick/SPARK-20506-2.2-migration-guide-2.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@dongjoon-hyun dongjoon-hyun dongjoon-hyun left review comments

@yanboliang yanboliang yanboliang left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@MLnick @SparkQA @yanboliang @felixcheung @dongjoon-hyun