[SPARK-6724] [MLlib] Support model save/load for FPGrowthModel #9267
Conversation
|
Test build #44311 has finished for PR 9267 at commit
|
| throw new UnsupportedOperationException(s"Schema for type $other is not supported") | ||
| } | ||
| result | ||
| } |
There was a problem hiding this comment.
Maybe we can make the inferring one of Spark SQL functions just like ScalaReflection.schemaFor?
|
Test build #47509 has finished for PR 9267 at commit
|
|
I'll take a look at this now. |
| @@ -42,8 +53,9 @@ import org.apache.spark.storage.StorageLevel | |||
| */ | |||
| @Since("1.3.0") | |||
| @Experimental | |||
| class FPGrowthModel[Item: ClassTag] @Since("1.3.0") ( | |||
| @Since("1.3.0") val freqItemsets: RDD[FreqItemset[Item]]) extends Serializable { | |||
| class FPGrowthModel[Item: ClassTag: TypeTag] @Since("1.3.0") ( | |||
There was a problem hiding this comment.
This will break API compatibility since old user code might provide a class for which a TypeTag is not available.
|
Your PR looks good, but I hope we can eliminate the need for Item to be restricted by TypeTag. Also, eliminating This should probably include a Java unit test b/c of how it uses types. Thanks! I'll watch for updates. |
yanboliang
force-pushed
the
spark-6724
branch
from
81f667a to
c5e528b
Compare
|
@jkbradley I updated the PR and it's not necessary to provide |
|
Test build #48546 has finished for PR 9267 at commit
|
| @@ -20,17 +20,27 @@ package org.apache.spark.mllib.fpm | |||
| import java.{util => ju} | |||
| import java.lang.{Iterable => JavaIterable} | |||
|
|
|||
| import org.json4s.DefaultFormats | |||
There was a problem hiding this comment.
Organize imports (scala before 3rd-party libraries)
|
@yanboliang This change should let you remove inferItemType. (Tests pass at least.) |
|
Test build #48731 has finished for PR 9267 at commit
|
|
@jkbradley That's cool! It also works well in my environment. Thanks for your kindly help! |
|
@yanboliang No problem; thanks for your updates! This LGTM Merging with master |
```PrefixSpanModel``` supports ```save/load```. It's similar with apache#9267. cc jkbradley Author: Yanbo Liang <[email protected]> Closes apache#10664 from yanboliang/spark-10386.
Support model save/load for FPGrowthModel