[SPARK-3359][DOCS] Make javadoc8 working for unidoc/genjavadoc compatibility in Java API documentation #16013
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -645,7 +645,7 @@ class SparkContext(config: SparkConf) extends Logging { | |
|
|
||
| /** | ||
| * Get a local property set in this thread, or null if it is missing. See | ||
| * `org.apache.spark.SparkContext.setLocalProperty`. | ||
| */ | ||
| def getLocalProperty(key: String): String = | ||
| Option(localProperties.get).map(_.getProperty(key)).orNull | ||
|
|
@@ -663,7 +663,7 @@ class SparkContext(config: SparkConf) extends Logging { | |
| * Application programmers can use this method to group all those jobs together and give a | ||
| * group description. Once set, the Spark web UI will associate such jobs with this group. | ||
| * | ||
| * The application can also use `org.apache.spark.SparkContext.cancelJobGroup` to cancel all | ||
| * running jobs in this group. For example, | ||
| * {{{ | ||
| * // In the main thread: | ||
|
|
@@ -1384,7 +1384,7 @@ class SparkContext(config: SparkConf) extends Logging { | |
| } | ||
|
|
||
| /** | ||
| * Create and register a `CollectionAccumulator`, which starts with empty list and accumulates | ||
| * inputs by adding them into the list. | ||
| */ | ||
| def collectionAccumulator[T]: CollectionAccumulator[T] = { | ||
|
|
@@ -1394,7 +1394,7 @@ class SparkContext(config: SparkConf) extends Logging { | |
| } | ||
|
|
||
| /** | ||
| * Create and register a `CollectionAccumulator`, which starts with empty list and accumulates | ||
| * inputs by adding them into the list. | ||
| */ | ||
| def collectionAccumulator[T](name: String): CollectionAccumulator[T] = { | ||
|
|
@@ -2043,7 +2043,7 @@ class SparkContext(config: SparkConf) extends Logging { | |
| } | ||
|
|
||
| /** | ||
| * Cancel active jobs for the specified group. See `org.apache.spark.SparkContext.setJobGroup` | ||
| * for more information. | ||
| */ | ||
| def cancelJobGroup(groupId: String) { | ||
|
|
@@ -2063,6 +2063,7 @@ class SparkContext(config: SparkConf) extends Logging { | |
| * @param jobId the job ID to cancel | ||
| * @throws InterruptedException if the cancel message cannot be sent | ||
|
There was a problem hiding this comment. This is interesting too. This does not throws an error in javadoc8 unlike https://github.com/apache/spark/pull/16013/files#r89664921 There was a problem hiding this comment. It seems fine
|
||
| */ | ||
| @throws(classOf[InterruptedException]) | ||
|
There was a problem hiding this comment. I think these need to be reverted too; we don't want to introduce checked exceptions There was a problem hiding this comment. Oh, actually this complains as below: Let me move this to |
||
| def cancelJob(jobId: Int) { | ||
| dagScheduler.cancelJob(jobId) | ||
| } | ||
|
|
@@ -2073,6 +2074,7 @@ class SparkContext(config: SparkConf) extends Logging { | |
| * @param stageId the stage ID to cancel | ||
| * @throws InterruptedException if the cancel message cannot be sent | ||
| */ | ||
| @throws(classOf[InterruptedException]) | ||
| def cancelStage(stageId: Int) { | ||
| dagScheduler.cancelStage(stageId) | ||
| } | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -103,7 +103,8 @@ class JavaRDD[T](val rdd: RDD[T])(implicit val classTag: ClassTag[T]) | |
| * @param withReplacement can elements be sampled multiple times (replaced when sampled out) | ||
| * @param fraction expected size of the sample as a fraction of this RDD's size | ||
| * without replacement: probability that each element is chosen; fraction must be [0, 1] | ||
| * with replacement: expected number of times each element is chosen; fraction must be greater | ||
| * than or equal to 0 | ||
|
There was a problem hiding this comment. I can work around this to There was a problem hiding this comment. Prose is fine too, but, was this an error? is the point that the escape itself gets escaped? I might have missed this is your various comments. There was a problem hiding this comment. Oh, yes. Exactly. It was not an error but it was printed as they are ( I haven't looked into this super deeper as the output is incorrect anyway but I suspect this replacement in genjavadoc. There was a problem hiding this comment. In short, the only workarounds to print them I found were inlined tags such as |
||
| * | ||
| * @note This is NOT guaranteed to provide exactly the fraction of the count | ||
| * of the given `RDD`. | ||
|
|
@@ -117,7 +118,8 @@ class JavaRDD[T](val rdd: RDD[T])(implicit val classTag: ClassTag[T]) | |
| * @param withReplacement can elements be sampled multiple times (replaced when sampled out) | ||
| * @param fraction expected size of the sample as a fraction of this RDD's size | ||
| * without replacement: probability that each element is chosen; fraction must be [0, 1] | ||
| * with replacement: expected number of times each element is chosen; fraction must be greater | ||
| * than or equal to 0 | ||
| * @param seed seed for the random number generator | ||
| * | ||
| * @note This is NOT guaranteed to provide exactly the fraction of the count | ||
|
|
@@ -167,7 +169,7 @@ class JavaRDD[T](val rdd: RDD[T])(implicit val classTag: ClassTag[T]) | |
| * Return an RDD with the elements from `this` that are not in `other`. | ||
| * | ||
| * Uses `this` partitioner/partition size, because even if `other` is huge, the resulting | ||
| * RDD will be less than or equal to us. | ||
| */ | ||
| def subtract(other: JavaRDD[T]): JavaRDD[T] = wrapRDD(rdd.subtract(other)) | ||
|
|
||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -152,10 +152,12 @@ class DoubleRDDFunctions(self: RDD[Double]) extends Logging with Serializable { | |
|
|
||
| /** | ||
| * Compute a histogram using the provided buckets. The buckets are all open | ||
| * to the right except for the last which is closed. | ||
| * {{{ | ||
| * e.g. for the array | ||
| * [1, 10, 20, 50] the buckets are [1, 10) [10, 20) [20, 50] | ||
| * e.g 1<=x<10 , 10<=x<20, 20<=x<=50 | ||
| * }}} | ||
|
There was a problem hiding this comment. Why put the whole section as code? it's just the portion with several inequalities. Can those be back-tick-escaped? Or is the point that the back-ticks don't work? does BTW, very minor, but the second "e.g" misses a period. "e.g." isn't really great either. You could expand the first to "For example" and the second to "that is", which was really what was meant. There was a problem hiding this comment. I just wanted to avoid to use inlined tags simply because my IDE shows some warnings.. This seems a valid javadoc markdown but it seems any inlined tags are treated as backticts in scaladoc. (I am sorry for messing around with many comments but this is also related with #16013 (comment)) There was a problem hiding this comment. I will try to treat the comments below too but let me maybe try to reduce the usages of the inlined tags where possible. There was a problem hiding this comment.
There was a problem hiding this comment. Yeap. I tested some cases with it (If I remember correctly) and it seems any |
||
| * And on the input of 1 and 50 we would have a histogram of 1, 0, 1 | ||
| * | ||
| * @note If your histogram is evenly spaced (e.g. [0, 10, 20, 30]) this can be switched | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just decided to keep original format rather than trying to make this pretty.
The original was as below:
Scala
Java
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
After this PR it still prints the same.
Scala
Java