[SPARK-21705][CORE][DOC]Add spark.internal.config parameter description

[heary-cao]

What changes were proposed in this pull request?

Currently, some of the configuration parameters of spark.internal.config without adding a description, which is incorrectly, this PR is based on http://spark.apache.org/docs/latest/configuration.html Property Name to supplement spark.internal.config parameter description.

How was this patch tested?

the existing test.

[srowen]

I think the problem is that this duplicates a bunch of documentation. The extended description don't need to be in the code or help messages.

[srowen]

I think the problem is that this duplicates a bunch of documentation. The extended description don't need to be in the code or help messages.

[heary-cao]

we can get the description of these configuration parameters directly from the code, except documents. so it's always good to add these descriptions to the code.

[srowen]

I'm saying that it's because they're already documented along with another configs at length in the documentation, we don't need to duplicate them in the source code. Now we have to remember to update two sets of docs at once.

[heary-cao]

Is it possible that we can delete the already deleted comments in the source code and add them to the another configs at length in the documentation?
because they don't seem to have to be, and some exist, some do not. it's weird,

[srowen]

What do you mean specifically? there is some documentation that doesn't exist anymore but should be re-added? or is there some inconsistency to resolve?

[heary-cao]

Currently, some descriptions exist in the source code that are already documented along with another configs at length in the documentation, such as: spark.files.openCostInBytes, do we need to delete them and save only one copy?

[srowen]

I agree, I don't see a value in this duplicated documentation. It's not in comments, but in the argument doc strings. It is applied inconsistently. I would favor consistency. I don't know if it's worth adding more docs, or removing the ones that are there. I guess I'm neutral on doing anything here.

[heary-cao]

cc @HyukjinKwon @gatorsmile

[gatorsmile]

In Spark SQL, the description of all the SQLConf can be displayed through the SQL commands. However, I am not sure how end users can get the descriptions of these Spark Conf.

[HyukjinKwon]

I am -0 by the same reason above ^.

[jiangxb1987]

Should we continue with this PR? @gatorsmile @srowen @HyukjinKwon

[vanzin]

There's no code to print docs for non-SQL configs, so this change at this moment in time doesn't really do anything useful. We should close this.

[AmplabJenkins]

Can one of the admins verify this patch?