[SPARK-21372] spark writes one log file even I set the number of spark_rotate_log to 0

[liu-zhaokun]

https://issues.apache.org/jira/browse/SPARK-21372
Spark writes one log file even I set the number of spark_rotate_log to 0.Meanwhile,the number of spark_rotate_log also can set to a string type and other type.So I think we should make some changes.

[AmplabJenkins]

Can one of the admins verify this patch?

[jiangxb1987]

Did you see any failure in production environment caused by this issue?

[srowen]

Are you trying to verify if num is a number? I get it but you would already get a bash error in $num -eq 0 if so and we don't otherwise validate script input that way. Isn't this forcing num to be nonzero, when I thought the issue was handling the case of 0 correctly?

[liu-zhaokun]

@srowen
Yes, we should avoid this scenario that num isn't number.If num isn't a number,num will be set to 5.

[liu-zhaokun]

@jiangxb1987
When I want to change the number of log file,I test the scenario that num=0,and spark writes one log file.When I test the scenario that set num to Invaild symbol，the logic is wrong.So I think we should modify it to guarantee this feature.

[jiangxb1987]

AFAIK the function spark_rotate_log is never used with num param set, unless you set log param manually, in that case I think you are responsible for the data validation, it would be interesting to see how this issue is triggered, could you share your script please? Thank you!

[liu-zhaokun]

@jiangxb1987
I want to change the number of log file ,so I passed num param manually by modify code in this file.In fact,this bug is there，even I don't trigger it,right?Shouldn't we modify it?

[srowen]

@liu-zhaokun I don't understand what bug you are referring to. We'd need to understand how an unmodified script displays some incorrect behavior.

[liu-zhaokun]

@srowen
If I set num to 0,there shouldn't create a logfile or we should make some deal for 0，and If I set num some invaild symbol,the function doesn't work normally,right？I am sorry for my poor English.

[srowen]

How would you set this with normal operation?
If you change the script to set a value that wasn't expected at that point given how the script works, of course it may fail. That's not a bug.

[liu-zhaokun]

@srowen
There provide a param,num, to set the number of logfile,but it wasn't used in line 179.Since it doesn't work,can I remove the param "num" and use it just as a variable？

[liu-zhaokun]

@srowen
I think spark provides the param,so I can pass it in the script.

[srowen]

Let's close this. You aren't providing a scenario where an input causes a problem, and I think are still saying the error happens when you modify the script.

[liu-zhaokun]

@srowen
The param "num" which defined in function spark_rotate_log() isn't used,if users don't modify the script,it will never be invoked,so could I create a new PR to delete the param and replace it with a fixed number 5?