Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Exception messages need to be richer\clearer #3995

Closed
nicolehaugen opened this issue Jul 12, 2019 · 2 comments · Fixed by #5444
Closed

Exception messages need to be richer\clearer #3995

nicolehaugen opened this issue Jul 12, 2019 · 2 comments · Fixed by #5444
Assignees
Labels
lightgbm Bugs related lightgbm P1 Priority of the issue for triage purpose: Needs to be fixed soon. ranking Bugs related ranking tasks

Comments

@nicolehaugen
Copy link

Many exception messages thrown are unclear - as a result, when an exception occurs, it's challenging to identify whether the issue in with the ML.NET code, with the underlying data, with how the algorithm is being applied, etc. Often it takes stepping through the ML.NET fwk in attempt to get further context.

I logged this as a single issue because I think there would be benefit in looking at all places where exceptions are being thrown\rethrown to ensure that default exception messages aren't provided and that the messages are as clear\rich as possible. Let me know if you would like these broken into separate issues rather than having them combined in one.

Here are some specific examples:

  Trainer Scenario Actual Message Suggested Message
1. N/A Occurs when invalid field index is provided to the LoadColumn attribute.    For example:
[LoadColumn(100)] public uint Label { get; set; }
In the above code, the value of 100 is an invalid index value since the underlying data has less than 100 columns.
System.ArgumentNullException: 'Value cannot be null. Parameter name: items' Message should indicate which column has the issue; the reference to parameter ‘items’ is unclear.
2. N/A Occurs when Feature column is of some other type than float\single.   For example:
[ColumnName("Test"), LoadColumn(135)] public uint Test { get; set; }
System.InvalidOperationException: 'Column ‘Test’ has values of UInt32, which is not the same as earlier observed type of Single.' It’s unclear what “same as earlier observed type” means.  Consider rewording to state that the Feature columns must all be of a certain type (e.g. Single).
3. LightGbm Occurs when custom gains are specified without providing a group id column.   For example:
var customGains = new LightGbmRankingTrainer.Options();           customGains.CustomGains = new int[] { 0, 1, 2, 3 };IEstimator<ITransformer> trainer = mlContext.Ranking.Trainers.LightGbm(customGains);IEstimator<ITransformer> trainerPipeline = dataPipeline.Append(trainer);
Notice that in the above code, the Group Id isn’t being explicitly set as follows:
customGains.RowGroupColumnName = "GroupId";
System.ArgumentOutOfRangeException: 'Need a group column. Parameter name: data' ArgumentOutOfRangeException is confusing; instead, throw ArgumentNullException or InvalidOperationException.   Message should also indicate the ‘Group Id’ column is missing\null; the reference to parameter ‘data’ is unclear.
4. LightGbm Occurs when custom gains cardinality doesn’t match the cardinality of the relevance label values.     For example:
var customGains = new LightGbmRankingTrainer.Options(); customGains.CustomGains = new int[] { 0, 1, 2 };            customGains.RowGroupColumnName = "GroupId";
In the underlying data, the relevance label values are: {0, 1, 2, 3, 4 } – in other words, the cardinality of the relevance label values is greater than the specified custom gains.
System.InvalidOperationException: 'LightGBM Error, code is -1, error message is 'label (0) excel the max range 3'.' There appears to be a typo – “excel” should say “exceeds”.  Also, the message should state that the cardinality of the relevance label values must less than or equal to the cardinality of the custom gains.   Note: Refer to similar issue logged directly against LightGBM: microsoft/LightGBM#1090
@codemzs codemzs added the P1 Priority of the issue for triage purpose: Needs to be fixed soon. label Jul 12, 2019
@codemzs codemzs assigned Lynx1820 and unassigned anujshrotriya Sep 9, 2019
@Lynx1820
Copy link
Contributor

Hi Nicole. For scenario 4, please open an issue on the LighGBM repository, as the error message is actually coming from their code base.

@harishsk harishsk added ranking Bugs related ranking tasks lightgbm Bugs related lightgbm labels Apr 29, 2020
@Lynx1820
Copy link
Contributor

Summary: Scenario 1 is not reproducible, Scenario 2 and 3 were fixed. Scenario 4 is an error thrown by LightGBM, not ML.NET.

@ghost ghost locked as resolved and limited conversation to collaborators Mar 21, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
lightgbm Bugs related lightgbm P1 Priority of the issue for triage purpose: Needs to be fixed soon. ranking Bugs related ranking tasks
Projects
None yet
Development

Successfully merging a pull request may close this issue.

5 participants