Trainer estimator cleanup for FastTrees and LightGBM #1352

sfilipi · 2018-10-24T05:57:35Z

1- Adding the GroupId to the TrainerEstimatorBase class.
2- Adding the static xtensions for LightGM Multiclass and Ranking. Fixes #1314
3- Reorganizing the static and dynamic xtensions for FastTree and LightGBM

Addresses part of #1318

Adding tests, and refactoring catalog and pigsty statics

adding groupid to the trainer estimator base refactoring the catalog and static extensions for trees

rauhs · 2018-10-24T06:10:02Z

src/Microsoft.ML.Data/Training/TrainerUtils.cs

@@ -377,9 +377,10 @@ public static SchemaShape.Column MakeR4VecFeature(string featureColumn)
        /// The <see cref="SchemaShape.Column"/> for the weight column.
        /// </summary>
        /// <param name="weightColumn">name of the weight column</param>
-        public static SchemaShape.Column MakeR4ScalarWeightColumn(string weightColumn)
+        /// <param name="isExplicit">whether the column is implicitely, or explicitely defined</param>


Spelling: implicitly & explicitly #Closed

Zruty0 · 2018-10-24T16:31:50Z

src/Microsoft.ML.Data/Training/TrainerEstimatorBase.cs

+        /// <summary>
+        /// The optional groupID column that the ranking trainers expects.
+        /// </summary>
+        public readonly SchemaShape.Column GroupIdColumn;


GroupIdColumn [](start = 43, length = 13)

I am not sure I like the idea that every estimator will have the group ID column, but it only makes sense for ranking trainers. #Closed

The alternative would be to modify the role mapped data after the estimator constructs it, and inject the groupId, and that feels error-prone. It's just one reference :)

In reply to: 227866406 [](ancestors = 227866406)

So we have a TrainerEstimatorBase, is there anything that prevents us from having another subclass, if we wanted to have additional roles? We may have to change something below to be a virtual method or have some other extension point, but I feel like this would be helpful anyway.

I am not really a fan of having a single utility base class handle everything, since if it is capable of doing everything I suspect it will become too unwieldly.

In reply to: 227895594 [](ancestors = 227895594,227866406)

idk what i like less; the GroupId field here, or making MakeRoles protected virtual ... changing it anyways.

In reply to: 228312151 [](ancestors = 228312151,227895594,227866406)

Zruty0 · 2018-10-24T16:32:31Z

src/Microsoft.ML.LightGBM/LightGbmStatic.cs


-    internal static class LightGbmStaticsUtils {
+        /// <summary>
+        /// LightGbm <see cref="RankingContext"/> extension method.


LightGbm extension method. [](start = 12, length = 55)

Change the summary to look like the below summary #Resolved

still here

In reply to: 227866657 [](ancestors = 227866657)

Zruty0 · 2018-10-24T16:41:01Z

src/Microsoft.ML.LightGBM/LightGbmRankingTrainer.cs

+
+            if (advancedSettings != null)
+                CheckArgsAndAdvancedSettingMismatch(numLeaves, minDataPerLeaf, learningRate, numBoostRound, new LightGbmArguments(), Args);
+


I think there has been a misconception.

It is OK to have the parameters (like numLeaves and learningRate) available in 2 places (inside Args and as a ctor argument.
The correct pattern is

var args = new Arguments(); args.NumLeaves = numLeaves; args.LearningRate = learningRate; //... advancedSettings?.Invoke(args);

That is, we set the args' fields to the user-provided ctor arguments, and then let the user optionally change them again in the delegate: this is not a bug.

One exception is for column names: we are always ignoring the column names if they are set in the arguments. So I suggested to check (and throw) if the user sets something like FeatureColumn in the args to the value different to featureColumn ctor argument (as we will not respect that change).

Apparently, the latter is not yet done, and it's probably fine for now. But let's not check the other args. #Closed

what I have been doing, is the exact opposite, and i do prefer that tbh because of two reasons:

1- We do have parms in two places, and i think whenever we can deprecate those params from the arguments, we should do so. I don;t think we should promote the buried arguments in favor of the ones that surfaced in the signature.

2- It keeps the behavior consistent with the column names.

cc @TomFinley for his opinion.

In reply to: 227869641 [](ancestors = 227869641)

I am not a fan of enforcing a mismatch, since I could easily imagine situations where someone sets some reasonable arguments, but might thanks to some complicated logic decide to think better of it in the more complex setter.

However there is an addition point I wanted to address, since my first reaction is to disagree with it somewhat.

We do have parms in two places, and i think whenever we can deprecate those params from the arguments, we should do so.

I am strongly not in favor of this. I believe it is essential that the object with the advanced setting also continue to have the basic settings.

The reason is, a delegate for setting the object might be something used alongside the constructor, but in more programmatic, non-direct instantiation settings, especially in situations where several decisions have to be taken, it might be a real honest-to-god separate method living in a class somewhere. If the options argument has omitted the most common options, as suggested here, then we would force this scenario to spread its setting objects across two places, in a way that becomes suddenly rather more complex than just setting a bunch of properties in a single object.

A more minor consideration is that it seems more in-line with the idiom from ASP.NET, which is where @alexdegroot originally suggested we lift the idiom here.

My inclination therefore is to agree with @Zruty0 on this one, sorry @sfilipi. :)

In reply to: 227886248 [](ancestors = 227886248,227869641)

thanks for the elaborate and convincing argument @[email protected]. changing it everywhere.

In reply to: 228319239 [](ancestors = 228319239,227886248,227869641)

Zruty0

🕐

singlis · 2018-10-24T23:09:02Z

src/Microsoft.ML.Data/Training/TrainerUtils.cs

+        /// <param name="columnName">name of the weight column</param>
+        public static SchemaShape.Column MakeU4ScalarColumn(string columnName)
+        {
+            if (columnName == null)


Is this check needed? It looks like SchemaShape.Column constructor also checks the name for null or empty string. #Resolved

yep, because the check inside the constructor will throw if we pass null.

In reply to: 227989462 [](ancestors = 227989462)

singlis · 2018-10-24T23:12:12Z

src/Microsoft.ML.FastTree/TreeTrainersCatalog.cs

+        /// </summary>
+        /// <param name="ctx">The <see cref="BinaryClassificationContext"/>.</param>
+        /// <param name="label">The label column.</param>
+        /// <param name="features">The features colum.</param>


column, same on the other function comments. #Resolved

sfilipi · 2018-10-26T23:11:52Z

src/Microsoft.ML.StandardLearners/Standard/LogisticRegression/LbfgsPredictorBase.cs

            float l2Weight,
            float optimizationTolerance,
            int memorySize,
            bool enforceNoNegativity)
-            : this(env, ArgsInit(featureColumn, labelColumn, weightColumn, advancedSettings), labelColumn,
-                  l1Weight, l2Weight, optimizationTolerance, memorySize, enforceNoNegativity)
+            : this(env, new TArgs(), labelColumn,


TArgs() [](start = 27, length = 8)

this needs to get all the other parameters in #Resolved

Zruty0 · 2018-10-27T01:10:56Z

src/Microsoft.ML.FastTree/BoostingFastTree.cs

+            if (Args.LearningRates != learningRate)
+            {
+                using (var ch = Host.Start($"Setting learning rate to: {learningRate} as supplied in the direct arguments."))
+                Args.LearningRates = learningRate;


Args [](start = 16, length = 4)

indenting #Resolved

Zruty0

singlis · 2018-10-27T02:07:36Z

src/Microsoft.ML.FastTree/GamRegression.cs

@@ -51,12 +51,18 @@ internal RegressionGamTrainer(IHostEnvironment env, Arguments args)
        /// <param name="labelColumn">The name of the label column.</param>
        /// <param name="featureColumn">The name of the feature column.</param>
        /// <param name="weightColumn">The name for the column containing the initial weight.</param>
+        /// <param name="learningRate">The learning rate.</param>


minor nit, the ordering of parameters, minDocumentsInLeafs and learningRate. #Resolved

singlis · 2018-10-27T02:09:05Z

src/Microsoft.ML.FastTree/RandomForestClassification.cs

        /// <param name="weightColumn">The name for the column containing the initial weight.</param>
+        /// <param name="learningRate">The learning rate.</param>


ordering of parameters here too. #Resolved

singlis · 2018-10-27T02:10:43Z

src/Microsoft.ML.FastTree/TreeTrainersStatic.cs


-    internal class FastTreeStaticsUtils
-    {
        internal static void CheckUserValues(PipelineColumn label, Vector<float> features, Scalar<float> weights,


indent is off here... #Resolved

singlis · 2018-10-27T02:13:33Z

src/Microsoft.ML.StandardLearners/Standard/LinearClassificationTrainer.cs

        public LinearClassificationTrainer(IHostEnvironment env,
-            string featureColumn,
+        string featureColumn,


indent is off here

singlis

LGTM - I have a few minor nit picks on formatting.

sfilipi · 2018-10-27T06:00:48Z

thank you for reviewing @singlis and @Zruty0

sfilipi added 2 commits October 23, 2018 16:25

adding multiclass and ranking extensions for LightGBM.

054ed9c

Adding tests, and refactoring catalog and pigsty statics

removing duplicate method from TrainUtils

c41eb6c

adding groupid to the trainer estimator base refactoring the catalog and static extensions for trees

sfilipi requested review from Ivanidzo4ka, TomFinley, artidoro, singlis and Zruty0 October 24, 2018 05:57

rauhs reviewed Oct 24, 2018

View reviewed changes

sfilipi self-assigned this Oct 24, 2018

sfilipi added the API Issues pertaining the friendly API label Oct 24, 2018

Zruty0 reviewed Oct 24, 2018

View reviewed changes

Zruty0 suggested changes Oct 24, 2018

View reviewed changes

sfilipi added 3 commits October 24, 2018 12:01

Merge branch 'master' into trainerEstimatorCleanup

67c33ae

Addressing PR comments

a5840b2

merging from master

571b2bb

singlis reviewed Oct 24, 2018

View reviewed changes

sfilipi added 3 commits October 25, 2018 13:55

Merge branch 'master' into trainerEstimatorCleanup

57f20a7

advanced arguments override the directly provided ones.

ad51d71

merge from master

036f733

sfilipi mentioned this pull request Oct 26, 2018

FastTreeRankingTrainer expose non-advanced args(#1246) #1393

Closed

null check

e03ea66

sfilipi commented Oct 26, 2018

View reviewed changes

fixing passing down the correct column names in lbfgs predictor base.

cea3b67

Zruty0 reviewed Oct 27, 2018

View reviewed changes

Zruty0 approved these changes Oct 27, 2018

View reviewed changes

singlis reviewed Oct 27, 2018

View reviewed changes

singlis approved these changes Oct 27, 2018

View reviewed changes

sfilipi added 2 commits October 26, 2018 21:53

formatting and comments

577c9c4

fix XML

2c6d446

sfilipi merged commit 5123aee into dotnet:master Oct 27, 2018

sfilipi deleted the trainerEstimatorCleanup branch October 27, 2018 06:00

feiyun0112 mentioned this pull request Oct 28, 2018

FastTreeRanking doesn't have non-advanced args #1246

Closed

TomFinley mentioned this pull request Nov 30, 2018

Creation of components through MLContext: advanced options and other feedback #1798

Closed

ghost locked as resolved and limited conversation to collaborators Mar 27, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Trainer estimator cleanup for FastTrees and LightGBM #1352

Trainer estimator cleanup for FastTrees and LightGBM #1352

sfilipi commented Oct 24, 2018

rauhs Oct 24, 2018 •

edited by Zruty0

Loading

Zruty0 Oct 24, 2018 •

edited

Loading

sfilipi Oct 24, 2018

TomFinley Oct 25, 2018

sfilipi Oct 25, 2018

Zruty0 Oct 24, 2018 •

edited by sfilipi

Loading

Zruty0 Oct 27, 2018

Zruty0 Oct 24, 2018 •

edited

Loading

sfilipi Oct 24, 2018

TomFinley Oct 25, 2018 •

edited

Loading

sfilipi Oct 25, 2018

Zruty0 left a comment

singlis Oct 24, 2018 •

edited by sfilipi

Loading

sfilipi Oct 25, 2018

singlis Oct 24, 2018 •

edited by sfilipi

Loading

sfilipi Oct 26, 2018 •

edited

Loading

Zruty0 Oct 27, 2018 •

edited by sfilipi

Loading

Zruty0 left a comment

singlis Oct 27, 2018 •

edited by sfilipi

Loading

singlis Oct 27, 2018 •

edited by sfilipi

Loading

singlis Oct 27, 2018 •

edited by sfilipi

Loading

singlis Oct 27, 2018

singlis left a comment

sfilipi commented Oct 27, 2018


		if (advancedSettings != null)
		CheckArgsAndAdvancedSettingMismatch(numLeaves, minDataPerLeaf, learningRate, numBoostRound, new LightGbmArguments(), Args);

		/// <param name="weightColumn">The name for the column containing the initial weight.</param>
		/// <param name="learningRate">The learning rate.</param>

Trainer estimator cleanup for FastTrees and LightGBM #1352

Trainer estimator cleanup for FastTrees and LightGBM #1352

Conversation

sfilipi commented Oct 24, 2018

rauhs Oct 24, 2018 • edited by Zruty0 Loading

Choose a reason for hiding this comment

Zruty0 Oct 24, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zruty0 Oct 24, 2018 • edited by sfilipi Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zruty0 Oct 24, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

TomFinley Oct 25, 2018 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Zruty0 left a comment

Choose a reason for hiding this comment

singlis Oct 24, 2018 • edited by sfilipi Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

singlis Oct 24, 2018 • edited by sfilipi Loading

Choose a reason for hiding this comment

sfilipi Oct 26, 2018 • edited Loading

Choose a reason for hiding this comment

Zruty0 Oct 27, 2018 • edited by sfilipi Loading

Choose a reason for hiding this comment

Zruty0 left a comment

Choose a reason for hiding this comment

singlis Oct 27, 2018 • edited by sfilipi Loading

Choose a reason for hiding this comment

singlis Oct 27, 2018 • edited by sfilipi Loading

Choose a reason for hiding this comment

singlis Oct 27, 2018 • edited by sfilipi Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

singlis left a comment

Choose a reason for hiding this comment

sfilipi commented Oct 27, 2018

rauhs Oct 24, 2018 •

edited by Zruty0

Loading

Zruty0 Oct 24, 2018 •

edited

Loading

Zruty0 Oct 24, 2018 •

edited by sfilipi

Loading

Zruty0 Oct 24, 2018 •

edited

Loading

TomFinley Oct 25, 2018 •

edited

Loading

singlis Oct 24, 2018 •

edited by sfilipi

Loading

singlis Oct 24, 2018 •

edited by sfilipi

Loading

sfilipi Oct 26, 2018 •

edited

Loading

Zruty0 Oct 27, 2018 •

edited by sfilipi

Loading

singlis Oct 27, 2018 •

edited by sfilipi

Loading

singlis Oct 27, 2018 •

edited by sfilipi

Loading

singlis Oct 27, 2018 •

edited by sfilipi

Loading