[BUG] LightGBMRegressor Verbosity = 1, isProvideTrainingMetric= True do nothing

[sohaibsyed21]

SynapseML version

1.0.4

System information

• Language version (e.g. python 3.8, scala 2.12): Python 3.9
• Spark Version (e.g. 3.2.3): 3.5.1
• Spark Platform (e.g. Synapse, Databricks): Amazon EMR version 7.2.0

Describe the problem

Working on a feature selection function using LightGBMRegressor() and according to documentation there are parameters named 'verbosity' and 'isProvideTrainingMetric' that I would assume print out information for training and evaluation metrics as the model trains. However, this is not the case. There are no errors raised, simply missing output or logs that I assume should be there. An additional detail is that I am not setting up an iteration loop; this is a single pass fit.

Code to reproduce issue

    # Add a 'validation' column
    train_df_withVal = train_df.withColumn(
        'validation', 
        F.when(rand() < val_frac, 1).otherwise(0)
    )

    pandas_df_withVal = train_df_withVal.toPandas()
    pandas_df_withVal.to_excel(f"{model_output_archive_directory}/traindf_withVal.xlsx", index=False)


    model = LightGBMRegressor(**params, featuresCol='features',\
        labelCol=target_feature_name, seed=random_seed, featuresShapCol='shap_values',validationIndicatorCol='validation',verbosity=1,isProvideTrainingMetric= True).fit(train_df_withVal)

    model.saveNativeModel('/mnt/model')

Other info / logs

No response

What component(s) does this bug affect?

• area/cognitive: Cognitive project
• area/core: Core project
• area/deep-learning: DeepLearning project
• area/lightgbm: Lightgbm project
• area/opencv: Opencv project
• area/vw: VW project
• area/website: Website
• area/build: Project build system
• area/notebooks: Samples under notebooks folder
• area/docker: Docker usage
• area/models: models related issue

What language(s) does this bug affect?

• language/scala: Scala source code
• language/python: Pyspark APIs
• language/r: R APIs
• language/csharp: .NET APIs
• language/new: Proposals for new client languages

What integration(s) does this bug affect?

• integrations/synapse: Azure Synapse integrations
• integrations/azureml: Azure ML integrations
• integrations/databricks: Databricks integrations