[BUG] LightGBMRegressor : 'JavaPackage' object is not callable

[shancs09]

SynapseML version

0.11.4

System information

• Language version (e.g. python 3.8, scala 2.12): 3.10.13
• Spark Version (e.g. 3.2.3): 3.3.4
• Spark Platform (e.g. Synapse, Databricks): Synapse

PySpark version: 3.3.4
Spark version: 3.3.4
SynapseML version: 0.11.4
Python version: 3.10.13 (main, Sep 11 2023, 13:44:35) [GCC 11.2.0]

Describe the problem

Trying to get start with SynapseML - But when ever we call model train function.
Getting 'JavaPackage' object is not callable error either in my local mac or remote jupyter server

Code to reproduce issue

import pyspark
spark = pyspark.sql.SparkSession.builder.appName("MyApp")
.config("spark.jars.packages", "com.microsoft.azure:synapseml_2.12:0.11.4-spark3.3")
.config("spark.jars.repositories", "https://mmlspark.azureedge.net/maven")
.getOrCreate()
import synapse.ml

train, test = (
spark.read.parquet(
"wasbs://[email protected]/BookReviewsFromAmazon10K.parquet"
)
.limit(1000)
.cache()
.randomSplit([0.8, 0.2])
)

display(train)

from pyspark.ml import Pipeline
from synapse.ml.featurize.text import TextFeaturizer
from synapse.ml.lightgbm import LightGBMRegressor

model = Pipeline(
stages=[
TextFeaturizer(inputCol="text", outputCol="features"),
LightGBMRegressor(featuresCol="features", labelCol="rating"),
]
).fit(train)

Other info / logs

---

TypeError Traceback (most recent call last)
Cell In[10], line 7
2 from synapse.ml.featurize.text import TextFeaturizer
3 from synapse.ml.lightgbm import LightGBMRegressor
5 model = Pipeline(
6 stages=[
----> 7 TextFeaturizer(inputCol="text", outputCol="features"),
8 LightGBMRegressor(featuresCol="features", labelCol="rating"),
9 ]
10 ).fit(train)

File /opt/shan/spark/python/lib/pyspark.zip/pyspark/init.py:135, in keyword_only..wrapper(self, *args, **kwargs)
133 raise TypeError("Method %s forces keyword arguments." % func.name)
134 self._input_kwargs = kwargs
--> 135 return func(self, **kwargs)

File ~/user-libs/python/synapse/ml/featurize/text/TextFeaturizer.py:106, in TextFeaturizer.init(self, java_obj, binary, caseSensitiveStopWords, defaultStopWordLanguage, inputCol, minDocFreq, minTokenLength, nGramLength, numFeatures, outputCol, stopWords, toLowercase, tokenizerGaps, tokenizerPattern, useIDF, useNGram, useStopWordsRemover, useTokenizer)
104 super(TextFeaturizer, self).init()
105 if java_obj is None:
--> 106 self._java_obj = self._new_java_obj("com.microsoft.azure.synapse.ml.featurize.text.TextFeaturizer", self.uid)
107 else:
108 self._java_obj = java_obj

File /opt/shan/spark/python/lib/pyspark.zip/pyspark/ml/wrapper.py:86, in JavaWrapper._new_java_obj(java_class, *args)
84 java_obj = getattr(java_obj, name)
85 java_args = [_py2java(sc, arg) for arg in args]
---> 86 return java_obj(*java_args)

TypeError: 'JavaPackage' object is not callable

What component(s) does this bug affect?

• area/cognitive: Cognitive project
• area/core: Core project
• area/deep-learning: DeepLearning project
• area/lightgbm: Lightgbm project
• area/opencv: Opencv project
• area/vw: VW project
• area/website: Website
• area/build: Project build system
• area/notebooks: Samples under notebooks folder
• area/docker: Docker usage
• area/models: models related issue

What language(s) does this bug affect?

• language/scala: Scala source code
• language/python: Pyspark APIs
• language/r: R APIs
• language/csharp: .NET APIs
• language/new: Proposals for new client languages

What integration(s) does this bug affect?

• integrations/synapse: Azure Synapse integrations
• integrations/azureml: Azure ML integrations
• integrations/databricks: Databricks integrations