Skip to content

Commit

Permalink
saving signature only for pandas udf, removed useless try/except
Browse files Browse the repository at this point in the history
  • Loading branch information
e-dorigatti committed May 29, 2018
1 parent f7b53c2 commit 8fac2a8
Showing 1 changed file with 6 additions and 6 deletions.
12 changes: 6 additions & 6 deletions python/pyspark/sql/udf.py
Original file line number Diff line number Diff line change
Expand Up @@ -159,13 +159,13 @@ def _create_judf(self):

func = fail_on_stopiteration(self.func)

# prevent inspect to fail
# e.g. inspect.getargspec(sum) raises
# TypeError: <built-in function sum> is not a Python function
try:
# for pandas UDFs the worker needs to know if the function takes
# one or two arguments, but the signature is lost when wrapping with
# fail_on_stopiteration, so we store it here
if self.evalType in (PythonEvalType.SQL_SCALAR_PANDAS_UDF,
PythonEvalType.SQL_GROUPED_MAP_PANDAS_UDF,
PythonEvalType.SQL_GROUPED_AGG_PANDAS_UDF):
func._argspec = _get_argspec(self.func)
except TypeError:
pass

wrapped_func = _wrap_function(sc, func, self.returnType)
jdt = spark._jsparkSession.parseDataType(self.returnType.json())
Expand Down

0 comments on commit 8fac2a8

Please sign in to comment.