Skip to content

Commit

Permalink
Fix spark regression (#830)
Browse files Browse the repository at this point in the history
Signed-off-by: Kevin Su <[email protected]>
  • Loading branch information
pingsutw authored Jan 26, 2022
1 parent 97f3882 commit c20372e
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion plugins/flytekit-spark/flytekitplugins/spark/task.py
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ def pre_execute(self, user_params: ExecutionParameters) -> ExecutionParameters:

ctx = FlyteContextManager.current_context()
sess_builder = _pyspark.sql.SparkSession.builder.appName(f"FlyteSpark: {user_params.execution_id}")
if not (ctx.execution_state and ctx.execution_state.Mode == ExecutionState.Mode.TASK_EXECUTION):
if not (ctx.execution_state and ctx.execution_state.mode == ExecutionState.Mode.TASK_EXECUTION):
# If either of above cases is not true, then we are in local execution of this task
# Add system spark-conf for local/notebook based execution.
spark_conf = _pyspark.SparkConf()
Expand Down

0 comments on commit c20372e

Please sign in to comment.