Skip to content

Commit

Permalink
Add a flag for HttpRemoteTask to avoid eager but unnecessary sendUpdate
Browse files Browse the repository at this point in the history
It should not sendUpdate before task been started,
while the needsUpdate flag in RemoteHttpTask is not enough to ensure this,
which case will happen on all non-leaf stage, as 450-470 lines in
SqlStageExecution.java
Although task will start soon after noMoreSplits, but we should avoid
this, which is unnecessary and not strictly accurate in principle.
  • Loading branch information
guhanjie authored and sopel39 committed Apr 14, 2021
1 parent c9b64f0 commit 6968386
Showing 1 changed file with 3 additions and 1 deletion.
Original file line number Diff line number Diff line change
Expand Up @@ -159,6 +159,7 @@ public final class HttpRemoteTask

private final PartitionedSplitCountTracker partitionedSplitCountTracker;

private final AtomicBoolean started = new AtomicBoolean(false);
private final AtomicBoolean aborting = new AtomicBoolean(false);

public HttpRemoteTask(
Expand Down Expand Up @@ -318,6 +319,7 @@ public void start()
{
try (SetThreadName ignored = new SetThreadName("HttpRemoteTask-%s", taskId)) {
// to start we just need to trigger an update
started.set(true);
scheduleUpdate();

dynamicFiltersFetcher.start();
Expand Down Expand Up @@ -513,7 +515,7 @@ private synchronized void sendUpdate()
{
TaskStatus taskStatus = getTaskStatus();
// don't update if the task hasn't been started yet or if it is already finished
if (!needsUpdate.get() || taskStatus.getState().isDone()) {
if (!started.get() || !needsUpdate.get() || taskStatus.getState().isDone()) {
return;
}

Expand Down

0 comments on commit 6968386

Please sign in to comment.