Skip to content

Commit

Permalink
[enhance](mtmv)MTMV no longer generate cache when replaying logs (apa…
Browse files Browse the repository at this point in the history
…che#44283)

### What problem does this PR solve?

When replaying logs, cache will no longer be generated because the
catalog may not have been initialized or the dependent environment may
not be connected, causing it to freeze here and preventing FE from
starting

The cost is that after the materialized view is refreshed, the cache of
the follower node will be empty, and a cache will be generated when the
query is first used

Problem Summary:
MTMV no longer generate cache when replaying logs
  • Loading branch information
zddr authored Nov 20, 2024
1 parent ee9252a commit 946b4ae
Show file tree
Hide file tree
Showing 2 changed files with 9 additions and 4 deletions.
3 changes: 2 additions & 1 deletion fe/fe-core/src/main/java/org/apache/doris/alter/Alter.java
Original file line number Diff line number Diff line change
Expand Up @@ -1022,7 +1022,8 @@ public void processAlterMTMV(AlterMTMV alterMTMV, boolean isReplay) {
mtmv.alterMvProperties(alterMTMV.getMvProperties());
break;
case ADD_TASK:
mtmv.addTaskResult(alterMTMV.getTask(), alterMTMV.getRelation(), alterMTMV.getPartitionSnapshots());
mtmv.addTaskResult(alterMTMV.getTask(), alterMTMV.getRelation(), alterMTMV.getPartitionSnapshots(),
isReplay);
break;
default:
throw new RuntimeException("Unknown type value: " + alterMTMV.getOpType());
Expand Down
10 changes: 7 additions & 3 deletions fe/fe-core/src/main/java/org/apache/doris/catalog/MTMV.java
Original file line number Diff line number Diff line change
Expand Up @@ -190,15 +190,19 @@ public MTMVStatus alterStatus(MTMVStatus newStatus) {
}

public void addTaskResult(MTMVTask task, MTMVRelation relation,
Map<String, MTMVRefreshPartitionSnapshot> partitionSnapshots) {
Map<String, MTMVRefreshPartitionSnapshot> partitionSnapshots, boolean isReplay) {
MTMVCache mtmvCache = null;
boolean needUpdateCache = false;
if (task.getStatus() == TaskStatus.SUCCESS && !Env.isCheckpointThread()
&& !Config.enable_check_compatibility_mode) {
needUpdateCache = true;
try {
// shouldn't do this while holding mvWriteLock
mtmvCache = MTMVCache.from(this, MTMVPlanUtil.createMTMVContext(this), true);
// The replay thread may not have initialized the catalog yet to avoid getting stuck due
// to connection issues such as S3, so it is directly set to null
if (!isReplay) {
// shouldn't do this while holding mvWriteLock
mtmvCache = MTMVCache.from(this, MTMVPlanUtil.createMTMVContext(this), true);
}
} catch (Throwable e) {
mtmvCache = null;
LOG.warn("generate cache failed", e);
Expand Down

0 comments on commit 946b4ae

Please sign in to comment.