Skip to content

Commit

Permalink
[Fix](InternalSchema) Compute nodes should not be used for Internal s…
Browse files Browse the repository at this point in the history
…chema three replica (apache#36130)

## Proposed changes

Issue Number: close #xxx

When I add computing nodes to the cluster, a large number of warning
logs will appear in the FE log, as shown below:

```
2024-06-11 17:50:04,360 WARN (InternalSchemaInitializer|137) [InternalSchemaInitializer.modifyTblReplicaCount():146] Failed to scale replica of stats tbl:column_statistics to 3
org.apache.doris.common.AnalysisException: errCode = 2, detailMessage = errCode = 2, detailMessage = Failed to find enough backend, please check the replication num,replication tag and storage medium and avail capacity of backends.
Create failed replications:
replication tag: {"location" : "default"}, replication num: 3, storage medium: null
    at org.apache.doris.common.util.PropertyAnalyzer.analyzeReplicaAllocationImpl(PropertyAnalyzer.java:1217) ~[doris-fe.jar:1.2-SNAPSHOT]
    at org.apache.doris.common.util.PropertyAnalyzer.analyzeReplicaAllocation(PropertyAnalyzer.java:1136) ~[doris-fe.jar:1.2-SNAPSHOT]
    at org.apache.doris.catalog.Env.modifyTableReplicaAllocation(Env.java:4868) ~[doris-fe.jar:1.2-SNAPSHOT]
    at org.apache.doris.catalog.InternalSchemaInitializer.modifyTblReplicaCount(InternalSchemaInitializer.java:123) ~[doris-fe.jar:1.2-SNAPSHOT]
    at org.apache.doris.catalog.InternalSchemaInitializer.run(InternalSchemaInitializer.java:93) ~[doris-fe.jar:1.2-SNAPSHOT]
```


![image](https://github.com/apache/doris/assets/24907215/53ea2f34-1012-4f96-a2c9-be9c2b39b772)

This is because the computing node is also considered a replica in the
code, and the computing node does not have storage resources, resulting
in an error message.

---------

Co-authored-by: camby <[email protected]>
  • Loading branch information
2 people authored and wsjz committed Jul 15, 2024
1 parent 86e8a48 commit 43bdb01
Show file tree
Hide file tree
Showing 2 changed files with 3 additions and 3 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -109,7 +109,7 @@ public static void modifyTblReplicaCount(Database database, String tblName) {
return;
}
while (true) {
int backendNum = Env.getCurrentSystemInfo().getBackendNumFromDiffHosts(true);
int backendNum = Env.getCurrentSystemInfo().getStorageBackendNumFromDiffHosts(true);
if (FeConstants.runningUnitTest) {
backendNum = Env.getCurrentSystemInfo().getAllBackendIds().size();
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -420,11 +420,11 @@ public List<Backend> getCnBackends() {
}

// return num of backends that from different hosts
public int getBackendNumFromDiffHosts(boolean aliveOnly) {
public int getStorageBackendNumFromDiffHosts(boolean aliveOnly) {
Set<String> hosts = Sets.newHashSet();
ImmutableMap<Long, Backend> idToBackend = idToBackendRef;
for (Backend backend : idToBackend.values()) {
if (aliveOnly && !backend.isAlive()) {
if ((aliveOnly && !backend.isAlive()) || backend.isComputeNode()) {
continue;
}
hosts.add(backend.getHost());
Expand Down

0 comments on commit 43bdb01

Please sign in to comment.