Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(algorithm): support single source shortest path algorithm #285

Merged
merged 24 commits into from
Mar 12, 2024
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
28691fd
feat: ListValue add clear() method
diaohancai Nov 21, 2023
d9c5c84
feat: Source Target Shortest Path
diaohancai Nov 21, 2023
3aff95d
test: test data is complicated
diaohancai Nov 21, 2023
b6c0f4c
test: ListValue.clear unit test
diaohancai Nov 21, 2023
f5a5de9
fix: ring loop
diaohancai Nov 21, 2023
4c37b2d
optimize: message combine
diaohancai Nov 21, 2023
15f95f6
Merge branch 'master' into pr/285
imbajin Dec 4, 2023
6630b82
refactor(algorithm): Single Source Shortest Path
diaohancai Dec 26, 2023
9524900
refactor(algorithm): output json
diaohancai Dec 26, 2023
2804bd7
feat(algorithm): multiple target optimization
diaohancai Dec 26, 2023
cdaba23
feat(core): IdList Merge Combiner
diaohancai Dec 26, 2023
125bba6
chore(algorithm): simple adjustments
diaohancai Dec 31, 2023
9587802
optimization(algorithm): change reachedTargets from IdList to IdSet
diaohancai Jan 1, 2024
01cde50
chore: json style key
diaohancai Jan 22, 2024
6b42bf5
improve: convert id from string to ID with type
diaohancai Jan 25, 2024
55b0ed3
chore: add IdUtilTest unit test
diaohancai Jan 25, 2024
2828bb7
fix: all targets reached
diaohancai Feb 6, 2024
a0a0103
improve: remove unnecessary member var
diaohancai Feb 6, 2024
c4ca8fe
improve: source vertex and target vertex specify idType
diaohancai Feb 6, 2024
174584e
chore: get properties
diaohancai Feb 7, 2024
aa8238d
improve: input vertex id parse
diaohancai Feb 29, 2024
0b1c720
chore: log improvement
diaohancai Feb 29, 2024
6510424
test: apply exception testing
diaohancai Mar 3, 2024
d55c7c7
test: parse empty id throws IllegalArgumentException
diaohancai Mar 3, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,10 @@
package org.apache.hugegraph.computer.algorithm.path.shortest;

import java.util.Arrays;
import java.util.HashSet;
import java.util.Iterator;
import java.util.List;
import java.util.Set;
import java.util.stream.Collectors;

import org.apache.commons.lang3.StringUtils;
Expand All @@ -28,7 +30,7 @@
import org.apache.hugegraph.computer.core.graph.edge.Edge;
import org.apache.hugegraph.computer.core.graph.id.Id;
import org.apache.hugegraph.computer.core.graph.value.DoubleValue;
import org.apache.hugegraph.computer.core.graph.value.IdList;
import org.apache.hugegraph.computer.core.graph.value.IdSet;
import org.apache.hugegraph.computer.core.graph.value.Value;
import org.apache.hugegraph.computer.core.graph.vertex.Vertex;
import org.apache.hugegraph.computer.core.worker.Computation;
Expand All @@ -55,12 +57,17 @@

/**
* target vertex id.
* 1. single target
* 2. multiple target: comma separated
* 1. single target: one vertex id
* 2. multiple target: multiple vertex ids separated by comma
* 3. all: *
*/
private String targetId;

/**
* cache of targetId
*/
private Set<String> targetIdSet;
diaohancai marked this conversation as resolved.
Show resolved Hide resolved

/**
* target quantity type
*/
Expand All @@ -80,9 +87,9 @@

//****************** global data ******************//
/**
* reach target
* reached targets
*/
private IdList reachTarget;
private IdSet reachedTargets;

@Override
public String category() {
Expand All @@ -98,31 +105,27 @@
public void init(Config config) {
this.sourceId = config.getString(OPTION_SOURCE_ID, "");
if (StringUtils.isBlank(this.sourceId)) {
throw new ComputerException("The param '%s' must not be blank", OPTION_SOURCE_ID);

Check warning on line 108 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L108

Added line #L108 was not covered by tests
}

this.targetId = config.getString(OPTION_TARGET_ID, "");
if (StringUtils.isBlank(this.targetId)) {
throw new ComputerException("The param '%s' must not be blank", OPTION_TARGET_ID);

Check warning on line 113 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L113

Added line #L113 was not covered by tests
}

if (this.targetId.equals("*")) {
this.targetQuantityType = QuantityType.ALL;
} else if (this.targetId.split(",").length > 1) {
// remove spaces
this.targetId = Arrays.stream(this.targetId.split(","))
.map(e -> e.trim())
.collect(Collectors.joining(","));
this.targetQuantityType = QuantityType.MULTIPLE;
} else {
this.targetQuantityType = QuantityType.SINGLE;
}
// remove spaces
this.targetId = Arrays.stream(this.targetId.split(","))
.map(e -> e.trim())
.collect(Collectors.joining(","));
// cache targetId
this.targetIdSet = new HashSet<>(Arrays.asList(this.targetId.split(",")));
this.targetQuantityType = this.getQuantityType();

this.weightProperty = config.getString(OPTION_WEIGHT_PROPERTY, "");

this.defaultWeight = config.getDouble(OPTION_DEFAULT_WEIGHT, 1);
if (this.defaultWeight <= 0) {
throw new ComputerException("The param '%s' must be greater than 0, " +

Check warning on line 128 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L128

Added line #L128 was not covered by tests
"actual got '%s'",
OPTION_DEFAULT_WEIGHT, this.defaultWeight);
}
Expand All @@ -142,19 +145,19 @@
value.zeroDistance(); // source vertex

// single target && source vertex == target vertex
if (this.targetQuantityType.equals(QuantityType.SINGLE) &&
if (this.targetQuantityType == QuantityType.SINGLE &&
this.sourceId.equals(this.targetId)) {
LOG.debug("source vertex {} equals target vertex {}", this.sourceId, this.targetId);
vertex.inactivate();
return;

Check warning on line 152 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L150-L152

Added lines #L150 - L152 were not covered by tests
}

if (vertex.numEdges() <= 0) {
// isolated vertex
LOG.debug("source vertex {} can not reach target vertex {}",

Check warning on line 157 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L157

Added line #L157 was not covered by tests
this.sourceId, this.targetId);
vertex.inactivate();
return;

Check warning on line 160 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L159-L160

Added lines #L159 - L160 were not covered by tests
}

vertex.edges().forEach(edge -> {
Expand All @@ -170,9 +173,9 @@
@Override
public void compute(ComputationContext context, Vertex vertex,
Iterator<SingleSourceShortestPathValue> messages) {
if (this.isTarget(vertex) && !this.reachTarget.contains(vertex.id())) {
// reach target
this.reachTarget.add(vertex.id());
if (this.isTarget(vertex) && !this.reachedTargets.contains(vertex.id())) {
// reached targets
this.reachedTargets.add(vertex.id());
}

while (messages.hasNext()) {
Expand All @@ -187,7 +190,7 @@
}

// reach all target or nowhere to go
if (this.isAllTargetReach(vertex) || vertex.numEdges() <= 0) {
if (this.isAllTargetsReached(vertex) || vertex.numEdges() <= 0) {
continue;
}

Expand All @@ -205,15 +208,28 @@

@Override
public void beforeSuperstep(WorkerContext context) {
this.reachTarget = context.aggregatedValue(
SingleSourceShortestPathMaster.SINGLE_SOURCE_SHORTEST_PATH_REACH_TARGET);
this.reachedTargets = context.aggregatedValue(
SingleSourceShortestPathMaster.SINGLE_SOURCE_SHORTEST_PATH_REACHED_TARGETS);
}

@Override
public void afterSuperstep(WorkerContext context) {
context.aggregateValue(
SingleSourceShortestPathMaster.SINGLE_SOURCE_SHORTEST_PATH_REACH_TARGET,
this.reachTarget);
SingleSourceShortestPathMaster.SINGLE_SOURCE_SHORTEST_PATH_REACHED_TARGETS,
this.reachedTargets);
}

/**
* get QuantityType by this.targetId
*/
private QuantityType getQuantityType() {
if (this.targetId.equals("*")) {
return QuantityType.ALL;

Check warning on line 227 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L227

Added line #L227 was not covered by tests
} else if (this.targetId.contains(",")) {
return QuantityType.MULTIPLE;

Check warning on line 229 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L229

Added line #L229 was not covered by tests
} else {
return QuantityType.SINGLE;
}
}

/**
Expand All @@ -232,16 +248,16 @@
Value property = edge.property(this.weightProperty);
if (property != null) {
if (!property.isNumber()) {
throw new ComputerException("The value of %s must be a numeric value, " +

Check warning on line 251 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L251

Added line #L251 was not covered by tests
"actual got '%s'",
this.weightProperty, property.string());

Check warning on line 253 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L253

Added line #L253 was not covered by tests
}

weight = ((DoubleValue) property).doubleValue();
if (weight <= 0) {
throw new ComputerException("The value of %s must be greater than 0, " +

Check warning on line 258 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L258

Added line #L258 was not covered by tests
"actual got '%s'",
this.weightProperty, property.string());

Check warning on line 260 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L260

Added line #L260 was not covered by tests
}
}
return weight;
Expand All @@ -251,42 +267,29 @@
* determine whether vertex is one of the target
*/
private boolean isTarget(Vertex vertex) {
if (this.targetQuantityType.equals(QuantityType.SINGLE) &&
this.idEquals(vertex, this.targetId)) {
return true;
}

if (this.targetQuantityType.equals(QuantityType.MULTIPLE)) {
for (String targetId : this.targetId.split(",")) {
if (this.idEquals(vertex, targetId)) {
return true;
}
}
}
return false;
return this.targetIdSet.contains(vertex.id().toString());
}

/**
* determine whether reach all target
* determine whether all targets reached
*/
private boolean isAllTargetReach(Vertex vertex) {
if (this.targetQuantityType.equals(QuantityType.SINGLE) &&
private boolean isAllTargetsReached(Vertex vertex) {
if (this.targetQuantityType == QuantityType.SINGLE &&
this.idEquals(vertex, this.targetId)) {
return true;
}

if (this.targetQuantityType.equals(QuantityType.MULTIPLE)) {
String[] targetIds = this.targetId.split(",");
if (targetIds.length == this.reachTarget.size()) {
List<String> reachTargets = this.reachTarget.values()
.stream().map(Id::toString)
.collect(Collectors.toList());
for (String targetId : targetIds) {
if (!reachTargets.contains(targetId)) {
if (this.targetQuantityType == QuantityType.MULTIPLE) {
if (this.targetIdSet.size() == this.reachedTargets.size()) {
List<String> reachedTargets = this.reachedTargets.value()
diaohancai marked this conversation as resolved.
Show resolved Hide resolved
.stream().map(Id::toString)
.collect(Collectors.toList());

Check warning on line 286 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L284-L286

Added lines #L284 - L286 were not covered by tests
for (String targetId : this.targetIdSet) {
if (!reachedTargets.contains(targetId)) {
return false;

Check warning on line 289 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L289

Added line #L289 was not covered by tests
}
}
return true;

Check warning on line 292 in computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java

View check run for this annotation

Codecov / codecov/patch

computer-algorithm/src/main/java/org/apache/hugegraph/computer/algorithm/path/shortest/SingleSourceShortestPath.java#L291-L292

Added lines #L291 - L292 were not covered by tests
}
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,22 +17,22 @@

package org.apache.hugegraph.computer.algorithm.path.shortest;

import org.apache.hugegraph.computer.core.combiner.IdListMergeCombiner;
import org.apache.hugegraph.computer.core.combiner.IdSetMergeCombiner;
import org.apache.hugegraph.computer.core.graph.value.ValueType;
import org.apache.hugegraph.computer.core.master.MasterComputation;
import org.apache.hugegraph.computer.core.master.MasterComputationContext;
import org.apache.hugegraph.computer.core.master.MasterContext;

public class SingleSourceShortestPathMaster implements MasterComputation {

public static final String SINGLE_SOURCE_SHORTEST_PATH_REACH_TARGET =
"single_source_shortest_path.reach_target";
public static final String SINGLE_SOURCE_SHORTEST_PATH_REACHED_TARGETS =
"single_source_shortest_path.reached_targets";

@Override
public void init(MasterContext context) {
context.registerAggregator(SINGLE_SOURCE_SHORTEST_PATH_REACH_TARGET,
ValueType.ID_LIST,
IdListMergeCombiner.class);
context.registerAggregator(SINGLE_SOURCE_SHORTEST_PATH_REACHED_TARGETS,
ValueType.ID_SET,
IdSetMergeCombiner.class);
}

@Override
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with this
* work for additional information regarding copyright ownership. The ASF
* licenses this file to You under the Apache License, Version 2.0 (the
* "License"); you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
* WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
* License for the specific language governing permissions and limitations
* under the License.
*/

package org.apache.hugegraph.computer.core.combiner;

import org.apache.hugegraph.computer.core.graph.value.IdSet;

public class IdSetMergeCombiner implements Combiner<IdSet> {

@Override
public void combine(IdSet v1, IdSet v2, IdSet result) {
// merge
result.addAll(v1);
result.addAll(v2);
}
}
Original file line number Diff line number Diff line change
Expand Up @@ -107,4 +107,8 @@
public int compareTo(Value o) {
throw new UnsupportedOperationException("compareTo");
}

public int size() {
return this.values.size();

Check warning on line 112 in computer-api/src/main/java/org/apache/hugegraph/computer/core/graph/value/IdSet.java

View check run for this annotation

Codecov / codecov/patch

computer-api/src/main/java/org/apache/hugegraph/computer/core/graph/value/IdSet.java#L112

Added line #L112 was not covered by tests
}
}
Loading