[Model] Update `CuGraphRelGraphConv` to use `pylibcugraphops=23.02` #5217

tingyu66 · 2023-01-20T03:41:34Z

Description

This PR update CuGraphRelGraphConv module to use pylibcugraphops 23.02.

With pylibcugraphops now offering aggregation autograd functions, we tidy up the source file to only include the nn.Module and make a few improvements.

Detailed changes include:

support apply_norm option that enables normalized aggregation
self loop weight is fused into the the weights W for better performance
move max_in_degree to forward(), since it is a property of the graph not the model
support full graph input (we only support sampled graph before)
improve the test and update the example

Checklist

Please feel free to remove inapplicable items for your PR.

The PR title starts with [$CATEGORY] (such as [NN], [Model], [Doc], [Feature]])
I've leverage the tools to beautify the python and c++ code.
The PR is complete and small, read the Google eng practice (CL equals to PR) to understand more about small PR. In DGL, we consider PRs with less than 200 lines of core code change are small (example, test and documentation could be exempted).
All changes have test coverage
Code is well-documented
To the best of my knowledge, examples are either not affected by this change, or have been fixed to be compatible with this change
Related issue is referred in this PR
If the PR is for a new model/paper, I've updated the example index here.

Changes

dgl-bot · 2023-01-20T05:16:23Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-01-20T05:16:35Z

Commit ID: 3369346191274395137e71a3933415ed50e99df9

Build ID: 1

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-01-26T17:16:44Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-01-26T17:16:56Z

Commit ID: 6e409d7efddb7e4027993c318fd6212e5c659992

Build ID: 2

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-02-02T02:21:19Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-02T02:21:31Z

Commit ID: 59708d281a9f2c96ede8bc7208088b725e1f19cf

Build ID: 3

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

mufeili · 2023-02-02T05:17:02Z

@dgl-bot

mufeili · 2023-02-02T05:25:24Z

Are there breaking changes?

mufeili · 2023-02-02T05:28:23Z

python/dgl/nn/pytorch/conv/cugraph_relgraphconv.py

@@ -214,87 +87,68 @@ def __init__(
        regularizer=None,
        num_bases=None,
        bias=True,
-        activation=None,


Is this a breaking change?

Yes. I saw the this comment from RelGraphConv:

dgl/python/dgl/nn/pytorch/conv/relgraphconv.py

Lines 128 to 129 in 76bb540

# TODO(minjie): consider remove those options in the future to make

# the module only about graph convolution.

I also prefer to apply activation outside of the conv layer.

mufeili · 2023-02-02T05:28:55Z

python/dgl/nn/pytorch/conv/cugraph_relgraphconv.py

    ):
        if has_pylibcugraphops is False:
            raise ModuleNotFoundError(
-                "dgl.nn.CuGraphRelGraphConv requires pylibcugraphops "
+                "dgl.nn.CuGraphRelGraphConv requires pylibcugraphops >= 23.02 "


Is there a way to check the version number of the package?

https://github.com/rapidsai/dgl/blob/785e294b6e5a596df9c669eeb6ca56672a23d002/python/dgl/nn/pytorch/conv/cugraph_relgraphconv.py#L11-L13

Any version <23.02 does not have pylibcugraphops.torch.autograd API, and thus will throw an exception here.

dgl-bot · 2023-02-02T06:16:45Z

Commit ID: 0a3f04d268015ce04085ce6782fe477077ded4bb

Build ID: 4

Status: ✅ CI test succeeded

Report path: link

Full logs path: link

mufeili · 2023-02-02T06:17:40Z

python/dgl/nn/pytorch/conv/cugraph_relgraphconv.py

    self_loop : bool, optional
        True to include self loop message. Default: ``True``.
    dropout : float, optional
        Dropout rate. Default: ``0.0``.
-    layer_norm : bool, optional


Is this a breaking change?

Yes, same as the "activation" argument above.

mufeili · 2023-02-02T06:19:09Z

python/dgl/nn/pytorch/conv/cugraph_relgraphconv.py

@@ -309,57 +163,58 @@ def forward(self, g, feat, etypes, norm=None):
            so any input of other integer types will be casted into int32,
            thus introducing some overhead. Pass in int32 tensors directly
            for best performance.
-        norm : torch.Tensor, optional


Is this replaced by apply_norm=True?

Yes. This is an additional feature from CuGraphRelGraphConv: supporting the normalized aggregation presented in the RGCN paper.

Hence, users no longer need to compute the norm during training:

dgl/examples/pytorch/rgcn/entity_sample.py

Lines 60 to 61 in 465828c

for block in blocks:

block.edata['norm'] = dgl.norm_by_dst(block).unsqueeze(1)

mufeili · 2023-02-02T06:20:06Z

python/dgl/nn/pytorch/conv/cugraph_relgraphconv.py

-            A 1D tensor of edge norm value.  Shape: :math:`(|E|,)`.
+        max_in_degree : int, optional
+            Maximum in-degree of destination nodes. It is only effective when
+            :attr:`g` is a :class:`DGLBlock`, i.e., bipartite graph. When


Is this still only valid for DGLBlock?

mufeili · 2023-02-02T07:06:59Z

tests/cugraph/cugraph-ops/test_cugraph_relgraphconv.py

-import dgl
-from dgl.nn import CuGraphRelGraphConv
-from dgl.nn import RelGraphConv
+from dgl.nn import CuGraphRelGraphConv, RelGraphConv

 # TODO(tingyu66): Re-enable the following tests after updating cuGraph CI image.


Is this now re-enabled?

Not yet, I will create a new image after our 23.02 release and update these pytest markers in a separate PR.

mufeili · 2023-02-02T07:07:40Z

examples/advanced/cugraph/rgcn.py

@@ -8,19 +8,20 @@
 code changes from the current `entity_sample.py` example.


Did you see similar performance numbers after running this script?

Yes, performance is the same as before.

mufeili

done a pass

dgl-bot · 2023-02-10T19:16:57Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-10T19:17:09Z

Commit ID: 514aacf4df8715b75e2c064c94549a68061a2de0

Build ID: 5

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

tingyu66 · 2023-02-11T02:07:31Z

Are there breaking changes?

Apologies for the late reply. The norm option is now replaced by apply_norm. Other breaking changes include the removal of activation and layer_norm options.

dgl-bot · 2023-02-13T05:27:53Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-13T05:28:05Z

Commit ID: a03d0b3

Build ID: 6

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

dgl-bot · 2023-02-15T04:54:53Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-15T04:55:05Z

Commit ID: 474f8d2

Build ID: 7

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

mufeili · 2023-02-15T06:31:51Z

@dgl-bot

dgl-bot · 2023-02-15T07:57:07Z

Commit ID: 474f8d2

Build ID: 8

Status: ❌ CI test failed in Stage [Distributed Torch CPU Unit test].

Report path: link

Full logs path: link

dgl-bot · 2023-02-15T07:58:34Z

Not authorized to trigger CI. Please ask core developer to help trigger via issuing comment:

@dgl-bot

dgl-bot · 2023-02-15T07:58:46Z

Commit ID: 865a0ca

Build ID: 9

Status: ❌ CI test failed in Stage [Authentication].

Report path: link

Full logs path: link

mufeili · 2023-02-15T07:58:55Z

@dgl-bot

dgl-bot · 2023-02-15T09:41:13Z

Commit ID: 865a0ca

Build ID: 10

Status: ✅ CI test succeeded

Report path: link

Full logs path: link

…mlc#5217) * update cugraph_relgraphconv * update equality test * update cugraph rgcn example * update RelGraphConvAgg based on latest API changes * enable fallback option to fg when fanout is large --------- Co-authored-by: Mufei Li <[email protected]>

tingyu66 added 3 commits January 19, 2023 18:44

update cugraph_relgraphconv

785e294

update equality test

5644079

update cugraph rgcn example

7904efd

tingyu66 mentioned this pull request Jan 20, 2023

Add/Update cugraph-ops models #5218

Closed

update RelGraphConvAgg based on latest API changes

88da020

tingyu66 changed the title ~~[Do not merge][Model] Update CuGraphRelGraphConv to use pylibcugraphops=23.02~~ [Model] Update CuGraphRelGraphConv to use pylibcugraphops=23.02 Feb 2, 2023

rudongyu requested a review from mufeili February 2, 2023 03:01

mufeili reviewed Feb 2, 2023

View reviewed changes

enable fallback option to fg when fanout is large

53e4f11

Merge branch 'master' into update-cugraph-relgraphconv

a03d0b3

mufeili approved these changes Feb 13, 2023

View reviewed changes

Merge branch 'master' into update-cugraph-relgraphconv

474f8d2

Merge branch 'master' into update-cugraph-relgraphconv

865a0ca

mufeili merged commit 19b3cea into dmlc:master Feb 15, 2023

tingyu66 deleted the update-cugraph-relgraphconv branch February 15, 2023 14:02

	# TODO(minjie): consider remove those options in the future to make
	# the module only about graph convolution.

	for block in blocks:
	block.edata['norm'] = dgl.norm_by_dst(block).unsqueeze(1)

		@@ -8,19 +8,20 @@
		code changes from the current `entity_sample.py` example.

[Model] Update CuGraphRelGraphConv to use pylibcugraphops=23.02 #5217

[Model] Update CuGraphRelGraphConv to use pylibcugraphops=23.02 #5217

Conversation

tingyu66 commented Jan 20, 2023

Description

Checklist

Changes

dgl-bot commented Jan 20, 2023

dgl-bot commented Jan 20, 2023

dgl-bot commented Jan 26, 2023

dgl-bot commented Jan 26, 2023

dgl-bot commented Feb 2, 2023

dgl-bot commented Feb 2, 2023

mufeili commented Feb 2, 2023

mufeili commented Feb 2, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dgl-bot commented Feb 2, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mufeili left a comment

Choose a reason for hiding this comment

dgl-bot commented Feb 10, 2023

dgl-bot commented Feb 10, 2023

tingyu66 commented Feb 11, 2023

dgl-bot commented Feb 13, 2023

dgl-bot commented Feb 13, 2023

dgl-bot commented Feb 15, 2023

dgl-bot commented Feb 15, 2023

mufeili commented Feb 15, 2023

dgl-bot commented Feb 15, 2023

dgl-bot commented Feb 15, 2023

dgl-bot commented Feb 15, 2023

mufeili commented Feb 15, 2023

dgl-bot commented Feb 15, 2023

[Model] Update `CuGraphRelGraphConv` to use `pylibcugraphops=23.02` #5217

[Model] Update `CuGraphRelGraphConv` to use `pylibcugraphops=23.02` #5217