Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Branch 22.02 merge 21.12 #1953

Merged
merged 26 commits into from
Nov 18, 2021
Merged

Branch 22.02 merge 21.12 #1953

merged 26 commits into from
Nov 18, 2021

Conversation

rlratzel
Copy link
Contributor

Manual resolve of merge commits from auto-merge #1942

rlratzel and others added 26 commits November 4, 2021 22:39
@rlratzel
Updates to support correct comparisons of cuDF Series with different …
b740286 
…names (rapidsai#1928)

cuDF PR rapidsai/cudf#9542 fixes a problem where comparisons between Series objects would return `True` even if the Series names were different, which is inconsistent with Pandas.  Some cuGraph tests relied on the old behavior which ignored names and were failing.

This PR updates those tests to use the proper comparison utility provided by cuDF to restore the intended behavior of the test assertions.

This PR also updates the `gpu` CI script to use `mamba` for an install step to hopefully speed up CI runs.

Authors:
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Joseph Nke (https://github.com/jnke2016)
  - AJ Schmidt (https://github.com/ajschmidt8)

URL: rapidsai#1928
@ChuckHastings
C API for creating a graph (rapidsai#1907)
61e8bad 
Partially addresses rapidsai#1906

This PR defines the API for graph creation and the pagerank and bfs calls that we will use to experiment with transposing a graph.

Some notes on the design here.

1. The intention is that the C API will handle renumbering (when set to true on graph creation).  This means that the opaque cugraph_graph_t pointer being populated by cugraph_sg_graph_create will contain the renumbering device vector and that the C API implementation of algorithms (pagerank and bfs demonstrated here) will unrenumber the result before returning
2. The intention is that the C API will understand whether the algorithm wants `store_transposed=true` or `store_transposed=false` and will call the transpose method if required.

Authors:
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Andrei Schaffer (https://github.com/aschaffer)
  - Rick Ratzel (https://github.com/rlratzel)
  - Seunghwa Kang (https://github.com/seunghwak)

URL: rapidsai#1907
@rlratzel
Updated error message and using a proper TypeError exception when an …
ef286cc 
…invalid MultiGraph is passed in (rapidsai#1925)

* Updated error message and using a proper TypeError exception when an invalid MultiGraph is passed in.
* Added test to verify.

This PR replaces rapidsai#1914 since that one is not passing the style check and the author has not responded.

Authors:
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)

URL: rapidsai#1925
@BradReesWork
improved Nx conversion performance (rapidsai#1921)
5052200 
refactored how to convert from Nx to cuGraph to improve performance

Authors:
  - Brad Rees (https://github.com/BradReesWork)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#1921
@jnke2016
Fix metadata mismatch (rapidsai#1920)
99daea8 
A recent change causes some of the dask's partition dataframe to have different types. 
This PR ensures the dask partition after renumbering are the same as the initial dask dataframe

Note: Still looking for the root cause of this change

Authors:
  - Joseph Nke (https://github.com/jnke2016)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Rick Ratzel (https://github.com/rlratzel)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1920
@seunghwak
Bug fix in the R-mat generator (rapidsai#1929)
3840ef6 
There was a bug in the R-mat generator that always leaves the LSB to 0.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#1929
@bdice
Use collections.abc.Sequence instead of deprecated collections.Sequen…
3cf4d86 
…ce. (rapidsai#1932)

Hi cuGraph team! I was looking at a couple things across RAPIDS related to Python 3.10 compatibility. There is one spot where the deprecated name `collections.Sequence` is used. This was moved to `collections.abc.Sequence` in Python 3.3 and has been deprecated for a while. In Python 3.10 the deprecated names for abstract base classes like `collections.Sequence` were removed.

This PR fixes that module path to help with eventual compatibility with Python 3.10+. I think this is the only instance of this deprecated name across RAPIDS packages.

(I don't have permissions to set labels or project boards, so the label checker will fail until someone else does that.)

Authors:
  - Bradley Dice (https://github.com/bdice)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#1932
@BradReesWork
Disable HITS and setup 11.5 env (rapidsai#1930)
61950dd 
Temporarily disable HITS (PR 1898 will turn it back on)

Authors:
  - Brad Rees (https://github.com/BradReesWork)
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - AJ Schmidt (https://github.com/ajschmidt8)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1930
@ChuckHastings
add new demo notebook for louvain (rapidsai#1927)
2a84341 
Added a demonstration notebook for running MNMG Louvain

Authors:
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#1927
@kaatish
HITS primitive based implementation (rapidsai#1898)
b3d14d8 
Depends on rapidsai#1903. Fixes rapidsai#1657

Authors:
  - Kumar Aatish (https://github.com/kaatish)

Approvers:
  - Andrei Schaffer (https://github.com/aschaffer)
  - Seunghwa Kang (https://github.com/seunghwak)
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1898
@dantegd
Update rapids-cmake to 21.12 (rapidsai#1931)
a6c982b 
Required to merge rapidsai/raft#370

Authors:
  - Dante Gama Dessavre (https://github.com/dantegd)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1931
@jnke2016
Ensure empty shuffled columns have the appropriate dtype (rapidsai#1926)
7b0df6e 
Workers having no data after shuffling end up holding empty columns(series) of type object which are not supported by `__cuda_array_interface__`

Ensure that the empty columns have the appropriate dtype

Authors:
  - Joseph Nke (https://github.com/jnke2016)
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#1926
@seunghwak
Additional improvements to support (key, value) pairs when E/V is sma…
df7bb9a 
…ll and P is large (rapidsai#1919)

This PR pulls updates from rapidsai#1916 & rapidsai#1917. Better be reviewed after rapidsai#1916 &rapidsai#1917 gets merged.

(key, value) pairs are still disabled (by setting the threshold value to 0) as this has performance overhead and no clear benefit in analyzing larger graphs up to 32 GPUs (unless we artificially tweak 2D partitioning of GPUs). The threshold value needs to be adjusted with larger scale testing later.

- [x] Update copy_v_transform_reduce_in_out_nbr to use minor_properties_t as a temporary buffer.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1919
@seunghwak
K-core decomposition API update (rapidsai#1924)
7bffab8 
Partially addresses rapidsai#1909.

- [x] Add a K-core  function declaration.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#1924
@seunghwak
Add options to drop self-loops & multi_edges in C++ test graph genera…
0730fea 
…tion (rapidsai#1934)

Necessary for rapidsai#1909 

Add options to filter out self-loops & multi-edges after edge generation and before graph creation in C++ test graph generation API.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1934
@seunghwak
Count self-loops and multi-edges (rapidsai#1939)
0678065 
Close rapidsai#537

Count the number of self-loops & multi-edges.

- [x] API update
- [x] Implement count self-loops
- [x] Implement count multi-edges
- [x] Add C++ test suites.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Kumar Aatish (https://github.com/kaatish)

URL: rapidsai#1939
@rlratzel
Re-enable HITS in the python API using the new primitive-based implem…
2770c87 
…entation (rapidsai#1941)

Followup PR to rapidsai#1930

This PR re-enables HITS in the python API using the new C++ primitive-based implementation. This also refactors the tests to use the benchmark fixture plugin, and adds an additional dataset to read for more diverse comparison to Nx.

![image](https://user-images.githubusercontent.com/3039903/141236018-16557063-9d2a-4fd7-b8c1-789f78958ea7.png)

Authors:
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Brad Rees (https://github.com/BradReesWork)
  - Joseph Nke (https://github.com/jnke2016)

URL: rapidsai#1941
@divyegala
Accounting for raft::random detail changes (rapidsai#1937)
fda2466 
Depends on rapidsai/raft#356

Authors:
  - Divye Gala (https://github.com/divyegala)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Seunghwa Kang (https://github.com/seunghwak)

URL: rapidsai#1937
@seunghwak
K-core implementation for undirected graphs (rapidsai#1933)
5db6636 
Close rapidsai#1909.

This PR aims to implement K-core only for undirected graphs.

- [x] Implement K-core implementation
- [x] Test/debug on SG
- [x] Test/debug on MG

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Kumar Aatish (https://github.com/kaatish)
  - Divye Gala (https://github.com/divyegala)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Kumar Aatish (https://github.com/kaatish)

URL: rapidsai#1933
@jnke2016
Enforce renumbering for MNMG algos (rapidsai#1943)
5c8a5b7 
A bug was reported when calling a MNMG algo with unrenumbered edge list. 
This PR returns an error if `renumber=False `

Change the algo name for asv reporting 

closes rapidsai#1936

Authors:
  - Joseph Nke (https://github.com/jnke2016)

Approvers:
  - Rick Ratzel (https://github.com/rlratzel)

URL: rapidsai#1943
@ChuckHastings
C API Create Graph Implementation (rapidsai#1940)
15be981 
Implementation of the following capabilities into the C API:
* Graph creation
* Pagerank
* BFS

Note that the C API in this implementation supports:
* Renumbering in the C layer
* Transposing the graph if necessary to support the desired algorithm

Closes rapidsai#1906

Authors:
  - Chuck Hastings (https://github.com/ChuckHastings)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Kumar Aatish (https://github.com/kaatish)

URL: rapidsai#1940
@seunghwak
Add a new graph primitive to filter edges (extract_if_e) (rapidsai#1938)
8096c53 
Close rapidsai#1911

- [x] Implement the new primitive extract_if_e to extract an edge list (to construct a subgraph) by filtering the edges with ```e_op```
- [x] Implement test suites.

Authors:
  - Seunghwa Kang (https://github.com/seunghwak)
  - Divye Gala (https://github.com/divyegala)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1938
@ChuckHastings
initial creation of libcugraph_etl.so (rapidsai#1885)
e5b024e 
The new renumbering implementation will require C++ integration directly with cudf.  In order to facilitate that, but also support our customers that won't need cudf, this PR will create a separate library (`libcugraph_etl.so`) which will ultimately link with `libcudf.so` and contain the ETL portions of cugraph that require cudf features.

This way our other libcugraph customers that don't need to reference the new library will not need to install all of the cudf dependencies.

To seed this, the PR also includes a proposed API for the new renumbering capability.

Authors:
  - Chuck Hastings (https://github.com/ChuckHastings)
  - Rick Ratzel (https://github.com/rlratzel)

Approvers:
  - Seunghwa Kang (https://github.com/seunghwak)
  - https://github.com/chirayuG-nvidia
  - Rick Ratzel (https://github.com/rlratzel)
  - Jordan Jacobelli (https://github.com/Ethyling)

URL: rapidsai#1885
@ajschmidt8
Update conda recipes for Enhanced Compatibility effort (rapidsai#1889)
fd195b5 
This PR updates the `conda` recipe build strings and `cudatoolkit` version specifications as part of the Enhanced Compatibility efforts.

The build strings have been updated to only include the major CUDA version (i.e. `librmm-21.12.00a-cuda11_gc781527_12.tar.bz2`) and the `cudatoolkit` version specifications will now be formatted like `cudatoolkit >=x,<y.0a0` (i.e. `cudatoolkit >=11,<12.0a0`).

Moving forward, we'll build the packages with a single CUDA version (i.e. `11.4`) and test them in environments with different CUDA versions (i.e. `11.0`, `11.2`, `11.4`, etc.).

Authors:
  - AJ Schmidt (https://github.com/ajschmidt8)

Approvers:
  - Ray Douglass (https://github.com/raydouglass)

URL: rapidsai#1889
@robertmaynard
cugraph bring in the same cuco as raft and cudf (rapidsai#1945)
708c319 
Authors:
  - Robert Maynard (https://github.com/robertmaynard)

Approvers:
  - Chuck Hastings (https://github.com/ChuckHastings)

URL: rapidsai#1945
@rlratzel
Merge branch-21.12 into branch-22.02
75e1dd9
@rlratzel rlratzel added improvement Improvement / enhancement to an existing function non-breaking Non-breaking change labels Nov 18, 2021
@rlratzel rlratzel self-assigned this Nov 18, 2021
@rlratzel rlratzel requested a review from a team as a code owner November 18, 2021 00:18
@rlratzel rlratzel requested review from a team as code owners November 18, 2021 00:18
@review-notebook-app
Copy link

Check out this pull request on  ReviewNB

See visual diffs & provide feedback on Jupyter Notebooks.

Powered by ReviewNB

@ajschmidt8 ajschmidt8 merged commit ee2d9b5 into rapidsai:branch-22.02 Nov 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees

@rlratzel rlratzel

Labels
improvement Improvement / enhancement to an existing function non-breaking Non-breaking change
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.