Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Max auc #1513

Merged
merged 2 commits into from
Apr 14, 2022
Merged

Max auc #1513

merged 2 commits into from
Apr 14, 2022

Conversation

jperez999
Copy link
Contributor

This grabs max auc from hugectr training in criteo.

@jperez999 jperez999 self-assigned this Apr 14, 2022
@jperez999 jperez999 added the enhancement New feature or request label Apr 14, 2022
@jperez999 jperez999 added this to the Merlin 22.05 milestone Apr 14, 2022
@github-actions
Copy link

Documentation preview

https://nvidia-merlin.github.io/NVTabular/review/pr-1513

@karlhigley karlhigley requested a review from benfred April 14, 2022 18:05
@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #1513 of commit 1e2a1119f6947b998535ec3c9cea6835367a2574, no merge conflicts.
Running as SYSTEM
Setting status of 1e2a1119f6947b998535ec3c9cea6835367a2574 to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4435/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1513/*:refs/remotes/origin/pr/1513/* # timeout=10
 > git rev-parse 1e2a1119f6947b998535ec3c9cea6835367a2574^{commit} # timeout=10
Checking out Revision 1e2a1119f6947b998535ec3c9cea6835367a2574 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 1e2a1119f6947b998535ec3c9cea6835367a2574 # timeout=10
Commit message: "max auc recorded now"
 > git rev-list --no-walk 069a098c226156d1544971c418f4527e87c7f8f7 # timeout=10
First time build. Skipping changelog.
[nvtabular_tests] $ /bin/bash /tmp/jenkins1277188542896878241.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.2)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+44.g50eb7ef1
Uninstalling nvtabular-0.11.0+44.g50eb7ef1:
  Successfully uninstalled nvtabular-0.11.0+44.g50eb7ef1
Found existing installation: merlin-core 0.1.1+21.g32e4695
Uninstalling merlin-core-0.1.1+21.g32e4695:
  Successfully uninstalled merlin-core-0.1.1+21.g32e4695
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-ece3c_76/merlin-core_95a52f4d18c741f68d97fc358451c344
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-ece3c_76/merlin-core_95a52f4d18c741f68d97fc358451c344
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 9f373731fdc2e5e5325910af28a8a65e60ff8400
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.20.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.3.5)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.1)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (62.1.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2022.1)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.56.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.2.0+3.g9f37373-py3-none-any.whl size=132428 sha256=d7b6838bcd2ccfe6f4bd4a27dc403a8b412313a0ab57ece66da68e9b4c773437
  Stored in directory: /tmp/pip-ephem-wheel-cache-cjcjooqw/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
merlin-systems 0+untagged.28.gf4069d6 requires merlin-core==0.1.1, but you have merlin-core 0.2.0+3.g9f37373 which is incompatible.
Successfully installed merlin-core-0.2.0+3.g9f37373
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: natsort==8.1.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (8.1.0)
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/cpp
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+6.g1e2a1119 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+6.g1e2a1119 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+6.g1e2a1119 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+6.g1e2a1119 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-cpython-38
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 1.0.0+6.g1e2a1119 is already the active version in easy-install.pth

Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

132 files would be left unchanged.

Running flake8

Running isort

Skipped 2 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)




Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1415 items / 1 skipped


tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

..........................................                               [ 40%]

tests/unit/ops/test_column_similarity.py ........................        [ 42%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 42%]

tests/unit/ops/test_fill.py ............................................ [ 45%]

........                                                                 [ 46%]

tests/unit/ops/test_groupyby.py .........                                [ 46%]

tests/unit/ops/test_hash_bucket.py .........................             [ 48%]

tests/unit/ops/test_join.py ............................................ [ 51%]

........................................................................ [ 56%]

..................................                                       [ 59%]

tests/unit/ops/test_lambda.py ..........                                 [ 59%]

tests/unit/ops/test_normalize.py ....................................... [ 62%]

..                                                                       [ 62%]

tests/unit/ops/test_ops.py ............................................. [ 65%]

....................                                                     [ 67%]

tests/unit/ops/test_ops_schema.py ...................................... [ 70%]

........................................................................ [ 75%]

........................................................................ [ 80%]

........................................................................ [ 85%]

.......................................                                  [ 88%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 88%]

tests/unit/ops/test_target_encode.py .....................               [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]


=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):


../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


nvtabular/loader/init.py:19

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/loader/init.py:19: DeprecationWarning: The nvtabular.loader module has moved to merlin.models.loader. Support for importing from nvtabular.loader is deprecated, and will be removed in a future version. Please update your imports to refer to merlin.models.loader.

warnings.warn(


tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 6 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 93 warnings

tests/unit/ops/test_categorify.py: 69 warnings

tests/unit/ops/test_drop_low_cardinality.py: 2 warnings

tests/unit/ops/test_fill.py: 8 warnings

tests/unit/ops/test_hash_bucket.py: 4 warnings

tests/unit/ops/test_join.py: 88 warnings

tests/unit/ops/test_lambda.py: 3 warnings

tests/unit/ops/test_normalize.py: 9 warnings

tests/unit/ops/test_ops.py: 11 warnings

tests/unit/ops/test_ops_schema.py: 17 warnings

tests/unit/workflow/test_workflow.py: 34 warnings

tests/unit/workflow/test_workflow_chaining.py: 1 warning

tests/unit/workflow/test_workflow_node.py: 1 warning

tests/unit/workflow/test_workflow_schemas.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(


tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(


tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/ops/test_ops.py::test_data_stats[True-parquet]

tests/unit/ops/test_ops.py::test_data_stats[False-parquet]

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(


tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

tests/unit/workflow/test_workflow.py: 9 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(


tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(


tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(


tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1732: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame


See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

self._setitem_single_block(indexer, value, name)


tests/unit/ops/test_ops.py::test_difference_lag[False]

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:3041: FutureWarning: The as_gpu_matrix method will be removed in a future cuDF release. Consider using to_cupy instead.

warnings.warn(


tests/unit/workflow/test_cpu_workflow.py: 6 warnings

tests/unit/workflow/test_workflow.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 10 files.

warnings.warn(


tests/unit/workflow/test_workflow.py: 48 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 20 files.

warnings.warn(


tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-None]

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 4 files.

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing


examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

nvtabular/init.py                                             22      2      0      0    91%   44-45

nvtabular/dispatch.py                                              3      3      0      0     0%   18-23

nvtabular/framework_utils/init.py                              2      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     13     89      6    90%   60, 68->49, 122, 179, 231-239, 242, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 3      3      0      0     0%   18-23

nvtabular/inference/init.py                                    2      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      55     55     24      0     0%   27-128

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    3      3      0      0     0%   18-23

nvtabular/loader/init.py                                       2      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 125, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   31->34, 34->36, 41->43, 45, 46->67, 52-53, 61-63, 69-73, 88-93, 103-116

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      0     14      0   100%

nvtabular/ops/bucketize.py                                        40      9     20      3    73%   52-54, 58->exit, 61-64, 83-86

nvtabular/ops/categorify.py                                      656     71    348     50    86%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 391->395, 399-406, 452, 460, 483-484, 561-566, 637, 733, 750, 795, 873-874, 889-893, 894->858, 912, 920, 927->exit, 951, 954->957, 1006->1004, 1068-1070, 1073, 1075->1077, 1094->1098, 1100->1053, 1106-1109, 1121, 1125, 1129, 1136, 1141-1144, 1222, 1224, 1294->1317, 1300->1317, 1318-1323, 1368, 1381->1384, 1388->1393, 1392, 1398, 1401, 1409-1419

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   107->109, 111, 113->103

nvtabular/ops/difference_lag.py                                   43      0     14      1    98%   73->75

nvtabular/ops/drop_low_cardinality.py                             18      0     10      1    96%   85->84

nvtabular/ops/dropna.py                                            9      0      2      0   100%

nvtabular/ops/fill.py                                             76      5     30      1    92%   63-67, 109

nvtabular/ops/filter.py                                           20      1      8      1    93%   49

nvtabular/ops/groupby.py                                         130      4     84      5    96%   72, 83, 93->95, 131, 221

nvtabular/ops/hash_bucket.py                                      43      1     22      2    95%   79, 118->124

nvtabular/ops/hashed_cross.py                                     38      3     17      3    89%   52, 64, 92

nvtabular/ops/join_external.py                                    96      8     34      8    88%   20-21, 114, 116, 118, 137->141, 150->152, 205-206, 216->227, 221

nvtabular/ops/join_groupby.py                                    128      5     57      6    94%   113, 120, 129, 136->135, 178->175, 181->175, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   60, 64, 82, 94-109

nvtabular/ops/list_slice.py                                       89     29     42      0    64%   21-22, 146-160, 168-190

nvtabular/ops/logop.py                                            21      0      6      0   100%

nvtabular/ops/moments.py                                          69      0     24      0   100%

nvtabular/ops/normalize.py                                        93      4     22      1    94%   89, 139-140, 167

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                49      0     20      2    97%   68->77, 74->77

nvtabular/ops/rename.py                                           29      3     14      3    86%   46, 71-73

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182      9     76      5    93%   168->172, 176->185, 274, 283-284, 297-303, 396->399

nvtabular/ops/value_counts.py                                     34      0      6      1    98%   40->38

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 129-132, 142-146, 148, 170-171, 322, 332, 358, 361-370

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 3      3      0      0     0%   18-23

nvtabular/worker.py                                                3      3      0      0     0%   18-23

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   219     17     94     12    91%   28-29, 52, 85, 206, 212->226, 239-241, 373, 388-389, 431, 508, 538, 548-550, 563


TOTAL                                                           5330   1232   2099    214    75%

Coverage XML written to file coverage.xml


Required test coverage of 70% reached. Total coverage: 74.71%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========== 1414 passed, 2 skipped, 667 warnings in 1263.98s (0:21:03) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins6690104750764220965.sh

@karlhigley karlhigley merged commit c96e554 into NVIDIA-Merlin:main Apr 14, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@benfred benfred benfred approved these changes

Assignees

@jperez999 jperez999

Labels
enhancement New feature or request
Projects
None yet
Milestone
Merlin 22.05
Development

Successfully merging this pull request may close these issues.
4 participants
@jperez999 @nvidia-merlin-bot @benfred @karlhigley