Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add ascending param in the Groupby op #1525

Merged
merged 2 commits into from
Apr 26, 2022
Merged

Add ascending param in the Groupby op #1525

merged 2 commits into from
Apr 26, 2022

Conversation

rnyak
Copy link
Contributor

@rnyak rnyak commented Apr 25, 2022

This addressing #1523.

@github-actions
Copy link

Documentation preview

https://nvidia-merlin.github.io/NVTabular/review/pr-1525

@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #1525 of commit 576e816c44a9dd0834ebf66b409020a1bf071951, no merge conflicts.
Running as SYSTEM
Setting status of 576e816c44a9dd0834ebf66b409020a1bf071951 to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4441/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1525/*:refs/remotes/origin/pr/1525/* # timeout=10
 > git rev-parse 576e816c44a9dd0834ebf66b409020a1bf071951^{commit} # timeout=10
Checking out Revision 576e816c44a9dd0834ebf66b409020a1bf071951 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 576e816c44a9dd0834ebf66b409020a1bf071951 # timeout=10
Commit message: "add ascending param"
 > git rev-list --no-walk f17735297287d4980145206356db9c7260b5e94d # timeout=10
First time build. Skipping changelog.
[nvtabular_tests] $ /bin/bash /tmp/jenkins8887010390637326332.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.9.3)
Collecting setuptools
  Downloading setuptools-62.1.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 18.3 MB/s eta 0:00:00
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 60.9.3
    Uninstalling setuptools-60.9.3:
      Successfully uninstalled setuptools-60.9.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-auth 1.35.0 requires cachetools<5.0,>=2.0.0, but you have cachetools 5.0.0 which is incompatible.
tensorflow-gpu 2.8.0 requires keras<2.9,>=2.8.0rc0, but you have keras 2.6.0 which is incompatible.
tensorflow-gpu 2.8.0 requires tensorboard<2.9,>=2.8, but you have tensorboard 2.6.0 which is incompatible.
Successfully installed setuptools-62.1.0
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.2)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+44.g50eb7ef1
Uninstalling nvtabular-0.11.0+44.g50eb7ef1:
  Successfully uninstalled nvtabular-0.11.0+44.g50eb7ef1
Found existing installation: merlin-core 0.1.1+21.g32e4695
Uninstalling merlin-core-0.1.1+21.g32e4695:
  Successfully uninstalled merlin-core-0.1.1+21.g32e4695
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-ctp5ge6t/merlin-core_2228fe58912741419f977de8203b1aca
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-ctp5ge6t/merlin-core_2228fe58912741419f977de8203b1aca
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 69ccd2650c5cb3b19ca2ceb7863032dee99199be
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.1)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.3.5)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.20.0)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (62.1.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2022.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.56.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.2.0+6.g69ccd26-py3-none-any.whl size=133428 sha256=eeae3f524593fb0a053add5ae366550bf67db1b15484fc5558782036536d2eb4
  Stored in directory: /tmp/pip-ephem-wheel-cache-jtb7_10z/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
merlin-systems 0+untagged.28.gf4069d6 requires merlin-core==0.1.1, but you have merlin-core 0.2.0+6.g69ccd26 which is incompatible.
Successfully installed merlin-core-0.2.0+6.g69ccd26
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: natsort==8.1.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (8.1.0)
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/cpp
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+9.g576e816c -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+9.g576e816c -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+9.g576e816c -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+9.g576e816c -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-cpython-38
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 1.0.0+9.g576e816c is already the active version in easy-install.pth

Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

132 files would be left unchanged.

Running flake8

Running isort

Skipped 2 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)




Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1416 items / 1 skipped


tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

...........................................                              [ 40%]

tests/unit/ops/test_column_similarity.py ........................        [ 42%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 42%]

tests/unit/ops/test_fill.py ............................................ [ 45%]

........                                                                 [ 46%]

tests/unit/ops/test_groupyby.py .........                                [ 46%]

tests/unit/ops/test_hash_bucket.py .........................             [ 48%]

tests/unit/ops/test_join.py ............................................ [ 51%]

........................................................................ [ 56%]

..................................                                       [ 59%]

tests/unit/ops/test_lambda.py ..........                                 [ 59%]

tests/unit/ops/test_normalize.py ....................................... [ 62%]

..                                                                       [ 62%]

tests/unit/ops/test_ops.py ............................................. [ 65%]

....................                                                     [ 67%]

tests/unit/ops/test_ops_schema.py ...................................... [ 70%]

........................................................................ [ 75%]

........................................................................ [ 80%]

........................................................................ [ 85%]

.......................................                                  [ 88%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 88%]

tests/unit/ops/test_target_encode.py .....................               [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]


=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):


../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


nvtabular/loader/init.py:19

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/loader/init.py:19: DeprecationWarning: The nvtabular.loader module has moved to merlin.models.loader. Support for importing from nvtabular.loader is deprecated, and will be removed in a future version. Please update your imports to refer to merlin.models.loader.

warnings.warn(


tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 6 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 93 warnings

tests/unit/ops/test_categorify.py: 70 warnings

tests/unit/ops/test_drop_low_cardinality.py: 2 warnings

tests/unit/ops/test_fill.py: 8 warnings

tests/unit/ops/test_hash_bucket.py: 4 warnings

tests/unit/ops/test_join.py: 88 warnings

tests/unit/ops/test_lambda.py: 3 warnings

tests/unit/ops/test_normalize.py: 9 warnings

tests/unit/ops/test_ops.py: 11 warnings

tests/unit/ops/test_ops_schema.py: 17 warnings

tests/unit/workflow/test_workflow.py: 34 warnings

tests/unit/workflow/test_workflow_chaining.py: 1 warning

tests/unit/workflow/test_workflow_node.py: 1 warning

tests/unit/workflow/test_workflow_schemas.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(


tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(


tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/ops/test_ops.py::test_data_stats[True-parquet]

tests/unit/ops/test_ops.py::test_data_stats[False-parquet]

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(


tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

tests/unit/workflow/test_workflow.py: 9 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(


tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(


tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

tests/unit/ops/test_categorify.py::test_categorify_max_size_null_iloc_check

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(


tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1732: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame


See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

self._setitem_single_block(indexer, value, name)


tests/unit/ops/test_ops.py::test_difference_lag[False]

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:3041: FutureWarning: The as_gpu_matrix method will be removed in a future cuDF release. Consider using to_cupy instead.

warnings.warn(


tests/unit/workflow/test_cpu_workflow.py: 6 warnings

tests/unit/workflow/test_workflow.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 10 files.

warnings.warn(


tests/unit/workflow/test_workflow.py: 48 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 20 files.

warnings.warn(


tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-None]

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 4 files.

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing


examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

nvtabular/init.py                                             22      2      0      0    91%   44-45

nvtabular/dispatch.py                                              3      3      0      0     0%   18-23

nvtabular/framework_utils/init.py                              2      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     13     89      6    90%   60, 68->49, 122, 179, 231-239, 242, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 3      3      0      0     0%   18-23

nvtabular/inference/init.py                                    2      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      55     55     24      0     0%   27-128

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    3      3      0      0     0%   18-23

nvtabular/loader/init.py                                       2      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 142, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   31->34, 34->36, 41->43, 45, 46->67, 52-53, 61-63, 69-73, 88-93, 103-116

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      0     14      0   100%

nvtabular/ops/bucketize.py                                        40      9     20      3    73%   52-54, 58->exit, 61-64, 83-86

nvtabular/ops/categorify.py                                      660     70    348     49    86%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 391->395, 399-406, 452, 460, 483-484, 561-566, 637, 733, 750, 795, 873-874, 889-893, 894->858, 912, 920, 927->exit, 951, 954->957, 1006->1004, 1068, 1073, 1075->1090, 1107->1111, 1113->1053, 1119-1122, 1134, 1138, 1142, 1149, 1154-1157, 1235, 1237, 1307->1330, 1313->1330, 1331-1336, 1381, 1394->1397, 1401->1406, 1405, 1411, 1414, 1422-1432

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   107->109, 111, 113->103

nvtabular/ops/difference_lag.py                                   43      0     14      1    98%   73->75

nvtabular/ops/drop_low_cardinality.py                             18      0     10      1    96%   85->84

nvtabular/ops/dropna.py                                            9      0      2      0   100%

nvtabular/ops/fill.py                                             76      5     30      1    92%   63-67, 109

nvtabular/ops/filter.py                                           20      1      8      1    93%   49

nvtabular/ops/groupby.py                                         135      5     88      6    95%   74, 86, 96->98, 141, 233, 269

nvtabular/ops/hash_bucket.py                                      43      1     22      2    95%   79, 118->124

nvtabular/ops/hashed_cross.py                                     38      3     17      3    89%   52, 64, 92

nvtabular/ops/join_external.py                                    96      8     34      8    88%   20-21, 114, 116, 118, 137->141, 150->152, 205-206, 216->227, 221

nvtabular/ops/join_groupby.py                                    128      5     57      6    94%   113, 120, 129, 136->135, 178->175, 181->175, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   60, 64, 82, 94-109

nvtabular/ops/list_slice.py                                       89     29     42      0    64%   21-22, 146-160, 168-190

nvtabular/ops/logop.py                                            21      0      6      0   100%

nvtabular/ops/moments.py                                          69      0     24      0   100%

nvtabular/ops/normalize.py                                        93      4     22      1    94%   89, 139-140, 167

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                49      0     20      2    97%   68->77, 74->77

nvtabular/ops/rename.py                                           29      3     14      3    86%   46, 71-73

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182      9     76      5    93%   168->172, 176->185, 274, 283-284, 297-303, 396->399

nvtabular/ops/value_counts.py                                     34      0      6      1    98%   40->38

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 129-132, 142-146, 148, 170-171, 322, 332, 358, 361-370

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 3      3      0      0     0%   18-23

nvtabular/worker.py                                                3      3      0      0     0%   18-23

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   219     17     94     12    91%   28-29, 52, 85, 206, 212->226, 239-241, 373, 388-389, 431, 508, 538, 548-550, 563


TOTAL                                                           5339   1232   2103    214    75%

Coverage XML written to file coverage.xml


Required test coverage of 70% reached. Total coverage: 74.75%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========== 1415 passed, 2 skipped, 669 warnings in 1280.95s (0:21:20) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins2188619935007875613.sh

@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #1525 of commit eb6eca40ad42098c9666980ede48adf011471287, no merge conflicts.
Running as SYSTEM
Setting status of eb6eca40ad42098c9666980ede48adf011471287 to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4442/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1525/*:refs/remotes/origin/pr/1525/* # timeout=10
 > git rev-parse eb6eca40ad42098c9666980ede48adf011471287^{commit} # timeout=10
Checking out Revision eb6eca40ad42098c9666980ede48adf011471287 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f eb6eca40ad42098c9666980ede48adf011471287 # timeout=10
Commit message: "add unit test"
 > git rev-list --no-walk 576e816c44a9dd0834ebf66b409020a1bf071951 # timeout=10
[nvtabular_tests] $ /bin/bash /tmp/jenkins7822274283133420039.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.2)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+44.g50eb7ef1
Can't uninstall 'nvtabular'. No files were found to uninstall.
Found existing installation: merlin-core 0.1.1+21.g32e4695
Can't uninstall 'merlin-core'. No files were found to uninstall.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-kmeq4_0d/merlin-core_e990473697a7478eb7c151d73c95575f
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-kmeq4_0d/merlin-core_e990473697a7478eb7c151d73c95575f
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 69ccd2650c5cb3b19ca2ceb7863032dee99199be
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.1)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.20.0)
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.3.5)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (62.1.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2022.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.56.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.2.0+6.g69ccd26-py3-none-any.whl size=133428 sha256=4b97f42d0715d14221180a6667f54859e29e25c5ea39114e6e0aeaf0af9232ad
  Stored in directory: /tmp/pip-ephem-wheel-cache-o6p6z7w0/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
merlin-systems 0+untagged.28.gf4069d6 requires merlin-core==0.1.1, but you have merlin-core 0.2.0+6.g69ccd26 which is incompatible.
Successfully installed merlin-core-0.2.0+6.g69ccd26
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: natsort==8.1.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (8.1.0)
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/cpp
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+10.geb6eca40 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+10.geb6eca40 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+10.geb6eca40 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+10.geb6eca40 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-cpython-38
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 1.0.0+10.geb6eca40 is already the active version in easy-install.pth

Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

132 files would be left unchanged.

Running flake8

Running isort

Skipped 2 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)




Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1422 items / 1 skipped


tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

...........................................                              [ 40%]

tests/unit/ops/test_column_similarity.py ........................        [ 42%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 42%]

tests/unit/ops/test_fill.py ............................................ [ 45%]

........                                                                 [ 45%]

tests/unit/ops/test_groupyby.py ...............                          [ 47%]

tests/unit/ops/test_hash_bucket.py .........................             [ 48%]

tests/unit/ops/test_join.py ............................................ [ 51%]

........................................................................ [ 56%]

..................................                                       [ 59%]

tests/unit/ops/test_lambda.py ..........                                 [ 60%]

tests/unit/ops/test_normalize.py ....................................... [ 62%]

..                                                                       [ 62%]

tests/unit/ops/test_ops.py ............................................. [ 66%]

....................                                                     [ 67%]

tests/unit/ops/test_ops_schema.py ...................................... [ 70%]

........................................................................ [ 75%]

........................................................................ [ 80%]

........................................................................ [ 85%]

.......................................                                  [ 88%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 88%]

tests/unit/ops/test_target_encode.py .....................               [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]


=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):


../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


nvtabular/loader/init.py:19

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/loader/init.py:19: DeprecationWarning: The nvtabular.loader module has moved to merlin.models.loader. Support for importing from nvtabular.loader is deprecated, and will be removed in a future version. Please update your imports to refer to merlin.models.loader.

warnings.warn(


tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 6 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 93 warnings

tests/unit/ops/test_categorify.py: 70 warnings

tests/unit/ops/test_drop_low_cardinality.py: 2 warnings

tests/unit/ops/test_fill.py: 8 warnings

tests/unit/ops/test_hash_bucket.py: 4 warnings

tests/unit/ops/test_join.py: 88 warnings

tests/unit/ops/test_lambda.py: 3 warnings

tests/unit/ops/test_normalize.py: 9 warnings

tests/unit/ops/test_ops.py: 11 warnings

tests/unit/ops/test_ops_schema.py: 17 warnings

tests/unit/workflow/test_workflow.py: 34 warnings

tests/unit/workflow/test_workflow_chaining.py: 1 warning

tests/unit/workflow/test_workflow_node.py: 1 warning

tests/unit/workflow/test_workflow_schemas.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(


tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(


tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/ops/test_ops.py::test_data_stats[True-parquet]

tests/unit/ops/test_ops.py::test_data_stats[False-parquet]

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(


tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

tests/unit/workflow/test_workflow.py: 9 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(


tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(


tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

tests/unit/ops/test_categorify.py::test_categorify_max_size_null_iloc_check

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(


tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1732: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame


See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

self._setitem_single_block(indexer, value, name)


tests/unit/ops/test_ops.py::test_difference_lag[False]

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:3041: FutureWarning: The as_gpu_matrix method will be removed in a future cuDF release. Consider using to_cupy instead.

warnings.warn(


tests/unit/workflow/test_cpu_workflow.py: 6 warnings

tests/unit/workflow/test_workflow.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 10 files.

warnings.warn(


tests/unit/workflow/test_workflow.py: 48 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 20 files.

warnings.warn(


tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-None]

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 4 files.

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing


examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

nvtabular/init.py                                             22      2      0      0    91%   44-45

nvtabular/dispatch.py                                              3      3      0      0     0%   18-23

nvtabular/framework_utils/init.py                              2      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     13     89      6    90%   60, 68->49, 122, 179, 231-239, 242, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 3      3      0      0     0%   18-23

nvtabular/inference/init.py                                    2      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      55     55     24      0     0%   27-128

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    3      3      0      0     0%   18-23

nvtabular/loader/init.py                                       2      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 142, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   31->34, 34->36, 41->43, 45, 46->67, 52-53, 61-63, 69-73, 88-93, 103-116

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      0     14      0   100%

nvtabular/ops/bucketize.py                                        40      9     20      3    73%   52-54, 58->exit, 61-64, 83-86

nvtabular/ops/categorify.py                                      660     70    348     49    86%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 391->395, 399-406, 452, 460, 483-484, 561-566, 637, 733, 750, 795, 873-874, 889-893, 894->858, 912, 920, 927->exit, 951, 954->957, 1006->1004, 1068, 1073, 1075->1090, 1107->1111, 1113->1053, 1119-1122, 1134, 1138, 1142, 1149, 1154-1157, 1235, 1237, 1307->1330, 1313->1330, 1331-1336, 1381, 1394->1397, 1401->1406, 1405, 1411, 1414, 1422-1432

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   107->109, 111, 113->103

nvtabular/ops/difference_lag.py                                   43      0     14      1    98%   73->75

nvtabular/ops/drop_low_cardinality.py                             18      0     10      1    96%   85->84

nvtabular/ops/dropna.py                                            9      0      2      0   100%

nvtabular/ops/fill.py                                             76      5     30      1    92%   63-67, 109

nvtabular/ops/filter.py                                           20      1      8      1    93%   49

nvtabular/ops/groupby.py                                         135      4     88      5    96%   74, 86, 96->98, 141, 233

nvtabular/ops/hash_bucket.py                                      43      1     22      2    95%   79, 118->124

nvtabular/ops/hashed_cross.py                                     38      3     17      3    89%   52, 64, 92

nvtabular/ops/join_external.py                                    96      8     34      8    88%   20-21, 114, 116, 118, 137->141, 150->152, 205-206, 216->227, 221

nvtabular/ops/join_groupby.py                                    128      5     57      6    94%   113, 120, 129, 136->135, 178->175, 181->175, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   60, 64, 82, 94-109

nvtabular/ops/list_slice.py                                       89     29     42      0    64%   21-22, 146-160, 168-190

nvtabular/ops/logop.py                                            21      0      6      0   100%

nvtabular/ops/moments.py                                          69      0     24      0   100%

nvtabular/ops/normalize.py                                        93      4     22      1    94%   89, 139-140, 167

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                49      0     20      2    97%   68->77, 74->77

nvtabular/ops/rename.py                                           29      3     14      3    86%   46, 71-73

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182      9     76      5    93%   168->172, 176->185, 274, 283-284, 297-303, 396->399

nvtabular/ops/value_counts.py                                     34      0      6      1    98%   40->38

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 129-132, 142-146, 148, 170-171, 322, 332, 358, 361-370

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 3      3      0      0     0%   18-23

nvtabular/worker.py                                                3      3      0      0     0%   18-23

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   219     17     94     12    91%   28-29, 52, 85, 206, 212->226, 239-241, 373, 388-389, 431, 508, 538, 548-550, 563


TOTAL                                                           5339   1231   2103    213    75%

Coverage XML written to file coverage.xml


Required test coverage of 70% reached. Total coverage: 74.78%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========== 1421 passed, 2 skipped, 669 warnings in 1388.48s (0:23:08) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins5949926659227381477.sh

@rnyak rnyak added this to the Merlin 22.05 milestone Apr 26, 2022
@karlhigley karlhigley merged commit a94cc8c into NVIDIA-Merlin:main Apr 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karlhigley karlhigley karlhigley approved these changes

@rjzamora rjzamora Awaiting requested review from rjzamora

@sararb sararb Awaiting requested review from sararb

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
Merlin 22.05
Development

Successfully merging this pull request may close these issues.
3 participants
@rnyak @nvidia-merlin-bot @karlhigley