Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

added category name to domain for column properties #1508

Merged
merged 3 commits into from
Apr 12, 2022
Merged

added category name to domain for column properties #1508

merged 3 commits into from
Apr 12, 2022

Conversation

jperez999
Copy link
Contributor

This PR fixes issue #1484. When a column is categorified using joint encoding we will annotate the category that was used to encode the values of the target column.

@jperez999 jperez999 self-assigned this Apr 12, 2022
@jperez999 jperez999 requested a review from karlhigley April 12, 2022 15:43
@jperez999 jperez999 added the enhancement New feature or request label Apr 12, 2022
@karlhigley karlhigley added this to the Merlin 22.05 milestone Apr 12, 2022
@jperez999 jperez999 linked an issue Apr 12, 2022 that may be closed by this pull request
[FEA] Set the same IntDomain name for joint encoded categorical columns #1484
Closed
@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #1508 of commit fe771e297649de488184b5a8688bf85f48b754d0, no merge conflicts.
Running as SYSTEM
Setting status of fe771e297649de488184b5a8688bf85f48b754d0 to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4429/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1508/*:refs/remotes/origin/pr/1508/* # timeout=10
 > git rev-parse fe771e297649de488184b5a8688bf85f48b754d0^{commit} # timeout=10
Checking out Revision fe771e297649de488184b5a8688bf85f48b754d0 (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f fe771e297649de488184b5a8688bf85f48b754d0 # timeout=10
Commit message: "added category name to domain for column properties"
 > git rev-list --no-walk 37f3e5951c8bb41528207a4a2123a55892694226 # timeout=10
First time build. Skipping changelog.
[nvtabular_tests] $ /bin/bash /tmp/jenkins8098920405622848336.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (60.9.3)
Collecting setuptools
  Downloading setuptools-62.1.0-py3-none-any.whl (1.1 MB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 40.4 MB/s eta 0:00:00
Installing collected packages: setuptools
  Attempting uninstall: setuptools
    Found existing installation: setuptools 60.9.3
    Uninstalling setuptools-60.9.3:
      Successfully uninstalled setuptools-60.9.3
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-auth 1.35.0 requires cachetools<5.0,>=2.0.0, but you have cachetools 5.0.0 which is incompatible.
tensorflow-gpu 2.8.0 requires keras<2.9,>=2.8.0rc0, but you have keras 2.6.0 which is incompatible.
tensorflow-gpu 2.8.0 requires tensorboard<2.9,>=2.8, but you have tensorboard 2.6.0 which is incompatible.
Successfully installed setuptools-62.1.0
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.2)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+44.g50eb7ef1
Uninstalling nvtabular-0.11.0+44.g50eb7ef1:
  Successfully uninstalled nvtabular-0.11.0+44.g50eb7ef1
Found existing installation: merlin-core 0.1.1+21.g32e4695
Uninstalling merlin-core-0.1.1+21.g32e4695:
  Successfully uninstalled merlin-core-0.1.1+21.g32e4695
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-d0nh5dxi/merlin-core_08ac79daa6aa4b9a9673b1575cc2d1a8
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-d0nh5dxi/merlin-core_08ac79daa6aa4b9a9673b1575cc2d1a8
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 36ecbfffad1143cd0924a7773fe0e55cc1e4628b
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.20.0)
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.1)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.3.5)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (62.1.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2022.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.56.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.2.0+2.g36ecbff-py3-none-any.whl size=132262 sha256=0e0a4b14a65a5344ecd3e402c76b01b605de31945e814e4227c1031c4d643032
  Stored in directory: /tmp/pip-ephem-wheel-cache-kf5gq7w3/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
merlin-systems 0+untagged.28.gf4069d6 requires merlin-core==0.1.1, but you have merlin-core 0.2.0+2.g36ecbff which is incompatible.
Successfully installed merlin-core-0.2.0+2.g36ecbff
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: natsort==8.1.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (8.1.0)
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/cpp
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+3.gfe771e29 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+3.gfe771e29 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+3.gfe771e29 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+3.gfe771e29 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-cpython-38
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 1.0.0+3.gfe771e29 is already the active version in easy-install.pth

Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

132 files would be left unchanged.

Running flake8

Running isort

Skipped 2 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)




Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1413 items / 1 skipped


tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

........................................                                 [ 40%]

tests/unit/ops/test_column_similarity.py ........................        [ 42%]

tests/unit/ops/test_drop_low_cardinality.py F


=================================== FAILURES ===================================

_______________________ test_drop_low_cardinality[True] ________________________


tmpdir = local('/tmp/pytest-of-jenkins/pytest-6/test_drop_low_cardinality_True0')

cpu = True


@pytest.mark.parametrize("cpu", _CPU)
def test_drop_low_cardinality(tmpdir, cpu):
    df = pd.DataFrame()
    if not cpu:
        df = cudf.DataFrame(df)

    df["col1"] = ["a", "a", "a", "a", "a"]
    df["col2"] = ["a", "a", "a", "a", "b"]
    df["col3"] = ["a", "a", "b", "b", "c"]

    features = list(df.columns) >> nvt.ops.Categorify() >> nvt.ops.DropLowCardinality()

    workflow = nvt.Workflow(features)





  transformed = workflow.fit_transform(nvt.Dataset(df)).to_ddf().compute()





tests/unit/ops/test_drop_low_cardinality.py:43:




nvtabular/workflow/workflow.py:286: in fit_transform

self.fit(dataset)

nvtabular/workflow/workflow.py:261: in fit

self._transform_impl(dataset, capture_dtypes=True).sample_dtypes()

/core/merlin/io/dataset.py:1148: in sample_dtypes

_real_meta = self.engine.sample_data(n=n)

/core/merlin/io/dataset_engine.py:71: in sample_data

_head = _ddf.partitions[partition_index].head(n)

../../../.local/lib/python3.8/site-packages/dask/dataframe/core.py:1098: in head

return self._head(n=n, npartitions=npartitions, compute=compute, safe=safe)

../../../.local/lib/python3.8/site-packages/dask/dataframe/core.py:1132: in _head

result = result.compute()

../../../.local/lib/python3.8/site-packages/dask/base.py:288: in compute

(result,) = compute(self, traverse=False, **kwargs)

../../../.local/lib/python3.8/site-packages/dask/base.py:571: in compute

results = schedule(dsk, keys, **kwargs)

../../../.local/lib/python3.8/site-packages/dask/local.py:553: in get_sync

return get_async(

../../../.local/lib/python3.8/site-packages/dask/local.py:496: in get_async

for key, res_info, failed in queue_get(queue).result():

/usr/lib/python3.8/concurrent/futures/_base.py:437: in result

return self.__get_result()

/usr/lib/python3.8/concurrent/futures/_base.py:389: in __get_result

raise self._exception

../../../.local/lib/python3.8/site-packages/dask/local.py:538: in submit

fut.set_result(fn(*args, **kwargs))

../../../.local/lib/python3.8/site-packages/dask/local.py:234: in batch_execute_tasks

return [execute_task(a) for a in it]

../../../.local/lib/python3.8/site-packages/dask/local.py:234: in 

return [execute_task(a) for a in it]

../../../.local/lib/python3.8/site-packages/dask/local.py:225: in execute_task

result = pack_exception(e, dumps)

../../../.local/lib/python3.8/site-packages/dask/local.py:220: in execute_task

result = _execute_task(task, data)

../../../.local/lib/python3.8/site-packages/dask/core.py:119: in _execute_task

return func((_execute_task(a, cache) for a in args))

../../../.local/lib/python3.8/site-packages/dask/optimization.py:969: in call

return core.get(self.dsk, self.outkey, dict(zip(self.inkeys, args)))

../../../.local/lib/python3.8/site-packages/dask/core.py:149: in get

result = _execute_task(task, cache)

../../../.local/lib/python3.8/site-packages/dask/core.py:119: in _execute_task

return func((_execute_task(a, cache) for a in args))

../../../.local/lib/python3.8/site-packages/dask/utils.py:37: in apply

return func(*args, **kwargs)

nvtabular/workflow/workflow.py:523: in _transform_partition

col_series = output_df[col_name]

/usr/local/lib/python3.8/dist-packages/nvtx/nvtx.py:101: in inner

result = func(*args, **kwargs)

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1008: in getitem

return self._get_columns_by_label(arg, downcast=True)

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1864: in _get_columns_by_label

new_data = super()._get_columns_by_label(labels, downcast)

/usr/lib/python3.8/site-packages/cudf/core/frame.py:508: in _get_columns_by_label

return self._data.select_by_label(labels)

/usr/lib/python3.8/site-packages/cudf/core/column_accessor.py:344: in select_by_label

return self._select_by_label_grouped(key)




self = ColumnAccessor(multiindex=False, level_names=[None])

, key = 'col2'


def _select_by_label_grouped(self, key: Any) -> ColumnAccessor:





  result = self._grouped_data[key]





E       KeyError: 'col2'


/usr/lib/python3.8/site-packages/cudf/core/column_accessor.py:406: KeyError

------------------------------ Captured log call -------------------------------

ERROR    nvtabular:workflow.py:545 Failed to transform operator <nvtabular.ops.drop_low_cardinality.DropLowCardinality object at 0x7f4cb01b6790>

Traceback (most recent call last):

File "/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/workflow/workflow.py", line 523, in _transform_partition

col_series = output_df[col_name]

File "/usr/local/lib/python3.8/dist-packages/nvtx/nvtx.py", line 101, in inner

result = func(*args, **kwargs)

File "/usr/lib/python3.8/site-packages/cudf/core/dataframe.py", line 1008, in getitem

return self._get_columns_by_label(arg, downcast=True)

File "/usr/lib/python3.8/site-packages/cudf/core/dataframe.py", line 1864, in _get_columns_by_label

new_data = super()._get_columns_by_label(labels, downcast)

File "/usr/lib/python3.8/site-packages/cudf/core/frame.py", line 508, in _get_columns_by_label

return self._data.select_by_label(labels)

File "/usr/lib/python3.8/site-packages/cudf/core/column_accessor.py", line 344, in select_by_label

return self._select_by_label_grouped(key)

File "/usr/lib/python3.8/site-packages/cudf/core/column_accessor.py", line 406, in _select_by_label_grouped

result = self._grouped_data[key]

KeyError: 'col2'

=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):


../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


nvtabular/loader/init.py:19

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/loader/init.py:19: DeprecationWarning: The nvtabular.loader module has moved to merlin.models.loader. Support for importing from nvtabular.loader is deprecated, and will be removed in a future version. Please update your imports to refer to merlin.models.loader.

warnings.warn(


tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 6 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 93 warnings

tests/unit/ops/test_categorify.py: 68 warnings

tests/unit/ops/test_drop_low_cardinality.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(


tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(


tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(


tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(


tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(


tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing


examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

nvtabular/init.py                                             22      2      0      0    91%   44-45

nvtabular/dispatch.py                                              3      3      0      0     0%   18-23

nvtabular/framework_utils/init.py                              2      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     13     89      6    90%   60, 68->49, 122, 179, 231-239, 242, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 3      3      0      0     0%   18-23

nvtabular/inference/init.py                                    2      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      55     55     24      0     0%   27-128

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    3      3      0      0     0%   18-23

nvtabular/loader/init.py                                       2      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 142, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   31->34, 34->36, 41->43, 45, 46->67, 52-53, 61-63, 69-73, 88-93, 103-116

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      3     14      0    94%   34, 38, 42

nvtabular/ops/bucketize.py                                        40     20     20      2    40%   52-54, 58->exit, 59-64, 71-88, 92, 96

nvtabular/ops/categorify.py                                      656     75    348     51    85%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 391->395, 399-406, 452, 460, 483-484, 561-566, 637, 733, 750, 795, 873-874, 889-893, 894->858, 912, 920, 927-928, 945-946, 951, 954->957, 1006->1004, 1068-1070, 1073, 1075->1077, 1094->1098, 1100->1053, 1106-1109, 1121, 1125, 1129, 1136, 1141-1144, 1222, 1224, 1294->1317, 1300->1317, 1318-1323, 1368, 1381->1384, 1388->1393, 1392, 1398, 1401, 1409-1419

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   107->109, 111, 113->103

nvtabular/ops/difference_lag.py                                   43     21     14      1    44%   60->63, 70-81, 87, 90-95, 99, 103, 106

nvtabular/ops/drop_low_cardinality.py                             23      0      8      1    97%   41->46

nvtabular/ops/dropna.py                                            9      3      2      0    73%   36-38

nvtabular/ops/fill.py                                             76     13     30      7    77%   53-55, 63-67, 73, 79, 109, 113, 141, 147

nvtabular/ops/filter.py                                           20      3      8      3    79%   49, 56, 60

nvtabular/ops/groupby.py                                         130     12     84     10    86%   72, 83, 93->95, 104->109, 122, 131, 140->139, 221, 239, 267, 273-280

nvtabular/ops/hash_bucket.py                                      43      4     22      5    83%   75, 79, 91, 110, 118->124

nvtabular/ops/hashed_cross.py                                     38     22     17      1    35%   52, 58-68, 73-78, 82, 87-92

nvtabular/ops/join_external.py                                    96     74     34      0    18%   20-21, 101-119, 124-152, 155-159, 162-170, 181-182, 186-187, 190-199, 202-206, 209, 212, 216-227

nvtabular/ops/join_groupby.py                                    128     20     57      9    79%   111, 113, 120, 126-129, 136-139, 144-146, 177-180, 181->175, 230-231, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   60, 64, 82, 94-109

nvtabular/ops/list_slice.py                                       89     38     42      4    48%   21-22, 67-68, 74, 86-94, 121->127, 146-160, 168-190

nvtabular/ops/logop.py                                            21      2      6      1    89%   48-51

nvtabular/ops/moments.py                                          69      1     24      1    98%   71

nvtabular/ops/normalize.py                                        93     27     22      3    67%   72, 77, 82, 89, 126-128, 134-142, 148, 155-159, 162-163, 167, 176, 180

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                49     28     20      0    33%   36-39, 43, 46, 49-50, 54-56, 59-80

nvtabular/ops/rename.py                                           29      3     14      3    86%   46, 71-73

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182    127     76      0    22%   166-207, 210-213, 217, 226-227, 230-243, 246-251, 254-257, 261, 265, 269-274, 277-280, 283-284, 287-288, 291-292, 296-377, 381-413, 423-432

nvtabular/ops/value_counts.py                                     34     20      6      0    40%   37-53, 56, 59, 62-64, 67

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 129-132, 142-146, 148, 170-171, 322, 332, 358, 361-370

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 3      3      0      0     0%   18-23

nvtabular/worker.py                                                3      3      0      0     0%   18-23

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   219     18     94     13    90%   28-29, 52, 85, 126, 187->190, 206, 212->226, 239-241, 373, 388-389, 431, 508, 538, 548-550, 563


TOTAL                                                           5335   1597   2097    224    67%

Coverage XML written to file coverage.xml


FAIL Required test coverage of 70% not reached. Total coverage: 67.06%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

!!!!!!!!!!!!!!!!!!!!!!!!!! stopping after 1 failures !!!!!!!!!!!!!!!!!!!!!!!!!!!

====== 1 failed, 596 passed, 2 skipped, 398 warnings in 716.81s (0:11:56) ======

Build step 'Execute shell' marked build as failure

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins5089424734425581800.sh

@karlhigley karlhigley linked an issue Apr 12, 2022 that may be closed by this pull request
[BUG] Joint encoding of two or multiple columns outputs incomplete schema #1509
Closed
@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #1508 of commit 8172bc95387d0d6271fb455bcb527e5bc954f3cc, no merge conflicts.
Running as SYSTEM
Setting status of 8172bc95387d0d6271fb455bcb527e5bc954f3cc to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4430/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1508/*:refs/remotes/origin/pr/1508/* # timeout=10
 > git rev-parse 8172bc95387d0d6271fb455bcb527e5bc954f3cc^{commit} # timeout=10
Checking out Revision 8172bc95387d0d6271fb455bcb527e5bc954f3cc (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f 8172bc95387d0d6271fb455bcb527e5bc954f3cc # timeout=10
Commit message: "refactored low cardinality op"
 > git rev-list --no-walk fe771e297649de488184b5a8688bf85f48b754d0 # timeout=10
[nvtabular_tests] $ /bin/bash /tmp/jenkins6220356450550827094.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.2)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+44.g50eb7ef1
Can't uninstall 'nvtabular'. No files were found to uninstall.
Found existing installation: merlin-core 0.1.1+21.g32e4695
Can't uninstall 'merlin-core'. No files were found to uninstall.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-b4676x4n/merlin-core_4c426deefaf84ab18121b0a356e17cf9
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-b4676x4n/merlin-core_4c426deefaf84ab18121b0a356e17cf9
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 36ecbfffad1143cd0924a7773fe0e55cc1e4628b
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.1)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.20.0)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.3.5)
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (62.1.0)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2022.1)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.56.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.2.0+2.g36ecbff-py3-none-any.whl size=132262 sha256=dd1a3dbdb69b152a7cf84d32a07f91b86ee2da64032713c114719df365ab23a9
  Stored in directory: /tmp/pip-ephem-wheel-cache-oyeealou/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
merlin-systems 0+untagged.28.gf4069d6 requires merlin-core==0.1.1, but you have merlin-core 0.2.0+2.g36ecbff which is incompatible.
Successfully installed merlin-core-0.2.0+2.g36ecbff
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: natsort==8.1.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (8.1.0)
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/cpp
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+4.g8172bc95 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+4.g8172bc95 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+4.g8172bc95 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+4.g8172bc95 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-cpython-38
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 1.0.0+4.g8172bc95 is already the active version in easy-install.pth

Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

132 files would be left unchanged.

Running flake8

Running isort

Skipped 2 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)




Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1413 items / 1 skipped


tests/unit/test_dask_nvt.py Terminated

Build was aborted

Aborted by �[8mha:////4I6AZwo/1Z8Fal8AhZTEatjIwqNwCcqT21311HdysuK+AAAAlx+LCAAAAAAAAP9b85aBtbiIQTGjNKU4P08vOT+vOD8nVc83PyU1x6OyILUoJzMv2y+/JJUBAhiZGBgqihhk0NSjKDWzXb3RdlLBUSYGJk8GtpzUvPSSDB8G5tKinBIGIZ+sxLJE/ZzEvHT94JKizLx0a6BxUmjGOUNodHsLgAzWEgZu/dLi1CL9xJTczDwAj6GcLcAAAAA=�[0madmin

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins5843213806204272455.sh

@github-actions
Copy link

Documentation preview

https://nvidia-merlin.github.io/NVTabular/review/pr-1508

@nvidia-merlin-bot
Copy link
Contributor

Click to view CI Results
GitHub pull request #1508 of commit cbc34304fd27896620e415d214ff684a4473d7aa, no merge conflicts.
Running as SYSTEM
Setting status of cbc34304fd27896620e415d214ff684a4473d7aa to PENDING with url http://10.20.13.93:8080/job/nvtabular_tests/4431/ and message: 'Pending'
Using context: Jenkins Unit Test Run
Building on master in workspace /var/jenkins_home/workspace/nvtabular_tests
using credential nvidia-merlin-bot
Cloning the remote Git repository
Cloning repository https://github.com/NVIDIA-Merlin/NVTabular.git
 > git init /var/jenkins_home/workspace/nvtabular_tests/nvtabular # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
 > git --version # timeout=10
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
 > git config --add remote.origin.fetch +refs/heads/*:refs/remotes/origin/* # timeout=10
 > git config remote.origin.url https://github.com/NVIDIA-Merlin/NVTabular.git # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/NVTabular.git
using GIT_ASKPASS to set credentials This is the bot credentials for our CI/CD
 > git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/NVTabular.git +refs/pull/1508/*:refs/remotes/origin/pr/1508/* # timeout=10
 > git rev-parse cbc34304fd27896620e415d214ff684a4473d7aa^{commit} # timeout=10
Checking out Revision cbc34304fd27896620e415d214ff684a4473d7aa (detached)
 > git config core.sparsecheckout # timeout=10
 > git checkout -f cbc34304fd27896620e415d214ff684a4473d7aa # timeout=10
Commit message: "fix typo in categorify clear_stats def"
 > git rev-list --no-walk 8172bc95387d0d6271fb455bcb527e5bc954f3cc # timeout=10
[nvtabular_tests] $ /bin/bash /tmp/jenkins4669320383934701618.sh
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Installing NVTabular
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: pip in /var/jenkins_home/.local/lib/python3.8/site-packages (22.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (62.1.0)
Requirement already satisfied: wheel in /var/jenkins_home/.local/lib/python3.8/site-packages (0.37.1)
Requirement already satisfied: pybind11 in /var/jenkins_home/.local/lib/python3.8/site-packages (2.9.2)
Requirement already satisfied: numpy==1.20.3 in /var/jenkins_home/.local/lib/python3.8/site-packages (1.20.3)
Found existing installation: nvtabular 0.11.0+44.g50eb7ef1
Can't uninstall 'nvtabular'. No files were found to uninstall.
Found existing installation: merlin-core 0.1.1+21.g32e4695
Can't uninstall 'merlin-core'. No files were found to uninstall.
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Collecting merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git
  Cloning https://github.com/NVIDIA-Merlin/core.git to /tmp/pip-install-q8_a2giv/merlin-core_ec671376577240e1a500d1310ab491f4
  Running command git clone --filter=blob:none --quiet https://github.com/NVIDIA-Merlin/core.git /tmp/pip-install-q8_a2giv/merlin-core_ec671376577240e1a500d1310ab491f4
  Resolved https://github.com/NVIDIA-Merlin/core.git to commit 36ecbfffad1143cd0924a7773fe0e55cc1e4628b
  Installing build dependencies: started
  Installing build dependencies: finished with status 'done'
  Getting requirements to build wheel: started
  Getting requirements to build wheel: finished with status 'done'
  Preparing metadata (pyproject.toml): started
  Preparing metadata (pyproject.toml): finished with status 'done'
Requirement already satisfied: dask==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: protobuf>=3.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.20.0)
Requirement already satisfied: pyarrow>=1.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.0.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (21.3)
Requirement already satisfied: distributed==2021.11.2 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.2)
Requirement already satisfied: pandas<1.4.0dev0,>=1.2.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.3.5)
Requirement already satisfied: tensorflow-metadata>=1.2.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: tqdm>=4.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.63.1)
Requirement already satisfied: numba>=0.53.1 in /var/jenkins_home/.local/lib/python3.8/site-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.55.1)
Requirement already satisfied: betterproto<2.0.0 in /usr/local/lib/python3.8/dist-packages (from merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.5)
Requirement already satisfied: partd>=0.3.10 in /var/jenkins_home/.local/lib/python3.8/site-packages/partd-1.2.0-py3.8.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: pyyaml in /var/jenkins_home/.local/lib/python3.8/site-packages/PyYAML-5.4.1-py3.8-linux-x86_64.egg (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.4.1)
Requirement already satisfied: fsspec>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2021.11.0)
Requirement already satisfied: cloudpickle>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: toolz>=0.8.2 in /usr/local/lib/python3.8/dist-packages (from dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.11.2)
Requirement already satisfied: click>=6.6 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (8.0.4)
Requirement already satisfied: setuptools in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (62.1.0)
Requirement already satisfied: tornado>=6.0.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/tornado-6.1-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.1)
Requirement already satisfied: msgpack>=0.6.0 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.3)
Requirement already satisfied: sortedcontainers!=2.0.0,!=2.0.1 in /var/jenkins_home/.local/lib/python3.8/site-packages/sortedcontainers-2.4.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.4.0)
Requirement already satisfied: tblib>=1.6.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/tblib-1.7.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.7.0)
Requirement already satisfied: zict>=0.1.3 in /var/jenkins_home/.local/lib/python3.8/site-packages/zict-2.0.0-py3.8.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.0)
Requirement already satisfied: jinja2 in /usr/local/lib/python3.8/dist-packages (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.3)
Requirement already satisfied: psutil>=5.0 in /var/jenkins_home/.local/lib/python3.8/site-packages/psutil-5.8.0-py3.8-linux-x86_64.egg (from distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (5.8.0)
Requirement already satisfied: grpclib in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.4.2)
Requirement already satisfied: stringcase in /usr/local/lib/python3.8/dist-packages (from betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.2.0)
Requirement already satisfied: llvmlite<0.39,>=0.38.0rc1 in /usr/local/lib/python3.8/dist-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.38.0)
Requirement already satisfied: numpy<1.22,>=1.18 in /var/jenkins_home/.local/lib/python3.8/site-packages (from numba>=0.53.1->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.20.3)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (3.0.7)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2022.1)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.8/dist-packages (from pandas<1.4.0dev0,>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.8.2)
Requirement already satisfied: googleapis-common-protos<2,>=1.52.0 in /usr/local/lib/python3.8/dist-packages (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.56.0)
Requirement already satisfied: absl-py<2.0.0,>=0.9 in /var/jenkins_home/.local/lib/python3.8/site-packages/absl_py-0.12.0-py3.8.egg (from tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.12.0)
Requirement already satisfied: six in /var/jenkins_home/.local/lib/python3.8/site-packages (from absl-py<2.0.0,>=0.9->tensorflow-metadata>=1.2.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.15.0)
Requirement already satisfied: locket in /var/jenkins_home/.local/lib/python3.8/site-packages/locket-0.2.1-py3.8.egg (from partd>=0.3.10->dask==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (0.2.1)
Requirement already satisfied: heapdict in /var/jenkins_home/.local/lib/python3.8/site-packages/HeapDict-1.0.1-py3.8.egg (from zict>=0.1.3->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (1.0.1)
Requirement already satisfied: multidict in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.2)
Requirement already satisfied: h2<5,>=3.1.0 in /usr/local/lib/python3.8/dist-packages (from grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.1.0)
Requirement already satisfied: MarkupSafe>=2.0 in /usr/local/lib/python3.8/dist-packages (from jinja2->distributed==2021.11.2->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (2.0.1)
Requirement already satisfied: hyperframe<7,>=6.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (6.0.1)
Requirement already satisfied: hpack<5,>=4.0 in /usr/local/lib/python3.8/dist-packages (from h2<5,>=3.1.0->grpclib->betterproto<2.0.0->merlin-core@ git+https://github.com/NVIDIA-Merlin/core.git) (4.0.0)
Building wheels for collected packages: merlin-core
  Building wheel for merlin-core (pyproject.toml): started
  Building wheel for merlin-core (pyproject.toml): finished with status 'done'
  Created wheel for merlin-core: filename=merlin_core-0.2.0+2.g36ecbff-py3-none-any.whl size=132262 sha256=b1d458efb875645cd6c650c02b0a1c922a37a0566e1583a3c01cfebaf7534d5d
  Stored in directory: /tmp/pip-ephem-wheel-cache-9ld03qjy/wheels/c8/38/16/a6968787eafcec5fa772148af8408b089562f71af0752e8e84
Successfully built merlin-core
Installing collected packages: merlin-core
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
merlin-systems 0+untagged.28.gf4069d6 requires merlin-core==0.1.1, but you have merlin-core 0.2.0+2.g36ecbff which is incompatible.
Successfully installed merlin-core-0.2.0+2.g36ecbff
Looking in indexes: https://pypi.org/simple, https://pypi.ngc.nvidia.com
Requirement already satisfied: natsort==8.1.0 in /var/jenkins_home/.local/lib/python3.8/site-packages (8.1.0)
running develop
running egg_info
creating nvtabular.egg-info
writing nvtabular.egg-info/PKG-INFO
writing dependency_links to nvtabular.egg-info/dependency_links.txt
writing requirements to nvtabular.egg-info/requires.txt
writing top-level names to nvtabular.egg-info/top_level.txt
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
/usr/local/lib/python3.8/dist-packages/setuptools/command/easy_install.py:144: EasyInstallDeprecationWarning: easy_install command is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
/usr/local/lib/python3.8/dist-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
  warnings.warn(
package init file 'ci/__init__.py' not found (or not a regular file)
package init file 'images/__init__.py' not found (or not a regular file)
package init file 'docs/__init__.py' not found (or not a regular file)
package init file 'cpp/__init__.py' not found (or not a regular file)
package init file 'bench/__init__.py' not found (or not a regular file)
package init file 'merlin/__init__.py' not found (or not a regular file)
package init file 'examples/__init__.py' not found (or not a regular file)
package init file 'conda/__init__.py' not found (or not a regular file)
package init file 'docs/source/__init__.py' not found (or not a regular file)
package init file 'docs/source/_templates/__init__.py' not found (or not a regular file)
package init file 'docs/source/images/__init__.py' not found (or not a regular file)
package init file 'docs/source/training/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/__init__.py' not found (or not a regular file)
package init file 'docs/source/resources/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/ops/__init__.py' not found (or not a regular file)
package init file 'docs/source/api/workflow/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'docs/source/examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/__init__.py' not found (or not a regular file)
package init file 'cpp/nvtabular/inference/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/__init__.py' not found (or not a regular file)
package init file 'bench/torch/__init__.py' not found (or not a regular file)
package init file 'bench/examples/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/configs/__init__.py' not found (or not a regular file)
package init file 'bench/datasets/tools/__init__.py' not found (or not a regular file)
package init file 'bench/torch/criteo/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-toy-example/__init__.py' not found (or not a regular file)
package init file 'examples/tabular-data-rossmann/__init__.py' not found (or not a regular file)
package init file 'examples/advanced-ops-outbrain/__init__.py' not found (or not a regular file)
package init file 'examples/multi-gpu-movielens/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/__init__.py' not found (or not a regular file)
package init file 'examples/winning-solution-recsys2020-twitter/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/docker/__init__.py' not found (or not a regular file)
package init file 'examples/tensorflow/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/getting-started-movielens/imgs/__init__.py' not found (or not a regular file)
package init file 'examples/scaling-criteo/imgs/__init__.py' not found (or not a regular file)
package init file 'tests/integration/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/__init__.py' not found (or not a regular file)
package init file 'tests/integration/common/parsers/__init__.py' not found (or not a regular file)
package init file 'tests/unit/loader/__init__.py' not found (or not a regular file)
package init file 'tests/unit/framework_utils/__init__.py' not found (or not a regular file)
package init file 'tests/unit/ops/__init__.py' not found (or not a regular file)
package init file 'tests/unit/workflow/__init__.py' not found (or not a regular file)
package init file 'conda/environments/__init__.py' not found (or not a regular file)
package init file 'conda/recipes/__init__.py' not found (or not a regular file)
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.h' under directory 'cpp'
warning: no files found matching '*.cu' under directory 'cpp'
warning: no files found matching '*.cuh' under directory 'cpp'
adding license file 'LICENSE'
writing manifest file 'nvtabular.egg-info/SOURCES.txt'
running build_ext
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/usr/include/python3.8 -c flagcheck.cpp -o flagcheck.o -std=c++17
building 'nvtabular_cpp' extension
creating build
creating build/temp.linux-x86_64-cpython-38
creating build/temp.linux-x86_64-cpython-38/cpp
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular
creating build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+5.gcbc34304 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+5.gcbc34304 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/__init__.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+5.gcbc34304 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/categorify.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o -std=c++17 -fvisibility=hidden -g0
x86_64-linux-gnu-gcc -pthread -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -DVERSION_INFO=1.0.0+5.gcbc34304 -I./cpp/ -I/var/jenkins_home/.local/lib/python3.8/site-packages/pybind11/include -I/usr/include/python3.8 -c cpp/nvtabular/inference/fill.cc -o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -std=c++17 -fvisibility=hidden -g0
creating build/lib.linux-x86_64-cpython-38
x86_64-linux-gnu-g++ -pthread -shared -Wl,-O1 -Wl,-Bsymbolic-functions -Wl,-Bsymbolic-functions -Wl,-z,relro -g -fwrapv -O2 build/temp.linux-x86_64-cpython-38/cpp/nvtabular/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/__init__.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/categorify.o build/temp.linux-x86_64-cpython-38/cpp/nvtabular/inference/fill.o -L/usr/lib -o build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so
copying build/lib.linux-x86_64-cpython-38/nvtabular_cpp.cpython-38-x86_64-linux-gnu.so -> 
Generating nvtabular/inference/triton/model_config_pb2.py from nvtabular/inference/triton/model_config.proto
Creating /var/jenkins_home/.local/lib/python3.8/site-packages/nvtabular.egg-link (link to .)
nvtabular 1.0.0+5.gcbc34304 is already the active version in easy-install.pth

Installed /var/jenkins_home/workspace/nvtabular_tests/nvtabular

Running black --check

All done! ✨ 🍰 ✨

132 files would be left unchanged.

Running flake8

Running isort

Skipped 2 files

Running bandit

Running pylint

************* Module bench.datasets.tools.train_hugectr

bench/datasets/tools/train_hugectr.py:28:13: I1101: Module 'hugectr' has no 'solver_parser_helper' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)

bench/datasets/tools/train_hugectr.py:41:16: I1101: Module 'hugectr' has no 'optimizer' member, but source is unavailable. Consider adding this module to extension-pkg-allow-list if you want to perform analysis based on run-time introspection of living objects. (c-extension-no-member)




Your code has been rated at 10.00/10 (previous run: 10.00/10, +0.00)


Running flake8-nb

Building docs

make: Entering directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

/usr/lib/python3/dist-packages/requests/init.py:89: RequestsDependencyWarning: urllib3 (1.26.9) or chardet (3.0.4) doesn't match a supported version!

warnings.warn("urllib3 ({}) or chardet ({}) doesn't match a supported "

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

/usr/local/lib/python3.8/dist-packages/recommonmark/parser.py:75: UserWarning: Container node skipped: type=document

warn("Container node skipped: type={0}".format(mdnode.t))

make: Leaving directory '/var/jenkins_home/workspace/nvtabular_tests/nvtabular/docs'

============================= test session starts ==============================

platform linux -- Python 3.8.10, pytest-7.1.1, pluggy-1.0.0

rootdir: /var/jenkins_home/workspace/nvtabular_tests/nvtabular, configfile: pyproject.toml

plugins: xdist-2.5.0, forked-1.4.0, cov-3.0.0

collected 1413 items / 1 skipped


tests/unit/test_dask_nvt.py ............................................ [  3%]

........................................................................ [  8%]

[  8%]

tests/unit/test_notebooks.py ......                                      [  8%]

tests/unit/test_tf4rec.py .                                              [  8%]

tests/unit/test_tools.py ......................                          [ 10%]

tests/unit/test_triton_inference.py ................................     [ 12%]

tests/unit/framework_utils/test_tf_feature_columns.py .                  [ 12%]

tests/unit/framework_utils/test_tf_layers.py ........................... [ 14%]

...................................................                      [ 18%]

tests/unit/framework_utils/test_torch_layers.py .                        [ 18%]

tests/unit/loader/test_dataloader_backend.py ......                      [ 18%]

tests/unit/loader/test_tf_dataloader.py ................................ [ 20%]

........................................s..                              [ 23%]

tests/unit/loader/test_torch_dataloader.py ............................. [ 25%]

........................................................                 [ 29%]

tests/unit/ops/test_categorify.py ...................................... [ 32%]

........................................................................ [ 37%]

........................................                                 [ 40%]

tests/unit/ops/test_column_similarity.py ........................        [ 42%]

tests/unit/ops/test_drop_low_cardinality.py ..                           [ 42%]

tests/unit/ops/test_fill.py ............................................ [ 45%]

........                                                                 [ 46%]

tests/unit/ops/test_groupyby.py .........                                [ 46%]

tests/unit/ops/test_hash_bucket.py .........................             [ 48%]

tests/unit/ops/test_join.py ............................................ [ 51%]

........................................................................ [ 56%]

..................................                                       [ 59%]

tests/unit/ops/test_lambda.py ..........                                 [ 59%]

tests/unit/ops/test_normalize.py ....................................... [ 62%]

..                                                                       [ 62%]

tests/unit/ops/test_ops.py ............................................. [ 65%]

....................                                                     [ 67%]

tests/unit/ops/test_ops_schema.py ...................................... [ 69%]

........................................................................ [ 75%]

........................................................................ [ 80%]

........................................................................ [ 85%]

.......................................                                  [ 88%]

tests/unit/ops/test_reduce_dtype_size.py ..                              [ 88%]

tests/unit/ops/test_target_encode.py .....................               [ 89%]

tests/unit/workflow/test_cpu_workflow.py ......                          [ 90%]

tests/unit/workflow/test_workflow.py ................................... [ 92%]

..........................................................               [ 96%]

tests/unit/workflow/test_workflow_chaining.py ...                        [ 96%]

tests/unit/workflow/test_workflow_node.py ...........                    [ 97%]

tests/unit/workflow/test_workflow_ops.py ...                             [ 97%]

tests/unit/workflow/test_workflow_schemas.py ........................... [ 99%]

...                                                                      [100%]


=============================== warnings summary ===============================

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

../../../../../usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92

/usr/local/lib/python3.8/dist-packages/fsspec/spec.py:92: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

if pa_version and LooseVersion(pa_version) < LooseVersion("2.0"):


../../../../../usr/lib/python3.8/site-packages/dask_cudf/core.py:32

/usr/lib/python3.8/site-packages/dask_cudf/core.py:32: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

DASK_VERSION = LooseVersion(dask.version)


../../../../../usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: 34 warnings

/usr/local/lib/python3.8/dist-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.

other = LooseVersion(other)


nvtabular/loader/init.py:19

/var/jenkins_home/workspace/nvtabular_tests/nvtabular/nvtabular/loader/init.py:19: DeprecationWarning: The nvtabular.loader module has moved to merlin.models.loader. Support for importing from nvtabular.loader is deprecated, and will be removed in a future version. Please update your imports to refer to merlin.models.loader.

warnings.warn(


tests/unit/test_dask_nvt.py: 2 warnings

tests/unit/test_tf4rec.py: 1 warning

tests/unit/test_tools.py: 6 warnings

tests/unit/test_triton_inference.py: 8 warnings

tests/unit/loader/test_dataloader_backend.py: 6 warnings

tests/unit/loader/test_tf_dataloader.py: 142 warnings

tests/unit/loader/test_torch_dataloader.py: 93 warnings

tests/unit/ops/test_categorify.py: 68 warnings

tests/unit/ops/test_drop_low_cardinality.py: 2 warnings

tests/unit/ops/test_fill.py: 8 warnings

tests/unit/ops/test_hash_bucket.py: 4 warnings

tests/unit/ops/test_join.py: 88 warnings

tests/unit/ops/test_lambda.py: 3 warnings

tests/unit/ops/test_normalize.py: 9 warnings

tests/unit/ops/test_ops.py: 11 warnings

tests/unit/ops/test_ops_schema.py: 17 warnings

tests/unit/workflow/test_workflow.py: 34 warnings

tests/unit/workflow/test_workflow_chaining.py: 1 warning

tests/unit/workflow/test_workflow_node.py: 1 warning

tests/unit/workflow/test_workflow_schemas.py: 1 warning

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:1253: UserWarning: The deep parameter is ignored and is only included for pandas compatibility.

warnings.warn(


tests/unit/test_dask_nvt.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 8 files.

warnings.warn(


tests/unit/test_dask_nvt.py::test_merlin_core_execution_managers

/core/merlin/core/utils.py:433: UserWarning: Existing Dask-client object detected in the current context. New cuda cluster will not be deployed. Set force_new to True to ignore running clusters.

warnings.warn(


tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/test_tools.py::test_inspect_datagen[uniform-parquet]

tests/unit/ops/test_ops.py::test_data_stats[True-parquet]

tests/unit/ops/test_ops.py::test_data_stats[False-parquet]

/usr/lib/python3.8/site-packages/cudf/core/series.py:923: FutureWarning: Series.set_index is deprecated and will be removed in the future

warnings.warn(


tests/unit/loader/test_tf_dataloader.py: 2 warnings

tests/unit/loader/test_torch_dataloader.py: 12 warnings

tests/unit/workflow/test_workflow.py: 9 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 2 files.

warnings.warn(


tests/unit/loader/test_torch_dataloader.py::test_horovod_multigpu

tests/unit/loader/test_torch_dataloader.py::test_distributed_multigpu

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 5 files.

warnings.warn(


tests/unit/ops/test_categorify.py::test_categorify_max_size[6]

tests/unit/ops/test_categorify.py::test_categorify_max_size[max_emb_size1]

/usr/lib/python3.8/site-packages/cudf/core/frame.py:3077: FutureWarning: keep_index is deprecated and will be removed in the future.

warnings.warn(


tests/unit/ops/test_fill.py::test_fill_missing[True-True-parquet]

tests/unit/ops/test_fill.py::test_fill_missing[True-False-parquet]

tests/unit/ops/test_ops.py::test_filter[parquet-0.1-True]

/var/jenkins_home/.local/lib/python3.8/site-packages/pandas/core/indexing.py:1732: SettingWithCopyWarning:

A value is trying to be set on a copy of a slice from a DataFrame


See the caveats in the documentation: https://pandas.pydata.org/pandas-docs/stable/user_guide/indexing.html#returning-a-view-versus-a-copy

self._setitem_single_block(indexer, value, name)


tests/unit/ops/test_ops.py::test_difference_lag[False]

/usr/lib/python3.8/site-packages/cudf/core/dataframe.py:3041: FutureWarning: The as_gpu_matrix method will be removed in a future cuDF release. Consider using to_cupy instead.

warnings.warn(


tests/unit/workflow/test_cpu_workflow.py: 6 warnings

tests/unit/workflow/test_workflow.py: 12 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 1 files did not have enough partitions to create 10 files.

warnings.warn(


tests/unit/workflow/test_workflow.py: 48 warnings

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 20 files.

warnings.warn(


tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_parquet_output[True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[True-True-None]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_WORKER]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-Shuffle.PER_PARTITION]

tests/unit/workflow/test_workflow.py::test_workflow_apply[False-True-None]

/core/merlin/io/dataset.py:863: UserWarning: Only created 2 files did not have enough partitions to create 4 files.

warnings.warn(


-- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html


---------- coverage: platform linux, python 3.8.10-final-0 -----------

Name                                                           Stmts   Miss Branch BrPart  Cover   Missing


examples/multi-gpu-movielens/torch_trainer.py                     65      0      6      1    99%   32->36

examples/multi-gpu-movielens/torch_trainer_dist.py                63      0      2      0   100%

nvtabular/init.py                                             22      2      0      0    91%   44-45

nvtabular/dispatch.py                                              3      3      0      0     0%   18-23

nvtabular/framework_utils/init.py                              2      0      0      0   100%

nvtabular/framework_utils/tensorflow/init.py                   1      0      0      0   100%

nvtabular/framework_utils/tensorflow/feature_column_utils.py     134     78     90     15    39%   30, 99, 103, 114-130, 140, 143-158, 162, 166-167, 173-198, 207-217, 220-227, 229->233, 234, 239-279, 282

nvtabular/framework_utils/tensorflow/layers/init.py            4      0      0      0   100%

nvtabular/framework_utils/tensorflow/layers/embedding.py         153     13     89      6    90%   60, 68->49, 122, 179, 231-239, 242, 335->343, 357->360, 363-364, 367

nvtabular/framework_utils/tensorflow/layers/interaction.py        47     25     22      1    45%   49, 74-103, 106-110, 113

nvtabular/framework_utils/tensorflow/layers/outer_product.py      30     24     12      0    19%   37-38, 41-60, 71-84, 87

nvtabular/framework_utils/tensorflow/tfrecords_to_parquet.py      58     58     30      0     0%   16-111

nvtabular/framework_utils/torch/init.py                        0      0      0      0   100%

nvtabular/framework_utils/torch/layers/init.py                 2      0      0      0   100%

nvtabular/framework_utils/torch/layers/embeddings.py              32      2     18      2    92%   50, 91

nvtabular/framework_utils/torch/models.py                         45      2     30     10    84%   55->57, 57->61, 61->63, 66->68, 87->89, 89->93, 93->96, 96->100, 103, 108

nvtabular/framework_utils/torch/utils.py                          75     19     34      9    69%   22, 25-33, 51->53, 53->55, 64, 71->76, 75, 107, 118-120, 124-127, 132->115

nvtabular/graph.py                                                 3      3      0      0     0%   18-23

nvtabular/inference/init.py                                    2      0      0      0   100%

nvtabular/inference/triton/init.py                            36     12     14      1    58%   42-49, 68, 72, 76-82

nvtabular/inference/triton/benchmarking_tools.py                  52     52     10      0     0%   2-103

nvtabular/inference/triton/data_conversions.py                    87      3     58      4    95%   32-33, 84

nvtabular/inference/triton/ensemble.py                           266    200     82      3    25%   73-112, 157-196, 240-288, 292-323, 381-389, 418-434, 477-520, 548-588, 594-610, 614-681, 711, 733, 739-758, 764-788, 793-796

nvtabular/inference/triton/model/init.py                       0      0      0      0   100%

nvtabular/inference/triton/model/model_pt.py                     101    101     42      0     0%   27-220

nvtabular/inference/triton/model_config_pb2.py                   299      0      2      0   100%

nvtabular/inference/triton/workflow_model.py                      55     55     24      0     0%   27-128

nvtabular/inference/workflow/init.py                           0      0      0      0   100%

nvtabular/inference/workflow/base.py                             113    113     62      0     0%   27-209

nvtabular/inference/workflow/hugectr.py                           37     37     16      0     0%   27-87

nvtabular/inference/workflow/pytorch.py                           10     10      6      0     0%   27-46

nvtabular/inference/workflow/tensorflow.py                        32     32     10      0     0%   26-68

nvtabular/io.py                                                    3      3      0      0     0%   18-23

nvtabular/loader/init.py                                       2      0      0      0   100%

nvtabular/loader/backend.py                                      371     19    154     15    94%   27-28, 112, 116->108, 142, 158-159, 299->301, 311-315, 362-363, 402->406, 403->402, 478, 482-483, 512, 588-589, 598, 624, 632

nvtabular/loader/tensorflow.py                                   179     26     60     10    85%   38-39, 74, 85-89, 101, 105, 115, 124, 329, 357, 368, 383-385, 414-416, 426-434, 437-440

nvtabular/loader/tf_utils.py                                      57     27     22      6    46%   31->34, 34->36, 41->43, 45, 46->67, 52-53, 61-63, 69-73, 88-93, 103-116

nvtabular/loader/torch.py                                         87     16     26      3    78%   28-30, 33-39, 114, 158-159, 164, 202, 205

nvtabular/ops/init.py                                         26      0      0      0   100%

nvtabular/ops/add_metadata.py                                     34      0     14      0   100%

nvtabular/ops/bucketize.py                                        40      9     20      3    73%   52-54, 58->exit, 61-64, 83-86

nvtabular/ops/categorify.py                                      656     71    348     50    86%   251, 253, 271, 275, 283, 291, 293, 320, 341-342, 391->395, 399-406, 452, 460, 483-484, 561-566, 637, 733, 750, 795, 873-874, 889-893, 894->858, 912, 920, 927->exit, 951, 954->957, 1006->1004, 1068-1070, 1073, 1075->1077, 1094->1098, 1100->1053, 1106-1109, 1121, 1125, 1129, 1136, 1141-1144, 1222, 1224, 1294->1317, 1300->1317, 1318-1323, 1368, 1381->1384, 1388->1393, 1392, 1398, 1401, 1409-1419

nvtabular/ops/clip.py                                             18      2      8      3    81%   44, 52->54, 55

nvtabular/ops/column_similarity.py                               121     26     38      5    74%   19-20, 29-30, 81->exit, 111, 206-207, 216-218, 226-242, 259->262, 263, 273

nvtabular/ops/data_stats.py                                       56      1     24      3    95%   107->109, 111, 113->103

nvtabular/ops/difference_lag.py                                   43      0     14      1    98%   73->75

nvtabular/ops/drop_low_cardinality.py                             18      0     10      1    96%   85->84

nvtabular/ops/dropna.py                                            9      0      2      0   100%

nvtabular/ops/fill.py                                             76      5     30      1    92%   63-67, 109

nvtabular/ops/filter.py                                           20      1      8      1    93%   49

nvtabular/ops/groupby.py                                         130      4     84      5    96%   72, 83, 93->95, 131, 221

nvtabular/ops/hash_bucket.py                                      43      1     22      2    95%   79, 118->124

nvtabular/ops/hashed_cross.py                                     38      3     17      3    89%   52, 64, 92

nvtabular/ops/join_external.py                                    96      8     34      8    88%   20-21, 114, 116, 118, 137->141, 150->152, 205-206, 216->227, 221

nvtabular/ops/join_groupby.py                                    128      5     57      6    94%   113, 120, 129, 136->135, 178->175, 181->175, 260-261

nvtabular/ops/lambdaop.py                                         62     13     22      3    71%   60, 64, 82, 94-109

nvtabular/ops/list_slice.py                                       89     29     42      0    64%   21-22, 146-160, 168-190

nvtabular/ops/logop.py                                            21      0      6      0   100%

nvtabular/ops/moments.py                                          69      0     24      0   100%

nvtabular/ops/normalize.py                                        93      4     22      1    94%   89, 139-140, 167

nvtabular/ops/operator.py                                         11      1      2      0    92%   52

nvtabular/ops/reduce_dtype_size.py                                49      0     20      2    97%   68->77, 74->77

nvtabular/ops/rename.py                                           29      3     14      3    86%   46, 71-73

nvtabular/ops/stat_operator.py                                     8      0      2      0   100%

nvtabular/ops/target_encoding.py                                 182      9     76      5    93%   168->172, 176->185, 274, 283-284, 297-303, 396->399

nvtabular/ops/value_counts.py                                     34      0      6      1    98%   40->38

nvtabular/tools/init.py                                        0      0      0      0   100%

nvtabular/tools/data_gen.py                                      271     25     90      7    91%   26-27, 31-32, 129-132, 142-146, 148, 170-171, 322, 332, 358, 361-370

nvtabular/tools/dataset_inspector.py                              52      8     24      2    79%   33-40, 51

nvtabular/tools/inspector_script.py                               46     46      0      0     0%   17-168

nvtabular/utils.py                                                 3      3      0      0     0%   18-23

nvtabular/worker.py                                                3      3      0      0     0%   18-23

nvtabular/workflow/init.py                                     2      0      0      0   100%

nvtabular/workflow/node.py                                         7      0      4      0   100%

nvtabular/workflow/workflow.py                                   219     17     94     12    91%   28-29, 52, 85, 206, 212->226, 239-241, 373, 388-389, 431, 508, 538, 548-550, 563


TOTAL                                                           5330   1232   2099    214    75%

Coverage XML written to file coverage.xml


Required test coverage of 70% reached. Total coverage: 74.71%

=========================== short test summary info ============================

SKIPPED [1] ../../../../../usr/lib/python3.8/site-packages/dask_cudf/io/tests/test_s3.py:16: could not import 's3fs': cannot import name 'ParamSpec' from 'typing_extensions' (/var/jenkins_home/.local/lib/python3.8/site-packages/typing_extensions.py)

SKIPPED [1] tests/unit/loader/test_tf_dataloader.py:529: not working correctly in ci environment

========== 1412 passed, 2 skipped, 666 warnings in 1527.10s (0:25:27) ==========

Performing Post build task...

Match found for : : True

Logical operation result is TRUE

Running script  : #!/bin/bash

cd /var/jenkins_home/

CUDA_VISIBLE_DEVICES=1 python test_res_push.py "https://api.GitHub.com/repos/NVIDIA-Merlin/NVTabular/issues/$ghprbPullId/comments" "/var/jenkins_home/jobs/$JOB_NAME/builds/$BUILD_NUMBER/log"

[nvtabular_tests] $ /bin/bash /tmp/jenkins1348328130137618929.sh

@karlhigley karlhigley merged commit 15664bc into NVIDIA-Merlin:main Apr 12, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@karlhigley karlhigley karlhigley approved these changes

Assignees

@jperez999 jperez999

Labels
enhancement New feature or request
Projects
None yet
Milestone
Merlin 22.05
3 participants
@jperez999 @nvidia-merlin-bot @karlhigley