Add Operator for Implicit Models #134
Conversation
oliverholworthy
added
enhancement
Click to view CI ResultsGitHub pull request #134 of commit 0e2a7dab4b7303ac973cbe834075fc01bb6602af, no merge conflicts.
Running as SYSTEM
Setting status of 0e2a7dab4b7303ac973cbe834075fc01bb6602af to PENDING with url https://10.20.13.93:8080/job/merlin_systems/130/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 0e2a7dab4b7303ac973cbe834075fc01bb6602af^{commit} # timeout=10
Checking out Revision 0e2a7dab4b7303ac973cbe834075fc01bb6602af (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 0e2a7dab4b7303ac973cbe834075fc01bb6602af # timeout=10
Commit message: "Add docstrings to OperatorRunner and pass through model config"
> git rev-list --no-walk 609cc61933ae594080de9e1c48004cac1a0da7a2 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins10727564167416475803.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items / 1 skipped
|
Click to view CI ResultsGitHub pull request #134 of commit eee034fdf579d5e62575b89159f76fc22aaed600, no merge conflicts.
Running as SYSTEM
Setting status of eee034fdf579d5e62575b89159f76fc22aaed600 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/131/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse eee034fdf579d5e62575b89159f76fc22aaed600^{commit} # timeout=10
Checking out Revision eee034fdf579d5e62575b89159f76fc22aaed600 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f eee034fdf579d5e62575b89159f76fc22aaed600 # timeout=10
Commit message: "Add workflow for implicit op"
> git rev-list --no-walk 0e2a7dab4b7303ac973cbe834075fc01bb6602af # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins7531757995567704480.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items / 1 skipped
|
Click to view CI ResultsGitHub pull request #134 of commit d2a0476dacce18146b10bd56bb8d0d4745315d07, no merge conflicts.
Running as SYSTEM
Setting status of d2a0476dacce18146b10bd56bb8d0d4745315d07 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/132/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse d2a0476dacce18146b10bd56bb8d0d4745315d07^{commit} # timeout=10
Checking out Revision d2a0476dacce18146b10bd56bb8d0d4745315d07 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f d2a0476dacce18146b10bd56bb8d0d4745315d07 # timeout=10
Commit message: "Handle kwargs in all from_config operator methods"
> git rev-list --no-walk eee034fdf579d5e62575b89159f76fc22aaed600 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins17994122288879948751.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items / 1 skipped
|
Documentation preview |
Click to view CI ResultsGitHub pull request #134 of commit 285d68e8317a62301710af3827be8d10fa5b2675, no merge conflicts.
Running as SYSTEM
Setting status of 285d68e8317a62301710af3827be8d10fa5b2675 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/133/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 285d68e8317a62301710af3827be8d10fa5b2675^{commit} # timeout=10
Checking out Revision 285d68e8317a62301710af3827be8d10fa5b2675 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 285d68e8317a62301710af3827be8d10fa5b2675 # timeout=10
Commit message: "Remove depencency on merlin models for tests"
> git rev-list --no-walk d2a0476dacce18146b10bd56bb8d0d4745315d07 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins15980071591452208488.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items / 1 skipped
|
Click to view CI ResultsGitHub pull request #134 of commit 25755796a4a2239c32167236b19a2b63c95d66cc, no merge conflicts.
Running as SYSTEM
Setting status of 25755796a4a2239c32167236b19a2b63c95d66cc to PENDING with url https://10.20.13.93:8080/job/merlin_systems/134/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 25755796a4a2239c32167236b19a2b63c95d66cc^{commit} # timeout=10
Checking out Revision 25755796a4a2239c32167236b19a2b63c95d66cc (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 25755796a4a2239c32167236b19a2b63c95d66cc # timeout=10
Commit message: "Add "als" to ci/ignore_codespell_words.txt"
> git rev-list --no-walk 285d68e8317a62301710af3827be8d10fa5b2675 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins16496730202414173715.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items / 1 skipped
|
merlin/systems/dag/op_runner.py
Outdated
| @@ -30,14 +33,21 @@ def __init__(self, config, repository="./", version=1, kind=""): | |||
| op_module = importlib.import_module(module_name) | |||
| op_class = getattr(op_module, class_name) | |||
|
|
|||
| operator = op_class.from_config(op_config) | |||
| operator = op_class.from_config( | |||
There was a problem hiding this comment.
@karlhigley In order to load an artifact required by an operator, passing the model_* related params through to all ops that use the op runner here. I'm guessing this may have considered before because the existing OperatorRunner accepts a model repository and version (though currently unused). Does this seem like a reasonable thing to do?
|
The ensemble tests for the implicit op has been skipped in Jenkins because I think the implicit package needs to be installed/updated. |
Click to view CI ResultsGitHub pull request #134 of commit 62468f5f5103beaa39a10c333bcb97960611a075, no merge conflicts.
Running as SYSTEM
Setting status of 62468f5f5103beaa39a10c333bcb97960611a075 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/135/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 62468f5f5103beaa39a10c333bcb97960611a075^{commit} # timeout=10
Checking out Revision 62468f5f5103beaa39a10c333bcb97960611a075 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 62468f5f5103beaa39a10c333bcb97960611a075 # timeout=10
Commit message: "Add 'n' (number of items to recommend) to the inputs of the op"
> git rev-list --no-walk 25755796a4a2239c32167236b19a2b63c95d66cc # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins4185695878036075395.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 47 items / 1 skipped
|
| Returns a transformed dataframe for this operator""" | ||
| user_id = df["user_id"][0] | ||
| num_to_recommend = df["n"][0] | ||
| user_items = None |
There was a problem hiding this comment.
could move this to the inputs if it's a common pattern for users to pass in. My understanding is that this mostly serves as a mechanism to filter out items already seen.
There was a problem hiding this comment.
I think this is ok - user_items can be left out if the filter_already_liked_items is False, and recalculate_user is also False (otherwise we need to know what the items the user has liked).
| Returns a transformed dataframe for this operator""" | ||
| user_id = df["user_id"][0] | ||
| num_to_recommend = df["n"][0] | ||
| user_items = None |
There was a problem hiding this comment.
I think this is ok - user_items can be left out if the filter_already_liked_items is False, and recalculate_user is also False (otherwise we need to know what the items the user has liked).
merlin/systems/dag/ops/faiss.py
Outdated
| @@ -59,7 +59,7 @@ def __init__(self, index_path, topk=10): | |||
| super().__init__() | |||
|
|
|||
| @classmethod | |||
| def from_config(cls, config: dict) -> "QueryFaiss": | |||
| def from_config(cls, config: dict, **kwargs) -> "QueryFaiss": | |||
There was a problem hiding this comment.
Whats the benefit of adding the kwargs here ?
There was a problem hiding this comment.
This is required with the change made in op_runner.py. The from_config now receives model_repository, model_name, and model_version keyword arguments which enables us to read files stored alongside the model config.
This QueryFaiss operator in this case relies on the config containing the full path to the index file currently. However with this extra information could be updated to read from a location relative to the model repository and model config.
It could be clearer to update this **kwargs to specify the named arguments explicitly here (and in the other from_config methods.)
merlin/systems/dag/ops/implicit.py
Outdated
| class PredictImplicit(PipelineableInferenceOperator): | ||
| """Operator for running inference on Implicit models..""" | ||
|
|
||
| def __init__(self, model, **kwargs): |
There was a problem hiding this comment.
The PredictTensorflow op has model_or_path as its param and will load the model from a path if that's what is provided. I think we should try to be consistent with the constructor functionality, and PredictTensorflow is probably the most mature/popular, so matching that seems like a good idea to me.
There was a problem hiding this comment.
👍 I agree that having consistency across these Predict ops is a good idea.
However, the implicit serialization format doesn't appear to have a mechanism for knowing which model to load.
e.g. if you have created a model and saved to a file, for example model.npz.
import implicit
# random training data
n = 100 # number of both users and items
user_items = csr_matrix(np.random.choice([0, 1], size=n * n, p=[0.9, 0.1]).reshape(n, n))
model = implicit.bpr.BayesianPersonalizedRanking()
model.fit(user_items)
model.save("model.npz") # save model to fileto call the load method we require knowledge of the kind of model to be loaded. There's not an implicit.load function that I can see (correct me if there is something I've missed @benfred )
implicit.gpu.bpr.BayesianPersonalizedRanking.load("model.npz")We could change the signature to model_or_path at a later stage if the functionality becomes available and that is our preferred option for these predict ops.
Click to view CI ResultsGitHub pull request #134 of commit 4bd4c30d814ecc74005582cf1d177064c81184e7, no merge conflicts.
Running as SYSTEM
Setting status of 4bd4c30d814ecc74005582cf1d177064c81184e7 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/189/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 4bd4c30d814ecc74005582cf1d177064c81184e7^{commit} # timeout=10
Checking out Revision 4bd4c30d814ecc74005582cf1d177064c81184e7 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 4bd4c30d814ecc74005582cf1d177064c81184e7 # timeout=10
Commit message: "Add check for implicit version"
> git rev-list --no-walk 5ea4551c2082e2c308ce604baa76edc006dd9b91 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins10946567777681847258.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 53 items / 1 skipped
|
Click to view CI ResultsGitHub pull request #134 of commit d591a110e397fa1bff2f03ab4659b62abc9170ac, no merge conflicts.
Running as SYSTEM
Setting status of d591a110e397fa1bff2f03ab4659b62abc9170ac to PENDING with url https://10.20.13.93:8080/job/merlin_systems/191/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse d591a110e397fa1bff2f03ab4659b62abc9170ac^{commit} # timeout=10
Checking out Revision d591a110e397fa1bff2f03ab4659b62abc9170ac (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f d591a110e397fa1bff2f03ab4659b62abc9170ac # timeout=10
Commit message: "Uncomment ensemble tests for als/lmf"
> git rev-list --no-walk 476d3914b2f69960dd7252b706dd787bccf26988 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins12045396800483704919.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 53 items / 1 skipped
|
Click to view CI ResultsGitHub pull request #134 of commit 94d46c58ad0c63bdabd476347fd0b3a68eaded3c, no merge conflicts.
Running as SYSTEM
Setting status of 94d46c58ad0c63bdabd476347fd0b3a68eaded3c to PENDING with url https://10.20.13.93:8080/job/merlin_systems/192/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 94d46c58ad0c63bdabd476347fd0b3a68eaded3c^{commit} # timeout=10
Checking out Revision 94d46c58ad0c63bdabd476347fd0b3a68eaded3c (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 94d46c58ad0c63bdabd476347fd0b3a68eaded3c # timeout=10
Commit message: "Rename test for config for clarity"
> git rev-list --no-walk d591a110e397fa1bff2f03ab4659b62abc9170ac # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins4520550627757499299.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 53 items / 1 skipped
|
|
Click to view CI ResultsGitHub pull request #134 of commit 413f554d429c604f16fbadefa77d0bb7de10e49d, no merge conflicts.
Running as SYSTEM
Setting status of 413f554d429c604f16fbadefa77d0bb7de10e49d to PENDING with url https://10.20.13.93:8080/job/merlin_systems/193/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 413f554d429c604f16fbadefa77d0bb7de10e49d^{commit} # timeout=10
Checking out Revision 413f554d429c604f16fbadefa77d0bb7de10e49d (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 413f554d429c604f16fbadefa77d0bb7de10e49d # timeout=10
Commit message: "Specify low for random num_to_recommend to avoid zero"
> git rev-list --no-walk 94d46c58ad0c63bdabd476347fd0b3a68eaded3c # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins15738677089271391124.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 53 items / 1 skipped
|
Click to view CI ResultsGitHub pull request #134 of commit 8ba0eb40a13c8194fd45133344062480c489712b, no merge conflicts.
Running as SYSTEM
Setting status of 8ba0eb40a13c8194fd45133344062480c489712b to PENDING with url https://10.20.13.93:8080/job/merlin_systems/252/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 8ba0eb40a13c8194fd45133344062480c489712b^{commit} # timeout=10
Checking out Revision 8ba0eb40a13c8194fd45133344062480c489712b (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 8ba0eb40a13c8194fd45133344062480c489712b # timeout=10
Commit message: "Merge branch 'main' into op-implicit"
> git rev-list --no-walk 88e7864addd5809fd75b7554e85701eb9c1f26c4 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins1387852930417552259.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 81 items
|
Click to view CI ResultsGitHub pull request #134 of commit 50855cccba4dd20a8759095bfee872d76e71a4b7, no merge conflicts.
Running as SYSTEM
Setting status of 50855cccba4dd20a8759095bfee872d76e71a4b7 to PENDING with url https://10.20.13.93:8080/job/merlin_systems/255/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse 50855cccba4dd20a8759095bfee872d76e71a4b7^{commit} # timeout=10
Checking out Revision 50855cccba4dd20a8759095bfee872d76e71a4b7 (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f 50855cccba4dd20a8759095bfee872d76e71a4b7 # timeout=10
Commit message: "Remove workflow for implicit and add package to requirements-test"
> git rev-list --no-walk 0cc3b15d26c3b3a531895f4649a68178fb085b20 # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins1493194512218969676.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 81 items
|
| return str(model_repository_path) | ||
| # instead of path to model directory within the model repository | ||
| if model_repository.endswith(".py"): | ||
| return str(pathlib.Path(model_repository).parent.parent.parent) |
There was a problem hiding this comment.
example:
import pathlib
pathlib.Path("/tmp/my_model_repository/my_model_name/1/model.py").parent.parent.parent
# => PosixPath('/tmp/my_model_repository')
|
rerun tests |
Click to view CI ResultsGitHub pull request #134 of commit a4990ac3a4287173faa8c24cd0df79d1c217937a, no merge conflicts.
GitHub pull request #134 of commit a4990ac3a4287173faa8c24cd0df79d1c217937a, no merge conflicts.
Running as SYSTEM
Setting status of a4990ac3a4287173faa8c24cd0df79d1c217937a to PENDING with url https://10.20.13.93:8080/job/merlin_systems/260/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse a4990ac3a4287173faa8c24cd0df79d1c217937a^{commit} # timeout=10
Checking out Revision a4990ac3a4287173faa8c24cd0df79d1c217937a (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f a4990ac3a4287173faa8c24cd0df79d1c217937a # timeout=10
Commit message: "Correct function handling TritonPythonModel model_repository"
> git rev-list --no-walk a4990ac3a4287173faa8c24cd0df79d1c217937a # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins6225257845585263035.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 81 items
|
|
@oliverholworthy It looks like this is failing on an XGBoost test, but not I'm sure why. It can't find the ensemble, but I'm not sure if that's an issue with the test or an issue with the environment |
|
rerun tests |
|
@karlhigley triggered another run. This might be something to do with the triton isolation issue (the reason #141 can't be merged - as it is currently written) |
|
That was my guess too, probably due to the PR stampede now that CI is unblocked 👍🏻 |
Click to view CI ResultsGitHub pull request #134 of commit a4990ac3a4287173faa8c24cd0df79d1c217937a, no merge conflicts.
GitHub pull request #134 of commit a4990ac3a4287173faa8c24cd0df79d1c217937a, no merge conflicts.
Running as SYSTEM
Setting status of a4990ac3a4287173faa8c24cd0df79d1c217937a to PENDING with url https://10.20.13.93:8080/job/merlin_systems/264/console and message: 'Pending'
Using context: Jenkins
Building on master in workspace /var/jenkins_home/workspace/merlin_systems
using credential fce1c729-5d7c-48e8-90cb-b0c314b1076e
> git rev-parse --is-inside-work-tree # timeout=10
Fetching changes from the remote Git repository
> git config remote.origin.url https://github.com/NVIDIA-Merlin/systems # timeout=10
Fetching upstream changes from https://github.com/NVIDIA-Merlin/systems
> git --version # timeout=10
using GIT_ASKPASS to set credentials login for merlin-systems user + githubtoken
> git fetch --tags --force --progress -- https://github.com/NVIDIA-Merlin/systems +refs/pull/134/*:refs/remotes/origin/pr/134/* # timeout=10
> git rev-parse a4990ac3a4287173faa8c24cd0df79d1c217937a^{commit} # timeout=10
Checking out Revision a4990ac3a4287173faa8c24cd0df79d1c217937a (detached)
> git config core.sparsecheckout # timeout=10
> git checkout -f a4990ac3a4287173faa8c24cd0df79d1c217937a # timeout=10
Commit message: "Correct function handling TritonPythonModel model_repository"
> git rev-list --no-walk 90ad1f220fde95a6a6ed23f3b5fa4fd11175865d # timeout=10
[merlin_systems] $ /bin/bash /tmp/jenkins6790811923613998494.sh
PYTHONPATH=:/usr/local/lib/python3.8/dist-packages/:/usr/local/hugectr/lib:/var/jenkins_home/workspace/merlin_systems/systems
============================= test session starts ==============================
platform linux -- Python 3.8.10, pytest-7.1.2, pluggy-1.0.0
rootdir: /var/jenkins_home/workspace/merlin_systems/systems, configfile: pyproject.toml
plugins: anyio-3.6.1, xdist-2.5.0, forked-1.4.0, cov-3.0.0
collected 81 items
|
Adds an Operator for Implicit Models
PredictImplicit.Self-contained operator for serving implicit models.