Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

RHOAIENG-15105: Add offline support for LMEvalJobs #351

Conversation

ruivieira
Copy link
Member

Refers to RHOAIENG-15105.

Add support for offline flags in the LMEvalJob container and mount PVC containing models and datasets.

If the optional offline field is set:

 offline:
    storage:
      pvcName: pvc-with-my-models

offline mode will be enabled. If it is missing, offline mode will be disable.

@ruivieira ruivieira added kind/enhancement New feature or request lm-eval Issues related to LM-Eval labels Nov 4, 2024
@ruivieira ruivieira self-assigned this Nov 4, 2024
@openshift-ci-robot
Copy link
Collaborator

openshift-ci-robot commented Nov 4, 2024

@ruivieira: This pull request references RHOAIENG-15105 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the task to target the "4.18.0" version, but no target version was set.

In response to this:

Refers to RHOAIENG-15105.

Add support for offline flags in the LMEvalJob container and mount PVC containing models and datasets.

If the optional offline field is set:

offline:
   storage:
     pvcName: pvc-with-my-models

offline mode will be enabled. If it is missing, offline mode will be disable.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@ruivieira ruivieira linked an issue Nov 4, 2024 that may be closed by this pull request
Copy link

github-actions bot commented Nov 4, 2024

PR image build and manifest generation completed successfully!

📦 PR image: quay.io/trustyai/trustyai-service-operator-ci:8ed4f173bf593abf2915217f0d30d9a4de433e44

📦 LMES driver image: quay.io/trustyai/ta-lmes-driver:8ed4f173bf593abf2915217f0d30d9a4de433e44

📦 LMES job image: quay.io/trustyai/ta-lmes-job:8ed4f173bf593abf2915217f0d30d9a4de433e44

🗂️ CI manifests

devFlags:
  manifests:
    - contextDir: config
      sourcePath: ''
      uri: https://api.github.com/repos/trustyai-explainability/trustyai-service-operator-ci/tarball/operator-8ed4f173bf593abf2915217f0d30d9a4de433e44

Copy link

openshift-ci bot commented Nov 4, 2024

@ruivieira: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/trustyai-service-operator-e2e 8ed4f17 link true /test trustyai-service-operator-e2e

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Copy link
Collaborator

@yhwang yhwang left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/LGTM

While reviewing the code, it reminded me of one thing not directly related to this change. I believe it would be good to create an issue regarding enabling the validation webhook for LMES controller. Then we can enable some tedious value checking there.

Copy link

openshift-ci bot commented Nov 4, 2024

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: yhwang

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@ruivieira
Copy link
Member Author

@yhwang Opened #353

@ruivieira ruivieira merged commit f5163e9 into trustyai-explainability:main Nov 4, 2024
5 of 10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement New feature or request lgtm lm-eval Issues related to LM-Eval ok-to-test
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add LM-Eval support for disconnected clusters
3 participants