Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

fix: Start triton server via bash -c tritonserver instead of just tritonserver #5030

Merged
merged 2 commits into from
Oct 9, 2023
Merged

fix: Start triton server via bash -c tritonserver instead of just tritonserver #5030

merged 2 commits into from
Oct 9, 2023

Conversation

SDJustus
Copy link

What this PR does / why we need it:
adjusted the components.yaml and the corresponding helm chart file, to be able to launch newer triton versions with gpu support, according to this issue in the triton repository.

Which issue(s) this PR fixes:

Fixes #5029

Special notes for your reviewer:

@seldondev
Copy link
Collaborator

Hi @SDJustus. Thanks for your PR.

I'm waiting for a SeldonIO or todo member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the jenkins-x/lighthouse repository.

@SDJustus SDJustus changed the title (fix): Start triton server via bash -c tritonserver instead of just tritonserver fix: Start triton server via bash -c tritonserver instead of just tritonserver Jul 19, 2023
@ukclivecox
Copy link
Contributor

@SDJustus As at present we create the yaml/helm from kustomize you will actually need to:

  1. change

    seldon-core/operator/config/serverconfigs/triton.yaml

    Lines 118 to 126 in ea5f5d4

    command:
    - /opt/tritonserver/bin/tritonserver
    args:
    - --model-repository=$(SERVER_MODELS_DIR)
    - --http-port=$(SERVER_HTTP_PORT)
    - --grpc-port=$(SERVER_GRPC_PORT)
    - --log-verbose=1
    - --model-control-mode=explicit
    - --backend-config=python,shm-default-byte-size=16777216
  2. Rerun

    seldon-core/k8s/Makefile

    Line 10 in ea5f5d4

    create: create-helm-charts create-yaml

In future we plan for the helm to be the source of truth,

@ukclivecox ukclivecox self-requested a review July 21, 2023 08:01
@SDJustus SDJustus closed this Jul 21, 2023
@SDJustus SDJustus reopened this Jul 21, 2023
@SDJustus
Copy link
Author

Reopened the PR with new commits, which should generate the helm charts the correct way. Let me know, if i need to change more.

@SDJustus
Copy link
Author

SDJustus commented Jul 24, 2023 •
edited
Loading

@cliveseldon as i accidentally closed this PR and reopened it afterwards, just a ping for you so we could set the ready to test flag

@SDJustus
Copy link
Author

SDJustus commented Sep 4, 2023

Any updates on this? Maybe @cliveseldon

@SDJustus
Copy link
Author

SDJustus commented Oct 8, 2023

Hi,
Any updates on this?

@ukclivecox
Copy link
Contributor

/ok-to-test

@ukclivecox ukclivecox merged commit 932738a into SeldonIO:v2 Oct 9, 2023
RafalSkolasinski pushed a commit that referenced this pull request Jan 22, 2024
@SDJustus @sakoush
fix: Start triton server via bash -c tritonserver instead of just tri…
3a25e76 
…tonserver (#5030)

* Update triton.yaml for issue #5029

* generating new helm-charts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@ukclivecox ukclivecox ukclivecox approved these changes

Assignees
No one assigned
Labels
ok-to-test
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@SDJustus @seldondev @ukclivecox