Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

enhance(client): embed model serving spec into job spec #3032

Merged
merged 1 commit into from
Nov 27, 2023

Conversation

jialeicui
Copy link
Contributor

@jialeicui jialeicui commented Nov 27, 2023

Description

Generated job spec in example/web_handler:

serving:
- concurrency: 1
  expose: 8080
  extra_kwargs:
    search_modules:
    - main
  func_name: StandaloneModel._serve_handler
  module_name: starwhale.core.model.model
  name: serving
  replicas: 1
  service_spec:
    apis:
    - components:
      - component_spec_value_type: INT
        name: top_k
      - component_spec_value_type: LIST
        name: history
      - component_spec_value_type: INT
        name: max_new_tokens
      - component_spec_value_type: FLOAT
        name: top_p
      - component_spec_value_type: STRING
        name: user_input
      - component_spec_value_type: FLOAT
        name: temperature
      inference_type: llm_chat
      uri: fake_chat_bot
    version: 0.0.2
  show_name: virtual handler for model serving
  virtual: true

Modules

  • UI
  • Controller
  • Agent
  • Client
  • Python-SDK
  • Others

Checklist

  • run code format and lint check
  • add unit test
  • add necessary doc

Copy link

codecov bot commented Nov 27, 2023

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (39bc0bc) 82.74% compared to head (96c92fd) 82.81%.

Additional details and impacted files
@@             Coverage Diff              @@
##               main    #3032      +/-   ##
============================================
+ Coverage     82.74%   82.81%   +0.07%     
+ Complexity     3157     3156       -1     
============================================
  Files           558      556       -2     
  Lines         30635    30429     -206     
  Branches       1838     1777      -61     
============================================
- Hits          25350    25201     -149     
+ Misses         4473     4418      -55     
+ Partials        812      810       -2     
Flag Coverage Δ
console ?
controller 73.92% <100.00%> (+<0.01%) ⬆️
standalone 91.90% <100.00%> (-0.02%) ⬇️
unittests 91.90% <100.00%> (+0.26%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

Copy link
Contributor

@goldenxinxing goldenxinxing left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@goldenxinxing goldenxinxing merged commit 94fe64d into star-whale:main Nov 27, 2023
38 checks passed
@jialeicui jialeicui deleted the svc-in-job-yaml branch November 27, 2023 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants