feat: add nebullvm as backend #697

diegofiori · 2022-04-27T11:31:33Z

Add nebullvm as a model backend in a similar way to ONNXruntime backend

numb3r3 · 2022-04-27T12:12:45Z

awsome job. The current PR cannot pass CI checking. Please follow this guidelines https://github.com/jina-ai/jina/blob/master/CONTRIBUTING.md

numb3r3 · 2022-04-27T12:14:05Z

BTW., please also use pre-commit to format your codes

$ pre-commit install
$ git commit -am "feat: nebullm backend"

numb3r3

please check my comments 🍻

hanxiao · 2022-04-27T16:50:53Z

hi @morgoth95 is it possible to open PR modification permission to us? we could take care of it on the styling and minors things. Reference here: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork

diegofiori · 2022-04-27T17:04:50Z

hi @morgoth95 is it possible to open PR modification permission to us? we could take care of it on the styling and minors things. Reference here: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/allowing-changes-to-a-pull-request-branch-created-from-a-fork

Hello @hanxiao! Actually, I already activated the option "Allow edits by maintainers" when I created the PR 😄 In theory all of you with writing access to clip-as-a-service should be able to directly push new commits to my branch 😉

numb3r3 · 2022-04-28T02:43:28Z

README.md

@@ -45,13 +45,20 @@ curl -X POST http://demo-cas.jina.ai:51001/post -H 'Content-Type: application/js
 ## Install

 CLIP-as-service consists of two Python packages `clip-server` and `clip-client` that can be installed _independently_. Both require Python 3.7+. 
+`nebullvm` installation will need on some platforms python 


This sentence is more about the setup. It's better to be placed in the following Install server section.

fixed in 01f7f08

server/clip_server/executors/clip_nebullvm.py

server/clip_server/model/clip_nebullvm.py

numb3r3

And please add unit test for nebullvm

numb3r3 · 2022-05-05T02:56:06Z

docs/user-guides/server.md


+python -m clip_server nebullvm_flow.yml


should be nebullvm-flow.yml

solved in f06133e

numb3r3 · 2022-05-05T03:10:48Z

server/clip_server/model/clip_nebullvm.py

+
+    def __enter__(self):
+        if self.device == "cpu" and torch.cuda.is_available():
+            self.cuda_str = os.environ.get("CUDA_VISIBLE_DEVICES") or "1"


Is this a bug? maybe we can use self.cuda_str = os.environ.get('CUDA_VISIBLE_DEVICES', None), otherwise the CUDA_VISIBLE_DEVICES will be updated to "1" at exiting.

Solved in f9ce0ae

numb3r3 · 2022-05-09T06:42:43Z

@morgoth95 how is going now? Just wondering do we have the chance to merge this PR this week?

diegofiori · 2022-05-10T09:54:56Z

@morgoth95 how is going now? Just wondering do we have the chance to merge this PR this week?

Sorry for the delay, we were working on the latest release of nebullvm! I should have fixed the issues you pointed out! I think we can try to accelerate the process and merge the PR this week.

numb3r3 · 2022-05-11T09:10:24Z

@morgoth95 Congrats. nebullvm v0.3.0 comes yesterday, very cool. And there are still some bad commit messages in this PR, to fix them please refer to jina-ai/serve#553

numb3r3 · 2022-05-12T02:27:07Z

FYI, this PR has one ore more bad commit messages,


⧗   input: docs: Fix typo in docs
✖   subject must not be sentence-case, start-case, pascal-case, upper-case [subject-case]

diegofiori · 2022-05-13T13:19:17Z

@numb3r3

FYI, this PR has one ore more bad commit messages,


⧗   input: docs: Fix typo in docs
✖   subject must not be sentence-case, start-case, pascal-case, upper-case [subject-case]

it should have been fixed now! Let me know if further changes are needed

numb3r3 · 2022-05-16T02:46:38Z

@morgoth95 Please rebase this PR so that it does not show unwanted commits

numb3r3 · 2022-05-17T08:20:05Z

@morgoth95 please remember to install pre-commit to format your code before commits:

$ pre-commit install
$ git commit -am "fix: xxxx"

numb3r3

please format your code via:

$ pre-commit run --all

diegofiori · 2022-05-18T06:57:16Z

please format your code via:
$ pre-commit run --all

Solved in 932cfe6

codecov · 2022-05-18T07:45:04Z

Codecov Report

Merging #697 (932cfe6) into main (87fdc54) will decrease coverage by 33.71%.
The diff coverage is 0.00%.

❗ Current head 932cfe6 differs from pull request most recent head dce8c09. Consider uploading reports for the commit dce8c09 to get more accurate results

@@             Coverage Diff             @@
##             main     #697       +/-   ##
===========================================
- Coverage   81.58%   47.87%   -33.72%     
===========================================
  Files          21       18        -3     
  Lines        1575     1201      -374     
===========================================
- Hits         1285      575      -710     
- Misses        290      626      +336

Flag	Coverage Δ
cas	`47.87% <0.00%> (-33.72%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
server/clip_server/executors/clip_nebullvm.py	`0.00% <0.00%> (ø)`
server/clip_server/model/clip_nebullvm.py	`0.00% <0.00%> (ø)`
server/clip_server/executors/clip_torch.py	`0.00% <0.00%> (-83.34%)`	⬇️
server/clip_server/executors/clip_onnx.py	`0.00% <0.00%> (-81.95%)`	⬇️
server/clip_server/model/model.py	`15.83% <0.00%> (-54.03%)`	⬇️
server/clip_server/model/clip.py	`47.85% <0.00%> (-39.65%)`	⬇️
client/clip_client/client.py	`64.83% <0.00%> (-23.95%)`	⬇️
server/clip_server/model/clip_onnx.py	`52.00% <0.00%> (-20.73%)`	⬇️
server/clip_server/executors/helper.py	`91.22% <0.00%> (-5.84%)`	⬇️
server/clip_server/model/simple_tokenizer.py	`93.87% <0.00%> (-1.03%)`	⬇️
... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

numb3r3 · 2022-05-20T02:34:16Z

@morgoth95 The unittest seems broken

numb3r3 · 2022-06-01T04:24:25Z

@morgoth95 Hi, I hope you are well. Did you get a chance to continue this PR to pass the unit test?

diegofiori · 2022-06-02T13:40:48Z

@morgoth95 Hi, I hope you are well. Did you get a chance to continue this PR to pass the unit test?

Hi Felix, I'm working on the tests. A couple of days and I'll close the open tasks. Looking forward to the integration!

diegofiori · 2022-06-28T20:51:12Z

@numb3r3 Hi, I hope all is fine on the Jina side. I took a look at the failed tests and I may have identified the source of the problem. Unfortunately it will require some work on our end, but we think it will be fixed with the next nebullvm release (it will happen in about a week ;)). I will get back to you as soon as the problem is fixed :)

numb3r3 · 2022-06-30T02:37:40Z

@morgoth95 very nice, looking forward the next nebullvm release.

numb3r3 requested changes Apr 27, 2022

View reviewed changes

diegofiori force-pushed the main branch from 3be83e4 to 611a441 Compare April 27, 2022 15:34

diegofiori requested a review from numb3r3 April 27, 2022 16:23

numb3r3 requested changes Apr 28, 2022

View reviewed changes

diegofiori requested a review from numb3r3 May 3, 2022 08:00

numb3r3 requested changes May 5, 2022

View reviewed changes

diegofiori requested a review from numb3r3 May 10, 2022 09:55

diegofiori force-pushed the main branch 2 times, most recently from c0808e0 to b42d031 Compare May 11, 2022 16:33

diegofiori force-pushed the main branch from b42d031 to d2112b4 Compare May 12, 2022 17:13

diegofiori force-pushed the main branch from 081d0ce to 8575384 Compare May 17, 2022 12:12

numb3r3 requested changes May 18, 2022

View reviewed changes

diegofiori requested a review from numb3r3 May 18, 2022 06:57

jina-bot and others added 14 commits October 16, 2022 19:09

doc: add nebullvm to docs

ad3737d

style: adapt code to jina's style

a3c9bdc

docs: add python versions supported by nebullvm

ba1bc5f

feat: allow user to select the runtime device

8e3e8bb

test: add tests with nebullvm

f93bee2

fix: fix bug with cuda devices in clip-nebullvm

5930aa3

docs: add nebullvm in all the docs

5544a28

build: add nebullvm to the list of requirements

6089275

refactor: adapt nebullvm executor to the new interface

085f865

ci: add nebullvm installation to tests in ci/cd

4ca4300

style: fix format error

f110021

fix: fix nebullvm api

44634cf

fix: removed unnecessary param

638e538

fix: added support for dynamic shape

fbbd827

diegofiori force-pushed the main branch from b2fd227 to fbbd827 Compare October 16, 2022 17:32

fix: adapt code to the new structure

dce8c09

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: add nebullvm as backend #697

feat: add nebullvm as backend #697

diegofiori commented Apr 27, 2022

numb3r3 commented Apr 27, 2022

numb3r3 commented Apr 27, 2022

numb3r3 left a comment

hanxiao commented Apr 27, 2022

diegofiori commented Apr 27, 2022

numb3r3 Apr 28, 2022

diegofiori Apr 30, 2022

numb3r3 left a comment

numb3r3 May 5, 2022

diegofiori May 10, 2022

numb3r3 May 5, 2022

diegofiori May 10, 2022

numb3r3 commented May 9, 2022

diegofiori commented May 10, 2022

numb3r3 commented May 11, 2022

numb3r3 commented May 12, 2022

diegofiori commented May 13, 2022

numb3r3 commented May 16, 2022

numb3r3 commented May 17, 2022

numb3r3 left a comment

diegofiori commented May 18, 2022

codecov bot commented May 18, 2022 •

edited

Loading

numb3r3 commented May 20, 2022

numb3r3 commented Jun 1, 2022

diegofiori commented Jun 2, 2022

diegofiori commented Jun 28, 2022

numb3r3 commented Jun 30, 2022

feat: add nebullvm as backend #697

Are you sure you want to change the base?

feat: add nebullvm as backend #697

Conversation

diegofiori commented Apr 27, 2022

numb3r3 commented Apr 27, 2022

numb3r3 commented Apr 27, 2022

numb3r3 left a comment

Choose a reason for hiding this comment

hanxiao commented Apr 27, 2022

diegofiori commented Apr 27, 2022

numb3r3 Apr 28, 2022

Choose a reason for hiding this comment

diegofiori Apr 30, 2022

Choose a reason for hiding this comment

numb3r3 left a comment

Choose a reason for hiding this comment

numb3r3 May 5, 2022

Choose a reason for hiding this comment

diegofiori May 10, 2022

Choose a reason for hiding this comment

numb3r3 May 5, 2022

Choose a reason for hiding this comment

diegofiori May 10, 2022

Choose a reason for hiding this comment

numb3r3 commented May 9, 2022

diegofiori commented May 10, 2022

numb3r3 commented May 11, 2022

numb3r3 commented May 12, 2022

diegofiori commented May 13, 2022

numb3r3 commented May 16, 2022

numb3r3 commented May 17, 2022

numb3r3 left a comment

Choose a reason for hiding this comment

diegofiori commented May 18, 2022

codecov bot commented May 18, 2022 • edited Loading

Codecov Report

numb3r3 commented May 20, 2022

numb3r3 commented Jun 1, 2022

diegofiori commented Jun 2, 2022

diegofiori commented Jun 28, 2022

numb3r3 commented Jun 30, 2022

codecov bot commented May 18, 2022 •

edited

Loading