Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI][Caffe Frontend] add caffe environment #6023

Merged
merged 5 commits into from
Jul 28, 2020
Merged

Conversation

fernchen
Copy link
Contributor

@fernchen fernchen commented Jul 9, 2020

Recently,we try to introduce Caffe frontend to TVM, please to ref to #6000 , but original CI environment dose not support for Caffe. In order to pass CI test, we add Caffe environment including Caffe dependent packages and pycaffe for ssd-version, it comes from https://github.com/weiliu89/caffe

@zhiics @tqchen @FrozenGene
Could you please have a look on this PR? Thanks!

@fernchen
Copy link
Contributor Author

May you help me review this pr? @siju-samuel @Huyuwei @hlu1

@fernchen
Copy link
Contributor Author

Based on disscution above, there are two kinds of ways to support caffe frontend in TVM:

  1. We add caffe frontend based on SSD distribution, then this ci pr is good enough to test;
  2. We add caffe frontend based on our maintaining tvm caffe, then I will support the layers of BVLC distribution firstly.

We are very welcome to participate in the discussion!https://discuss.tvm.ai/t/introduce-new-frontend-for-caffe/6918

@tqchen @siju-samuel @zhiics @Huyuwei @hlu1 @kazum @srkreddy1238

@FrozenGene
Copy link
Member

Gental ping @tqchen @siju-samuel

@siju-samuel
Copy link
Member

  1. We add caffe frontend based on SSD distribution, then this ci pr is good enough to test;
  2. We add caffe frontend based on our maintaining tvm caffe, then I will support the layers of BVLC distribution firstly.

I vote for second option. BVLC caffe models will have more users compared to forked versions.
Another concern is lot of dependancies are getting added for running the ci with custom distribution, i think bvlc have support for precompiled caffe installation.

@fernchen
Copy link
Contributor Author

  1. We add caffe frontend based on SSD distribution, then this ci pr is good enough to test;
  2. We add caffe frontend based on our maintaining tvm caffe, then I will support the layers of BVLC distribution firstly.

I vote for second option. BVLC caffe models will have more users compared to forked versions.
Another concern is lot of dependancies are getting added for running the ci with custom distribution, i think bvlc have support for precompiled caffe installation.

Yes, bvlc distribution has supported for precompiled caffe installation, and It's really convenient to use.
If other people agree, we can support the official version first, and then gradually support other distribution. We can even maintain our own Caffe branch to support other layers, of course it is another topic. @tqchen

@fernchen
Copy link
Contributor Author

Hi @FrozenGene @siju-samuel @tqchen :
I have changed the caffe deps into BVLC distribuition, now we support the official caffe version for TVM firstly.

May you please review them? If there is no problem, I hope to merge as soon as possible in order to pass the CI test of caff front-end codes.

Copy link
Member

@tqchen tqchen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some final comments

@@ -83,6 +83,9 @@ RUN bash /install/ubuntu_install_dgl.sh
COPY install/ubuntu_install_vulkan.sh /install/ubuntu_install_vulkan.sh
RUN bash /install/ubuntu_install_vulkan.sh

COPY install/ubuntu_install_caffe.sh /install/ubuntu_install_caffe.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just install the caffe on the CPU and run ci tests through the CPU side? This would reduce our pressure in terms of depending on the GPU resources

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good, if we just install the caffe on the CPU, it will reduce a lot of work.


apt install caffe-cpu -y

pip install --upgrade scikit-image
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need pip3 instead of pip to introduce it to the python3 env.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have updated the codes, please review them, thanks.

@fernchen fernchen requested a review from tqchen July 28, 2020 04:28
@tqchen tqchen merged commit d35a149 into apache:master Jul 28, 2020
@tqchen
Copy link
Member

tqchen commented Jul 28, 2020

Thanks @fernchen This PR is now merged. Note that we still need to update the ci-cpu image, will rely to this thread once that is done.

@tqchen
Copy link
Member

tqchen commented Jul 29, 2020

ci-cpu is now updated to the latest state

trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
* [CI][Caffe Frontend] add caffe environment

* [CI][Caffe Frontend] change the caffe deps into BVLC distribution.

* [CI][Caffe Fronted] simplify configuration while installing tzdata for precompiled caffe.

* [CI][Caffe Frontend] add more information about tzdata.

* [CI][CaffeFrontend]remove the ci for gpu env and change to pip3 env

Co-authored-by: fernchen <[email protected]>
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
* [CI][Caffe Frontend] add caffe environment

* [CI][Caffe Frontend] change the caffe deps into BVLC distribution.

* [CI][Caffe Fronted] simplify configuration while installing tzdata for precompiled caffe.

* [CI][Caffe Frontend] add more information about tzdata.

* [CI][CaffeFrontend]remove the ci for gpu env and change to pip3 env

Co-authored-by: fernchen <[email protected]>
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Aug 26, 2020
* [CI][Caffe Frontend] add caffe environment

* [CI][Caffe Frontend] change the caffe deps into BVLC distribution.

* [CI][Caffe Fronted] simplify configuration while installing tzdata for precompiled caffe.

* [CI][Caffe Frontend] add more information about tzdata.

* [CI][CaffeFrontend]remove the ci for gpu env and change to pip3 env

Co-authored-by: fernchen <[email protected]>
trevor-m pushed a commit to trevor-m/tvm that referenced this pull request Sep 2, 2020
* [CI][Caffe Frontend] add caffe environment

* [CI][Caffe Frontend] change the caffe deps into BVLC distribution.

* [CI][Caffe Fronted] simplify configuration while installing tzdata for precompiled caffe.

* [CI][Caffe Frontend] add more information about tzdata.

* [CI][CaffeFrontend]remove the ci for gpu env and change to pip3 env

Co-authored-by: fernchen <[email protected]>
trevor-m pushed a commit to neo-ai/tvm that referenced this pull request Sep 3, 2020
* [CI][Caffe Frontend] add caffe environment

* [CI][Caffe Frontend] change the caffe deps into BVLC distribution.

* [CI][Caffe Fronted] simplify configuration while installing tzdata for precompiled caffe.

* [CI][Caffe Frontend] add more information about tzdata.

* [CI][CaffeFrontend]remove the ci for gpu env and change to pip3 env

Co-authored-by: fernchen <[email protected]>
@mshr-h
Copy link
Contributor

mshr-h commented Sep 16, 2021

Since BVLC distribution support has successfully completed, it would be great if we could replace BVLC distribution with SSD distribution.

Anyone working on that?
If no, I'll try to do that.

@FrozenGene
Copy link
Member

Since BVLC distribution support has successfully completed, it would be great if we could replace BVLC distribution with SSD distribution.

Anyone working on that?
If no, I'll try to do that.

Please go ahead. And you could tag @fernchen when you pull request.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants