GSoC application under the NumFOCUS umbrella (2021)

[vfdev-5]

We are happy to annonce that PyTorch-Ignite project will apply two projects for Google Summer of Code 2021 under the NumFOCUS umbrella : https://github.com/numfocus/gsoc#pytorch-ignite

Google Summer of Code is a global program focused on bringing more student developers into open source software development. Students work with an open source organization on a 10 week programming project during their break from school.

This could be a huge opportunity for the project to continue its open-source development and bring motivated people.

In this discussion, we can talk more on the projects we will apply and detail or elaborate its points. Feel free to ask questions here.

Few dates to keep in mind:

• Application deadline: February 19th, 2021
• Organizations Announced on March 9, 2021

Stay tuned.

[sparkingdark]

I would love to know more about the projects and How to start contribute in the pytorch-ignite?I am 3rd year cs student.
I saw the projects the second project making a higher level api interesting one can you guide me @vfdev-5

[sparkingdark]

Thanks for guidance, would check the instructions and for doubts I will let you know.

[sparkingdark]

@vfdev-5 has ignite support for Nvidia GPU, PyTorch GPU version? I am installing it locally
config:
i3 6th gen
8Gb ram
2Gb Nvidia Geforce mx110

[sparkingdark]

And one thing if PyTorch not installed in the system then it raises this error due to a non-root account or sudo privileges...

╭─   debo@pop-os    ~/ignite     master  
╰─ pip3 install -e .
Obtaining file:///home/debo/ignite
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting torch<2,>=1.3
  Downloading torch-1.7.1-cp38-cp38-manylinux1_x86_64.whl (776.8 MB)
     |████████████████████████████████| 776.8 MB 2.0 kB/s 
Collecting numpy
  Downloading numpy-1.20.0-cp38-cp38-manylinux2010_x86_64.whl (15.4 MB)
     |████████████████████████████████| 15.4 MB 92 kB/s 
Collecting typing-extensions
  Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Installing collected packages: numpy, typing-extensions, torch, pytorch-ignite
  Running setup.py develop for pytorch-ignite
    ERROR: Command errored out with exit status 1:
     command: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/debo/ignite/setup.py'"'"'; __file__='"'"'/home/debo/ignite/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix=
         cwd: /home/debo/ignite/
    Complete output (28 lines):
    running develop
    WARNING: The user site-packages directory is disabled.
    error: can't create or remove files in install directory
    
    The following error occurred while trying to add or remove files in the
    installation directory:
    
        [Errno 13] Permission denied: '/usr/local/lib/python3.8/dist-packages/test-easy-install-6785.write-test'
    
    The installation directory you specified (via --install-dir, --prefix, or
    the distutils default setting) was:
    
        /usr/local/lib/python3.8/dist-packages/
    
    Perhaps your account does not have write access to this directory?  If the
    installation directory is a system-owned directory, you may need to sign in
    as the administrator or "root" account.  If you do not have administrative
    access to this machine, you may wish to choose a different installation
    directory, preferably one that is listed in your PYTHONPATH environment
    variable.
    
    For information on other options, you may wish to consult the
    documentation at:
    
      https://setuptools.readthedocs.io/en/latest/easy_install.html
    
    Please make the appropriate changes for your system and try again.
    
    ----------------------------------------
ERROR: Command errored out with exit status 1: /usr/bin/python3 -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/home/debo/ignite/setup.py'"'"'; __file__='"'"'/home/debo/ignite/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' develop --no-deps --user --prefix= Check the logs for full command output.
╭─   debo@pop-os    ~/ignite     master  

So instead of this error if we can raise an exception to solve this error, I think that will be great.
With sudo it is fixed

╰─ sudo pip3 install -e .
[sudo] password for debo: 
Obtaining file:///home/debo/ignite
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting torch<2,>=1.3
  Downloading torch-1.7.1-cp38-cp38-manylinux1_x86_64.whl (776.8 MB)
     |████████████████████████████████| 776.8 MB 2.1 kB/s 
Collecting numpy
  Downloading numpy-1.20.0-cp38-cp38-manylinux2010_x86_64.whl (15.4 MB)
     |████████████████████████████████| 15.4 MB 92 kB/s 
Collecting typing-extensions
  Downloading typing_extensions-3.7.4.3-py3-none-any.whl (22 kB)
Installing collected packages: numpy, typing-extensions, torch, pytorch-ignite
  Running setup.py develop for pytorch-ignite
Successfully installed numpy-1.20.0 pytorch-ignite torch-1.7.1 typing-extensions-3.7.4.3

[sdesrozis]

Hey thank you for your interest! Maybe we could start a new thread to talk about install details?

Btw yes, ignite supports PyTorch GPU version. The issue you faced is related to python install. I suggest you to use virtual envs rather than install packages in your native python.

[sparkingdark]

Thanks I will do that @sdesrozis ,Thanks for clear out but if someone also install from source I think --user flag reuired in pip.

[sparkingdark]

While running the cifar10 example in google collab I got this error, can you tell me why I am getting this error?

2021-02-07 13:50:34,164 ignite.distributed.launcher.Parallel INFO: Initialized distributed launcher with backend: 'xla-tpu'
2021-02-07 13:50:34,165 ignite.distributed.launcher.Parallel INFO: - Parameters to spawn processes: 
	nproc_per_node: 8
	nnodes: 1
	node_rank: 0
	with_trains: True
2021-02-07 13:50:34,166 ignite.distributed.launcher.Parallel INFO: Spawn function '<function training at 0x7f4c104e5f28>' in 8 processes
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-7-45b7651d6263> in <module>()
----> 1 run(backend="xla-tpu", num_workers=12, nproc_per_node=8, with_trains=True)

3 frames
/usr/local/lib/python3.6/dist-packages/ignite/distributed/comp_models/xla.py in spawn(fn, args, kwargs_dict, nproc_per_node, nnodes, node_rank, backend, **kwargs)
    126                 args=(backend, fn, args, kwargs_dict),
    127                 nprocs=nproc_per_node,
--> 128                 **kwargs,
    129             )
    130 

TypeError: spawn() got an unexpected keyword argument 'with_trains'

link to the cell of the collab

One thing that can be improved more in here the all pip install can be written in one line instead of multiple lines or adding a requirements.txt would work in cifar10 example I think that will be great.

[sparkingdark]

@vfdev-5 is there any high-level design available for ignite like under the hood how it evaluates the performance of a model?

[vfdev-5]

@sparkingdark please take a look at Concepts how it works with ignite. In Ignite we tend to avoid under the hood magic things and let user configure most of the details.

[sparkingdark]

Thanks will take a look and yeah it's a nice concept to give users much control.

[sparkingdark]

@vfdev-5 I sent you a friend request in discord profile name debomaster#2219.

[sparkingdark]

Okay so ignite relies mainly on engine which abstracts the loop ,now I see some events and handlers, are these use to create some specific events or tasks regarding model training and testing @vfdev-5

[sparkingdark]

@sdesrozis I installed ignite with pytorch cpu locally but for gpu what to ,I trying to utilise the google colab if possible to take less load on my machine.

[sparkingdark]

I just copied the project in my notepad for better understanding and will keep adding new resources/progress/idea/understanding in this note .

So I can get continuous feedback from @vfdev-5 @sdesrozis

Development of a Higher-level API - GSoc project 2021

[sparkingdark]

@vfdev-5 is there any slack or gitter channel available dedicated to PyTorch ignite so I can talk to there if required?

[vfdev-5]

Please, check this info : https://github.com/pytorch/ignite#communication .
I'd suggest to open new disscussion : https://github.com/pytorch/ignite/discussions/new if you'd like to ask question etc...

[sparkingdark]

Okay, thanks for helping Will do it.

[KickItLikeShika]

Hi @vfdev-5 We chatted on Twitter days ago and i told you i'm gonna have a look, i did walk through the code base to have some general idea what's going, and i have already started planning and experimenting some code from here #912. I'll keep experimenting some stuff and put a timeline for the GSoC proposal. Please tell me if i miss something here.

If more than one person is gonna work on this project i guess it's gonna be great to share some thoughts me and @sparkingdark because is see he is interested too, if not then i guess it's gonna be a great challenge between me and him to make the best work/proposal ;)

[sparkingdark]

Hey gsoc all about collaboration and learning together,don't take it as challenge between you and me,rather let's create a high level ideation and get the everything out of it.

[KickItLikeShika]

Sure, Good Luck

[sparkingdark]

yup let's start the work.

[vfdev-5]

Today, Google announced selected projects and great news that NumFOCUS has been selected for Google Summer of Code 2021 again!

https://summerofcode.withgoogle.com/organizations/5765643267211264/

More details will follow in the coming weeks. Stay tuned!

[sparkingdark]

Great news😁😁 looking forward to do more contribute. For some reasons I am busy for these days but will start working again.

…

On Wed, Mar 10, 2021, 02:35 vfdev ***@***.***> wrote: Today, Google announced selected projects and great news that NumFOCUS has been selected for Google Summer of Code 2021 again! More details will follow in the coming weeks. Stay tuned! — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#1603 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AL7XHCAKCLH67U6QDDQWXBDTC2ESLANCNFSM4W5P6GEQ> .

[vfdev-5]

Few suggestions for students thinking to apply to our project:

• Applying two proposals for our 2 project ideas is OK.

• Make sure to have read this "getting started".

• Proactive applicants (who take care of the project outside specific tasks) could have more chances to be selected.

  • for example, helping with reviewing PRs (if you feel comfortable and can bring important observations).

• Feel free to get in touch with us here or on Discord (for example, vfdev-5#4458) or create a new Github DIscussions.

• Nice to have: if you have a side hobby project using Ignite on your Github, do not hesiate to mention it.

[baheytharwat]

Hi @vfdev-5,
I want to express my interest in working on "Development of a Higher-level API". I read the "getting started" requirements, had a look through the attached issue and had a sense of what should be done.

I've made a simple notebook to try the use of a Higher-level API. Please, review the notebook and let me know your thoughts. and what do you think the next steps should be?

[vfdev-5]

@baheytharwat thanks for you interest in this project!

I checked the notebook, but it is to simple for a higher level API.

Please, read "The plan for this project" for the next steps we think of. Additionally, you can take a look at the discussion here: #1641

[vfdev-5]

Application period is approaching : March 29 - April 13

• Time to start crafting project proposals
  • It can be optionally (and not required at all) based on this template : https://www.overleaf.com/read/frtjdkwhcsky (thanks to @KickItLikeShika for the idea and Red Hen Lab for sharing their template 👍).
• Optionally reach out to us and ask questions about your ideas list and receive some feedback

Meanwhile, take a look to these GSoC videos:

• https://youtu.be/YN7uGCg5vLg
• https://www.youtube.com/watch?v=3J_eBuYxcyg