Workshop environment

[ldominguez50]

Hi! Looking at your instructions, I'm guessing we will run exercises and code from our local machines. Will there be any prepared environment for us to use during the workshop? I was asking because I read in the description that training models on GPUs would be part of the course. Thanks!

[rasbt]

That's a good question. I had hoped to set up an environment that would allow participants to run experiments on a GPU, but it turned out to be much more challenging than anticipated. Providing resources reliably for a large number of people at the same time is currently not feasible.

I believe the most reliable way for participants to run code would be on their local machines. Therefore, for the GPU-specific portions of the workshop, I will explain various concepts like data parallelism and tensor sharding. We will implement these and offer small coding exercises, but I will only demonstrate these concepts with actual code. And this will be code that can run on GPUs if you have any (either on the cloud or on your local hardware).

[rasbt]

I also wanted to mention that notebook environments like Google Colab would not be an option here because notebook environments, or interactive environments in general, have restrictions when it comes to multi-GPU computing, so they don't support the advanced multi-GPU paradigms in PyTorch or DeepSpeed. I will go over all of that in the workshop though and provide examples.

[ldominguez50]

Okay. I am a bit confused then about how this workshop ties back to Posit. Would you be able to explain that piece?

…

On Tue, Sep 5, 2023 at 8:33 PM Sebastian Raschka ***@***.***> wrote: I also wanted to mention that notebook environments like Google Colab would not be an option here because notebook environments, or interactive environments in general, have restrictions when it comes to multi-GPU computing, so they don't support the advanced multi-GPU paradigms in PyTorch or DeepSpeed. I will go over all of that in the workshop though and provide examples. — Reply to this email directly, view it on GitHub <#1 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ASDDKXUFDRZ5677WN5IXYXTXY7HGHANCNFSM6AAAAAA4MNX3BM> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[rasbt]

Do you mean how it is related to the Posit platform? I am not otherwise affiliated with Posit, and this workshop is going to be a general introduction to machine learning and deep learning using scikit-learn and PyTorch. I think several conference attendees are curious about using state-of-the-art deep learning methods, which are currently only available in Python (via PyTorch and to some extend Jax and Keras).

Please let me know if you have any additional questions or concerns, and I'd be happy to clarify more.

[rasbt]

I also wanted to emphasize that the focus of this workshop is more on the "how" these methods work rather than "where" to run these.

For the non-neural net machine learning portions, the code will run on a conventional laptop.

The deep learning portion will focus on explaining how the code works, but we will only run a subset of the code because of the time and resource demands. Training vision transformers and large language models usually take many hours to days (in terms of waiting time), so it would not be possible to run it within the context of this workshop. However, besides the multi-GPU paradigms we are going to cover, all the code should run on single GPUs in case you want to replicate the experiments later on.