Prerequisites for the course include basic knowledge of GitHub, Colab and python. It is thus required before the course to go through these slides as well as the two python basics notebooks:

• python_intro_part1.ipynb
  • Quickstart
  • Indentation
  • Comments
  • Variables
  • Conditions and if statements
  • Arrays
  • Strings
  • Loops: while and for
  • Dictionaries
• python_intro_part2.ipynb
  • Functions
  • Classes/Objects
  • Inheritance
  • Modules
  • JSON data format
  • Exception Handling
  • File Handling

• Lecture: ML basic concepts [slides: 1.MLBasics.pdf]

  • What is machine learning
  • Notation
  • Supervised Learning
  • Linear regression
  • Linear classification
  • Gradient Descent
  • Overfitting
  • Performance metrics

• Hands-on: advanced python

  • Intro to Numpy: numpy_intro.ipynb
  • Intro to Pandas: pandas_intro.ipynb
  • Intro to Matplotlib: matplotlib_intro.ipynb

• Lecture: Neural Networks [slides: 2.NNbasicsAndCNN.pdf]

  • Intro to neural networks
  • Training
  • Activation functions
  • Deep Neural Networks
  • Convolutional Neural Networks
  • Batch Normalization

• Hands-on: basic NN with Keras for LHC jet tagging task

  • Introduction to dataset and tasks [slides: 3.LHCJetTaggingIntro.pdf]
  • Dataset exploration: 1.LHCJetDatasetExploration.ipynb
  • MLP implementation with Keras: 2.JetTaggingMLP.ipynb
  • Conv2D implementation with Keras: 3.JetTaggingConv2D.ipynb
  • Conv1D implementation with Keras: 4.JetTaggingConv1D.ipynb

• Lecture: RNN and GNNs [slides: 4.RNNandGNN.pdf]

  • Vanilla RNN, LSTMs and GRUs
  • Message Passing framework for graph data
  • Graph Convolutional Neural Networks

• Hands-on: RNN and GNN implementations for different tasks

  • GRU for LHC jet tagging task: 5.JetTaggingRNN.ipynb
  • Intro to PyTorch: pytorch_intro.ipynb and pytorch_NeuralNetworks.ipynb
  • Intro to PyTorch Geometric (PyG): 6.IntroToPyG.ipynb
  • Node classification with PyG on Cora citation dataset: 7.KCNodeClassificationPyG.ipynb
  • Graph classification with PyG on molecular prediction dataset: 8.TUGraphClassification.ipynb
  • Graph classification with PyG on LHC jet dataset: 9.JetTaggingGCN.ipynb

• Lecture: Attention Mechanism and Transformers [slides: 5.AttentionAndTransformers.pdf]

  • Attension mechanism
  • Graph attention networks
  • Multi-head attention
  • Transformers

• Hands-on:

  • Transformer model for LHC jet tagging with tensorflow: 10.JetTaggingTransformer.ipynb

• Lecture: Unsupervised Learning [slides: 6.UnsupervisedLearning.pdf]

  • Unsupervised learning
  • Autoencoders
  • Generative Models
  • VariationalAutoencoders
  • Generative Adversarial Networks
  • Anomaly detection

• Hands-on:

  • Generate data with vanilla GAN: 11.VanillaGAN_FMNIST.ipynb
  • Generate data with VAE: 12.VAE_FMNIST.ipynb
  • Anomaly detection for LHC jets with AE 13.JetAnomalyDetectionAE.ipynb
  • Anomaly detection for LHC jets with VAE 14.JetAnomalyDetectionVAE.ipynb

• Pattern Recognition and Machine Learning, Bishop (2006)
• Deep Learning, Goodfellow et al. (2016) -- link
• Introduction to machine learning, Murray (2010) -- video lectures
• Stanford ML courses -- link