csci_535_multimodal_probabilistic_learning

USC Class : Multimodal Probabilistic Learning of Human Communication

HWs

• See hw folder

---

1. Annotation aggregation and exploratory analysis. Navigate to my code or see my thoughts and process

   • Inter-rater agreement (IRA) is understanding how much labelers/annotates agree (on certain emotions). In order to understand this agreement, you can compute Cohen’s kappa, Krippendorff’s alpha, etc. We calculated Fleiss’ Kappa.
   • After that, we annotated 20 videos WRT 2 cues (joy and surprise). By this, we stated if a particular video expressed joy (N or Y) or surprise (N or Y). We then performed a Students T-test to understand the significance of our votings.

2. Fusion

3. Bias

Play

• See play folder
• Models and algorithms I come across WRT to this course

Open Source models

• Will only provide models and reasonings for using models. Will not include their code as you can navigate to their page to pull the base code.

• openSMILE GitHub to extract hand-crafted features

  • MFCC (my notes)
  • PLP-CC (my notes)
  • Prosody (Pitch + Loudness) (my notes)

• OpenFace GitHub to extract end-to-end features

  • Facial action unit recognition (my notes)

---

Misc

• Summary of Lecture Notes by Topic
• Collection of Papers, Datasets, Repos, Labs
• Paper Discussions + Q&A