Emotional expressivity v1.0

Date completed	April 13, 2023
Release where first appeared	OpenWillis v1.0
Researcher / Developer	Vijay Yadav

---

1 – Use

import openwillis as ow

framewise, summary = ow.emotional_expressivity(filepath = 'video.mov' , baseline_filepath = 'video_baseline.mov')

---

2 – Methods

Measurement of emotional expressivity in the face

We utilize deepface to quantify framewise intensity of the following emotions:

• Happiness
• Sadness
• Anger
• Fear
• Disgust
• Surprise
• Neutral (the absence of any emotion)

We also calculate a composite expressivity score, which averages the expressivity of each of the emotions above (except for neutral).

Framewise values for each variable, ranging from 0-1, are saved in framewise.

In case a baseline input is provided, all values are baseline-corrected and normalized using the same method that is used by the **facial_expressivity **function. The resulting normalized values range between -1 and 1, with negative values signifying expressivity for that emotion being lower than baseline and positive values signifying expressivity greater than baseline.

The framewise expressivity values are compiled for the video and saved in the **summary **output, which contains the primary outcome measures of the function, namely the mean expressivity of each emotion over the course of the video.

---

3 – Inputs

3.1 – filepath

Type	str
Description	path to main video

3.2 – baseline_filepath

Type	str, optional
Description	path to baseline video

---

4 – Outputs

4.1 – framewise

Type	data-type
Description	dataframe with framewise output of facial emotion expressivity. columns are emotional expressivity measures, with the last column being composite emotional expressivity i.e. a mean of all individual emotions except neutral. range for these values is -1 to 1 in case of baselining and 0-1 otherwise. rows represent frames in the video.

What the data frame looks like:

frame	angry	disgust	fear	happiness	sadness	surprise	neutral	composite
0
1
...

4.2 – summary

Type	data-type
Description	dataframe with summary measurements. first column is name of statistic, subsequent columns are facial emotions, last column is composite expressivity i.e. the mean of all emotions except neutral. first row contains mean expressivity and the second row contains standard deviation.

What the data frame looks like:

stat	happiness	sadness	anger	fear	disgust	surprise	neutral	composite
mean
stdev

---

5 – Example

Here, we use the sample data included as part of the repository to calculate emotional expressivity.

import openwillis as ow

framewise, summary = ow.emotional_expressivity(filepath = 'data/subj01.mp4', baseline_filepath = 'data/subj01_base.mp4')

framewise.head(2)

frame	angry	disgust	fear	happiness	sadness	surprise	neutral	composite
0	0.051450	0.000031	0.250485	-0.113090	0638163	-0.000234	-0.446323	0.137801
1	0.006272	-0.000004	0.281984	-0.113109	0.662542	-0.000263	-0.452409	0.139570

6 – Dependencies

Below are dependencies specific to calculation of this measure.

Dependency	License	Justification
deepface	MIT	Free, open-source, MIT Emotion detection No action unit detection No facial landmark detection