This project is focused on identifying instances of violence in video sequences through the use of deep learning. Drawing inspiration from ViolenceNet, we have employed a customized & adapted version of the Densenet121 model, enhanced with convLSTM cells, to analyze video frames and make predictions about the presence of violence. To capture temporal information, optical flow was utilized as the model's input. Contrary to the approach used in the original paper that inspired us, we implemented frame skipping on the optical flow due to memory and computation constraints. Additionally, we used a minimized version of densenet (i.e., less channels/neurons) due to the same constraints. However, this modification did not have any significant impact on the results and could potentially be viewed as an enhancement to their original work.
Video : Click Here
This project is implemented using Python and the following libraries:
- OpenCV: Used for real-time computer vision to read and manipulate images and videos.
- NumPy: Used for numerical computations in Python.
- PyTorch: An open-source machine learning library used to create and train the neural network.
- Pandas: Used for data manipulation and analysis.
- Torchvision: Used to load the pre-trained Densenet121 model and perform image transformations.
- Matplotlib: Used for creating static, animated, and interactive visualizations in Python.
- Seaborn: A Python data visualization library based on Matplotlib. It provides a high-level interface for drawing attractive and informative statistical graphics.
- tqdm: A fast, extensible progress bar for Python and CLI.
- Scikit-learn: A machine learning library in Python. It features various classification, regression and clustering algorithms.
- ConvLSTM: Used to add ConvLSTM layers to the model.
- Clone this repository:
git clone https://github.com/akvnn/violence-detection - Install the required packages:
pip install -r requirements.txt
To use the violence detection model, run the cells in the app.ipynb Jupyter Notebook File
Note that the first half of the cells is for model training and the second half is for model inference
Please read comments and markdown cells before running a cell.
The dataset used for this project can be found on Kaggle: Hockey Fights
The path of the dataset's directory must be specified through the directory variable.
In our experiments, the adapted Densenet121 model demonstrated promising results in detecting violence in video sequences. The model achieved an accuracy of 86%, a weighted precision of 86%, a weighted recall of 86%, and a weighted F-score of 86%. These results suggest that our approach of using an adapted Densenet121 model with optical flow as input, and implementing frame skipping due to memory and computation constraints, is a viable strategy for violence detection in video sequences.
The performance of the model is further illustrated in the images below, which show the final confusion matrix, a plot of the training accuracy vs testing accuracy over epochs, and a plot of the training loss over epochs, respectively.
This project was developed as the course project for the Computer Vision course at American University of Sharjah by Ahmed, Youssef, Mohammed, Said
This project is under the MIT License.