GTNet

GTNet: Guided Transformer Network for Detecting Human-Object Interactions

A S M Iftekhar, Satish Kumar, R. Austin McEver, Suya You, B.S. Manjunath.

Paper.

GTNet got accepted to Pattern Recognition and Tracking XXXIV at SPIE commerce+ defence Program.

This codebase only contains code for vcoco dataset.

Our Results on V-COCO dataset

Method	mAP (Scenario 1)
VSGNet	51.8
ConsNet	53.2
IDN	53.3
OSGNet	53.4
Sun et al.	55.2
GTNet	58.3

Installation & Setup

1. Clone repository (recursively):

git clone --recursive https://github.com/UCSB-VRL/GTNet.git
cd GTNet

2. Please find the data,annotations,object detection results and embeddings for vcoco here. Download it, unzip and setup the path to directory by running:

python3 setup.py -d <full path to the downloaded folder>

Folder description can be found in our old work

3. Setup enviroment by running (used python 3.6.9):

pip3 install -r requirements.txt

4. Download the best model from here and keep it inside a folder in the repository. We assume that you put it inside soa_vcoco folder in the repository. You can change it to anything you want.

Inference & Training

All commands need to be run from the scripts folder.

To dump results from the best model:

bash run_inference.sh

Be sure to keep the downloaded best model in soa_vcoco folder in the repository, if you put it some other places, change the bash file accordingly. After that, to get the results in the paper run:

bash run_eval_vcoco.sh

To train with 8 GPUS run:

bash run_train.sh

Please check main.py for various flags.

Please contact Iftekhar ([email protected]) for any queries.