Skip to content

Commit

Permalink
Merge pull request #51 from tsaishien-chen/main
Browse files Browse the repository at this point in the history
Update readme
  • Loading branch information
AliaksandrSiarohin authored Apr 29, 2024
2 parents 0502e88 + dd20e9c commit 433ba6d
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 2 deletions.
2 changes: 2 additions & 0 deletions captioning/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
# 🐼 Panda-70M: Video Captioning

**[Note] To use our captioning code, please make sure you follow [this guideline](https://github.com/lm-sys/FastChat/blob/main/docs/vicuna_weights_version.md#how-to-apply-delta-weights-only-needed-for-weights-v0) and correctly prepare vicuna-7b-v0 weight. Basically, you need to first download the original weights and then apply delta weights. Improper weights preparation will lead to meaningless outputs.**

## Introduction
We propose a video captioning model to generate a caption for a short video clip.
The model includes vision (green) and textual (blue) branches to benefit video captioning by both video and text inputs.
Expand Down
4 changes: 2 additions & 2 deletions dataset_dataloading/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,8 @@ output-folder
- Each data comes with 3 files: `.mp4` (video), `.txt` (caption), `.json` (meta information)
- Meta information includes matching score (confidence score of each video-caption pair), caption, video title / description / categories / subtitles, to name but a few.
- **[Note 1]** The dataset is unshuffled and the clips from a same long video would be stored into a shard. Please manually shuffle them if needed.
- **[Note 2]** The videos are resized into 360 px height. You can change `download_size` in the [config](./video2dataset/video2dataset/configs/panda_70M.yaml) file to get different video resolutions.
- **[Note 3]** The videos are downloaded with audio by default. You can change `download_audio` in the [config](./video2dataset/video2dataset/configs/panda_70M.yaml) file to turn off the audio and increase download speed.
- **[Note 2]** The videos are resized into 360 px height. You can change `download_size` in the [config](./video2dataset/video2dataset/configs/panda70m.yaml) file to get different video resolutions.
- **[Note 3]** The videos are downloaded with audio by default. You can change `download_audio` in the [config](./video2dataset/video2dataset/configs/panda70m.yaml) file to turn off the audio and increase download speed.

## Acknowledgements
The code for data downloading is built upon [video2dataset](https://github.com/iejMac/video2dataset).
Expand Down

0 comments on commit 433ba6d

Please sign in to comment.