Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

"IndexError: list index out of range" on multiple GPU #421

Open
bruceisme opened this issue Oct 10, 2023 · 3 comments
Open

"IndexError: list index out of range" on multiple GPU #421

bruceisme opened this issue Oct 10, 2023 · 3 comments

Comments

@bruceisme
Copy link

Hi, when I try to run evaluate_vqa_beam.sh to generate test_dev's answers on multiple GPU, there always occurs error.

File "OFA/data/mm_data/vqa_gen_dataset.py", line 149, in getitem
item = self.dataset[index]
File "OFA/data/file_dataset.py", line 106, in getitem
column_l = [dtype(column_l[col_id]) for col_id, dtype in zip(self.selected_col_ids, self.dtypes)]
File "OFA/data/file_dataset.py", line 106, in
column_l = [dtype(column_l[col_id]) for col_id, dtype in zip(self.selected_col_ids, self.dtypes)]
IndexError: list index out of range

But when I run single GPUS_PER_NODE=1, the problem does not occur and could generate answers successfully.

@bruceisme
Copy link
Author

By the way, the vqa_val.tsv which download from #68 could run evaluate_vqa_beam.sh on multiple GPU, but my fine only could run on single GPU.

@vanloc1808
Copy link

Hi, I still met IndexError while fine-tuning at this line

@wzk913ysq
Copy link

Have you solved this issue ?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants