wip: dataloader first draft #17
Conversation
|
Could you just change the name of the file to sequence_dataloader.py, so we can differentiate from future possible loaders? |
|
Thanks guys! Two comments:
|
|
I understand you perspective @lucapinello. I would only like to apply this separation-by-class to diffusion code, denoising networks and dataloaders. The reasoning is that the code for these types of files tends to get repetitive and once you have 3-4 modules in the same file it becomes tiring to keep track of which section you're debugged or changed. I am ok with keeping all metrics, utils etc. in a small number of files in their designated directories. |
| self.num_workers = num_workers | ||
|
|
||
|
|
||
| def prepare_data(self): |
There was a problem hiding this comment.
We should probably drop this redundant function definition.
| import pytorch_lightning as pl | ||
| from torch.utils.data import Dataset, DataLoader | ||
|
|
||
| class SequenceDataset(Dataset): |
There was a problem hiding this comment.
Proposition: We could also separate this script into a dataloader.py and a dataset.py.
The dataset.py would contain the SequenceDataset Class and the dataloader.py can have the SequenceDataModule Class (or a BaseDataModule and further subclasses such as SequenceDataModule as suggested by @lucapinello)
|
@mateibejan1 Thanks for the suggestion. Can you help me better understand an example file naming breakdown you had in mind in regards to the entire project? My thought process was more inline with Luca's where all loaders can be kept in a single dataloader.py file, and they can then be called in a main script using something like "from dataloader import SequenceDataModule". This is in line with some other implementations using medical image data that I have seen. I'm also available on discord @ssenan |
|
There is currently a discussion on folder structure of the code base in #19 , referencing it here so that you guys can check it and add your ideas |
No description provided.