Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Separate setup_model_and_dataset into two functions #128

Open
drewoldag opened this issue Dec 4, 2024 · 1 comment
Open

Separate setup_model_and_dataset into two functions #128

drewoldag opened this issue Dec 4, 2024 · 1 comment

Comments

@drewoldag
Copy link
Collaborator

Currently setup_model_and_dataset does exactly what is says, but this is somewhat wasteful in most cases. It would be better to separate this into two distinct functions in pytorch_ignite.py:

def setup_dataset(config, split) - > data_set:
    data_set_cls = fetch_data_set_class(config)
    data_set = data_set_cls(config, split)
    return data_set

def setup_model(config, data_set) -> model:
    model_cls = fetch_model_class(config)
    model = model_cls(config, data_set.shape())
    return model

Then in train.py for instance:

train_data_set = setup_dataset(config, "train")
model = setup_model(config, train_data_set)
validation_data_set = setup_dataset(config, "validate")
...

This saves the unnecessary instantiation of the model in prepare.py and rebuild_manifest.py.
Additionally this makes it easier to instantiate the datasets that we want using the setup_dataset(config, split="blah") syntax.

@mtauraso
Copy link
Collaborator

mtauraso commented Dec 6, 2024

SGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants