Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Enhancement] Refactor DataPreprocessor #1537

Conversation

LeoXing1996
Copy link
Collaborator

Thanks for your contribution and we appreciate it a lot. The following instructions would make your pull request more healthy and more easily get feedback. If you do not understand some items, don't worry, just make the pull request and seek help from maintainers.

Motivation

Generative models (e.g., BigGAN, StyleGAN) and low-level models (e.g., BasicVSR) use different data preprocessors, and those data preprocessors take different kinds of input (e.g., GenDataPreprocessor support tensor, list of tensor, and dict as input, while EditDataPreprocessor only support list of tensor as input).

In this PR, we unify GenDatapreprocessor and EditDatapreprocessor.

Modification

In this PR, EditDatapreprocessor is refactored.

  1. Support List[Tensor], Tensor, Dict, List[Dict] as input.
  2. Decraped pad_args and change to pad_mode and pad_value
  3. Support calculate input_view and output_view automatically
  4. Support users control whether norm data_sample.gt_img in val/test stage via only_norm_gt_in_training
  5. destructor support un-pad image by key.
  6. If a list of tensors or a tensor is passed, the returned inputs is packed into a dict with the field img.

Who can help? @ them here!

BC-breaking (Optional)

Does the modification introduce changes that break the backward-compatibility of the downstream repositories?
If so, please describe how it breaks the compatibility and how the downstream projects should modify their code to keep compatibility with this PR.

Use cases (Optional)

If this PR introduces a new feature, it is better to list some use cases here, and update the documentation.

Checklist

Before PR:

  • I have read and followed the workflow indicated in the CONTRIBUTING.md to create this PR.
  • Pre-commit or linting tools indicated in CONTRIBUTING.md are used to fix the potential lint issues.
  • Bug fixes are covered by unit tests, the case that causes the bug should be added in the unit tests.
  • New functionalities are covered by complete unit tests. If not, please add more unit test to ensure the correctness.
  • The documentation has been modified accordingly, including docstring or example tutorials.

After PR:

  • If the modification has potential influence on downstream or other related projects, this PR should be tested with some of those projects.
  • CLA has been signed and all committers have signed the CLA in this PR.

@LeoXing1996 LeoXing1996 added the kind/enhancement refactoring, improving CI, template, etc. label Dec 15, 2022
@LeoXing1996 LeoXing1996 self-assigned this Dec 15, 2022
@LeoXing1996 LeoXing1996 force-pushed the leoxing/refactor-preprocessor branch from 19be629 to 57327a4 Compare December 15, 2022 07:53
@LeoXing1996 LeoXing1996 force-pushed the leoxing/refactor-preprocessor branch from ac7c142 to 2d32bfc Compare December 15, 2022 13:13
@codecov
Copy link

codecov bot commented Dec 15, 2022

Codecov Report

❗ No coverage uploaded for pull request base (refactor-metric-loop-and-preprocessor@b219a43). Click here to learn what that means.
Patch has no changes to coverable lines.

Additional details and impacted files
@@                           Coverage Diff                            @@
##             refactor-metric-loop-and-preprocessor    #1537   +/-   ##
========================================================================
  Coverage                                         ?   88.93%           
========================================================================
  Files                                            ?      400           
  Lines                                            ?    25954           
  Branches                                         ?     4079           
========================================================================
  Hits                                             ?    23081           
  Misses                                           ?     2004           
  Partials                                         ?      869           
Flag Coverage Δ
unittests 88.93% <0.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@LeoXing1996 LeoXing1996 force-pushed the refactor-metric-loop-and-preprocessor branch 2 times, most recently from ae4965c to b2fc76d Compare January 13, 2023 03:04
@LeoXing1996 LeoXing1996 force-pushed the leoxing/refactor-preprocessor branch from 2d32bfc to 3b93e4d Compare January 19, 2023 06:14
@LeoXing1996 LeoXing1996 force-pushed the leoxing/refactor-preprocessor branch from 7eaf5b7 to 6828c76 Compare February 7, 2023 07:49
@LeoXing1996 LeoXing1996 force-pushed the refactor-metric-loop-and-preprocessor branch from a071059 to b219a43 Compare February 7, 2023 07:50
@LeoXing1996 LeoXing1996 force-pushed the leoxing/refactor-preprocessor branch from 6828c76 to fcf6f61 Compare February 7, 2023 07:51
@LeoXing1996 LeoXing1996 merged commit 1867732 into open-mmlab:refactor-metric-loop-and-preprocessor Feb 8, 2023
LeoXing1996 added a commit to LeoXing1996/mmediting that referenced this pull request Feb 17, 2023
* unify GenDataPreprocessor and EditDataPreprocessor

* revise models with EditDataPreprocessor

* adopt configs with revised EditDataPreprocessor

* adopt train_step and unit tests to revised EditDataPreprocessor

* fix unit test of inpainter inferencer

* add more metainfo to EditDataSample

* refactor EditDataPreprocessor to standardize the output channel order and value range of data_samples and inputs

* adopt SR models to refactored EditDataPreprocessor

* revise padding checking in destruct

* revise color order conversion to support get input channel order from metafile and handle single channel images

* remove norm_data_samples_in_testing and use upper case for channel order

* adopt baseGAN and baseCondGAN for new data preprocessor

* convert is_list_of to is_seq_of

* refactor inpaintors to adopt new data preprocessor

* standardize channel order representation and refactor channel order conversion behaviors

* refactor channel conversion in destruct and remove unused codes

* update unit test for edit data preprocessor and fix some bugs

* adopt inpainting models' with data preprocessor

* adopt frame interpolation models' with data preprocessor

* revise train_step and unit tests of global_local and partial_conv

* adopt colorization models' with data preprocessor

* revise train_step and unit tests of aot-gan

* adopt SR and VSR models with data preprocessor

* revise test config of LIIF

* adopt dcgan and SWD, MS-SSIM to new data preprocessor

* adopt wgan, ggan and lsgan to new data preprocessor

* adopt pggan and styleganv1 to new data preprocessor

* adopt pix2pix to new data preprocessor

* adopt cyclegan to new data preprocessor

* adopt conditional-datasets to new data preprocessor

* adopt sngan and sagan to new data preprocessor

* adopt biggan to new data preprocessor

* adopt stylegan2, singan and mspie to new data preprocessor

* adopt stylegan3 to new data preprocessor

* adopt eg3d to new data preprocessor

* revise transform pipelines

* revise unit test of EditDataSample

* adopt baseEditModel, vis_hook and visualizer for new data preprocessor

* some minor fix of edit data preprocessor and add unit test for model utils

* adopt metrics for new data preprocessor

* add env var for deterministic training

* refactor mattor data preprocessor based on edit data preprocessor

* fix lint error

* fix some unit tests

* update unit test of inpainting inferencer, sad and data preprocessor

* optimize the memory cost for AOT-GAN and BasicVSR's unit tests

* optimize the memory cost for IconVSR's unit tests

* skip ttsr unit test on windows CI
LeoXing1996 added a commit that referenced this pull request Feb 21, 2023
* unify GenDataPreprocessor and EditDataPreprocessor

* revise models with EditDataPreprocessor

* adopt configs with revised EditDataPreprocessor

* adopt train_step and unit tests to revised EditDataPreprocessor

* fix unit test of inpainter inferencer

* add more metainfo to EditDataSample

* refactor EditDataPreprocessor to standardize the output channel order and value range of data_samples and inputs

* adopt SR models to refactored EditDataPreprocessor

* revise padding checking in destruct

* revise color order conversion to support get input channel order from metafile and handle single channel images

* remove norm_data_samples_in_testing and use upper case for channel order

* adopt baseGAN and baseCondGAN for new data preprocessor

* convert is_list_of to is_seq_of

* refactor inpaintors to adopt new data preprocessor

* standardize channel order representation and refactor channel order conversion behaviors

* refactor channel conversion in destruct and remove unused codes

* update unit test for edit data preprocessor and fix some bugs

* adopt inpainting models' with data preprocessor

* adopt frame interpolation models' with data preprocessor

* revise train_step and unit tests of global_local and partial_conv

* adopt colorization models' with data preprocessor

* revise train_step and unit tests of aot-gan

* adopt SR and VSR models with data preprocessor

* revise test config of LIIF

* adopt dcgan and SWD, MS-SSIM to new data preprocessor

* adopt wgan, ggan and lsgan to new data preprocessor

* adopt pggan and styleganv1 to new data preprocessor

* adopt pix2pix to new data preprocessor

* adopt cyclegan to new data preprocessor

* adopt conditional-datasets to new data preprocessor

* adopt sngan and sagan to new data preprocessor

* adopt biggan to new data preprocessor

* adopt stylegan2, singan and mspie to new data preprocessor

* adopt stylegan3 to new data preprocessor

* adopt eg3d to new data preprocessor

* revise transform pipelines

* revise unit test of EditDataSample

* adopt baseEditModel, vis_hook and visualizer for new data preprocessor

* some minor fix of edit data preprocessor and add unit test for model utils

* adopt metrics for new data preprocessor

* add env var for deterministic training

* refactor mattor data preprocessor based on edit data preprocessor

* fix lint error

* fix some unit tests

* update unit test of inpainting inferencer, sad and data preprocessor

* optimize the memory cost for AOT-GAN and BasicVSR's unit tests

* optimize the memory cost for IconVSR's unit tests

* skip ttsr unit test on windows CI
LeoXing1996 added a commit that referenced this pull request Feb 23, 2023
* unify GenDataPreprocessor and EditDataPreprocessor

* revise models with EditDataPreprocessor

* adopt configs with revised EditDataPreprocessor

* adopt train_step and unit tests to revised EditDataPreprocessor

* fix unit test of inpainter inferencer

* add more metainfo to EditDataSample

* refactor EditDataPreprocessor to standardize the output channel order and value range of data_samples and inputs

* adopt SR models to refactored EditDataPreprocessor

* revise padding checking in destruct

* revise color order conversion to support get input channel order from metafile and handle single channel images

* remove norm_data_samples_in_testing and use upper case for channel order

* adopt baseGAN and baseCondGAN for new data preprocessor

* convert is_list_of to is_seq_of

* refactor inpaintors to adopt new data preprocessor

* standardize channel order representation and refactor channel order conversion behaviors

* refactor channel conversion in destruct and remove unused codes

* update unit test for edit data preprocessor and fix some bugs

* adopt inpainting models' with data preprocessor

* adopt frame interpolation models' with data preprocessor

* revise train_step and unit tests of global_local and partial_conv

* adopt colorization models' with data preprocessor

* revise train_step and unit tests of aot-gan

* adopt SR and VSR models with data preprocessor

* revise test config of LIIF

* adopt dcgan and SWD, MS-SSIM to new data preprocessor

* adopt wgan, ggan and lsgan to new data preprocessor

* adopt pggan and styleganv1 to new data preprocessor

* adopt pix2pix to new data preprocessor

* adopt cyclegan to new data preprocessor

* adopt conditional-datasets to new data preprocessor

* adopt sngan and sagan to new data preprocessor

* adopt biggan to new data preprocessor

* adopt stylegan2, singan and mspie to new data preprocessor

* adopt stylegan3 to new data preprocessor

* adopt eg3d to new data preprocessor

* revise transform pipelines

* revise unit test of EditDataSample

* adopt baseEditModel, vis_hook and visualizer for new data preprocessor

* some minor fix of edit data preprocessor and add unit test for model utils

* adopt metrics for new data preprocessor

* add env var for deterministic training

* refactor mattor data preprocessor based on edit data preprocessor

* fix lint error

* fix some unit tests

* update unit test of inpainting inferencer, sad and data preprocessor

* optimize the memory cost for AOT-GAN and BasicVSR's unit tests

* optimize the memory cost for IconVSR's unit tests

* skip ttsr unit test on windows CI
LeoXing1996 added a commit that referenced this pull request Feb 28, 2023
* unify GenDataPreprocessor and EditDataPreprocessor

* revise models with EditDataPreprocessor

* adopt configs with revised EditDataPreprocessor

* adopt train_step and unit tests to revised EditDataPreprocessor

* fix unit test of inpainter inferencer

* add more metainfo to EditDataSample

* refactor EditDataPreprocessor to standardize the output channel order and value range of data_samples and inputs

* adopt SR models to refactored EditDataPreprocessor

* revise padding checking in destruct

* revise color order conversion to support get input channel order from metafile and handle single channel images

* remove norm_data_samples_in_testing and use upper case for channel order

* adopt baseGAN and baseCondGAN for new data preprocessor

* convert is_list_of to is_seq_of

* refactor inpaintors to adopt new data preprocessor

* standardize channel order representation and refactor channel order conversion behaviors

* refactor channel conversion in destruct and remove unused codes

* update unit test for edit data preprocessor and fix some bugs

* adopt inpainting models' with data preprocessor

* adopt frame interpolation models' with data preprocessor

* revise train_step and unit tests of global_local and partial_conv

* adopt colorization models' with data preprocessor

* revise train_step and unit tests of aot-gan

* adopt SR and VSR models with data preprocessor

* revise test config of LIIF

* adopt dcgan and SWD, MS-SSIM to new data preprocessor

* adopt wgan, ggan and lsgan to new data preprocessor

* adopt pggan and styleganv1 to new data preprocessor

* adopt pix2pix to new data preprocessor

* adopt cyclegan to new data preprocessor

* adopt conditional-datasets to new data preprocessor

* adopt sngan and sagan to new data preprocessor

* adopt biggan to new data preprocessor

* adopt stylegan2, singan and mspie to new data preprocessor

* adopt stylegan3 to new data preprocessor

* adopt eg3d to new data preprocessor

* revise transform pipelines

* revise unit test of EditDataSample

* adopt baseEditModel, vis_hook and visualizer for new data preprocessor

* some minor fix of edit data preprocessor and add unit test for model utils

* adopt metrics for new data preprocessor

* add env var for deterministic training

* refactor mattor data preprocessor based on edit data preprocessor

* fix lint error

* fix some unit tests

* update unit test of inpainting inferencer, sad and data preprocessor

* optimize the memory cost for AOT-GAN and BasicVSR's unit tests

* optimize the memory cost for IconVSR's unit tests

* skip ttsr unit test on windows CI
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/enhancement refactoring, improving CI, template, etc.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants