Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[ENHANCE] Parametrize saliency maps dumping in export parametrizable #1708

Merged
merged 14 commits into from
Feb 21, 2023

Conversation

GalyaZalesskaya
Copy link
Contributor

@GalyaZalesskaya GalyaZalesskaya commented Feb 15, 2023

Added dump_features parameter in otx export not to calculate and dump saliency maps and representation vectors for each export.

Checked export and inference of pth and IR model for:

  • Classification task (classic classification - MobileNet-V3-large-1x)
  • Detection task (ATSS model)

Expected result: improved efficiency of deployed models if no explanation is needed

TODO

  1. Implement dumping feature according to flag for Segmentation task, Action Detection and Anomaly task
  2. Tests

@github-actions github-actions bot added ALGO Any changes in OTX Algo Tasks implementation CLI Any changes in OTE CLI labels Feb 15, 2023
Copy link
Collaborator

@negvet negvet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor comment from my side

otx/cli/tools/export.py Outdated Show resolved Hide resolved
otx/cli/tools/export.py Show resolved Hide resolved
@negvet
Copy link
Collaborator

negvet commented Feb 16, 2023

Unit tests are required, although can be added in a separate PR

@GalyaZalesskaya GalyaZalesskaya marked this pull request as ready for review February 16, 2023 08:11
@GalyaZalesskaya GalyaZalesskaya requested a review from a team as a code owner February 16, 2023 08:11
@goodsong81 goodsong81 changed the title [OTX] Parametrize saliency maps dumping in export parametrizable [ENHANCE] Parametrize saliency maps dumping in export parametrizable Feb 16, 2023
@goodsong81 goodsong81 added this to the 1.0.0 milestone Feb 17, 2023
@negvet negvet requested a review from wonjuleee February 17, 2023 06:53
@JihwanEom
Copy link
Contributor

Could you also check pre-commit issues?

@github-actions github-actions bot added the API Any changes in OTX API label Feb 19, 2023
@GalyaZalesskaya
Copy link
Contributor Author

@JihwanEom @sovrasov Your comments were reflected
@negvet I'm currently facing issues with running unit-tests, I'll make a separate PR with unit tests for this

negvet
negvet previously approved these changes Feb 20, 2023
Copy link
Collaborator

@negvet negvet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, please check pre-commits

@github-actions github-actions bot added the TEST Any changes in tests label Feb 20, 2023
@codecov-commenter
Copy link

codecov-commenter commented Feb 20, 2023

Codecov Report

Base: 77.80% // Head: 77.76% // Decreases project coverage by -0.05% ⚠️

Coverage data is based on head (ef21b8c) compared to base (e5dd967).
Patch coverage: 29.50% of modified lines in pull request are covered.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #1708      +/-   ##
===========================================
- Coverage    77.80%   77.76%   -0.05%     
===========================================
  Files          469      469              
  Lines        32800    32829      +29     
===========================================
+ Hits         25521    25528       +7     
- Misses        7279     7301      +22     
Impacted Files Coverage Δ
otx/algorithms/common/tasks/training_base.py 50.72% <0.00%> (-0.75%) ⬇️
...x/mpa/modules/models/classifiers/sam_classifier.py 79.71% <0.00%> (-0.78%) ⬇️
...a/modules/models/detectors/custom_atss_detector.py 61.11% <0.00%> (-1.75%) ⬇️
...dules/models/detectors/custom_maskrcnn_detector.py 36.00% <0.00%> (-0.99%) ⬇️
...s/models/detectors/custom_single_stage_detector.py 27.65% <0.00%> (-0.61%) ⬇️
.../modules/models/detectors/custom_yolox_detector.py 32.92% <0.00%> (-0.83%) ⬇️
otx/algorithms/classification/tasks/openvino.py 86.66% <16.66%> (-1.23%) ⬇️
otx/algorithms/action/tasks/inference.py 88.38% <66.66%> (-0.29%) ⬇️
otx/algorithms/anomaly/tasks/inference.py 82.77% <66.66%> (-0.37%) ⬇️
otx/algorithms/segmentation/tasks/inference.py 88.96% <66.66%> (-0.98%) ⬇️
... and 8 more

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@GalyaZalesskaya
Copy link
Contributor Author

@JihwanEom @sovrasov @negvet Tests were fixed, can you review again please?

Copy link
Contributor

@goodsong81 goodsong81 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@sovrasov sovrasov merged commit 6c323ab into develop Feb 21, 2023
@sovrasov sovrasov deleted the gz/dump_features_export branch February 21, 2023 13:43
@harimkang
Copy link
Contributor

harimkang commented Feb 22, 2023

@GalyaZalesskaya I have a suspicion that instance-segmentation export is not working since this PR was merged.
Can you check it out?
i follow below steps:

otx train Custom_Counting_Instance_Segmentation_MaskRCNN_ResNet50 --train-data-roots tests/assets/car_tree_bug --val-data-roots tests/assets/car_tree_bug
cd otx-workspace-INSTANCE_SEGMENTATION
otx export

i got runtime error
RuntimeError: number of output names provided (5) exceeded number of outputs (3)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ALGO Any changes in OTX Algo Tasks implementation API Any changes in OTX API CLI Any changes in OTE CLI TEST Any changes in tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants