Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add check before loading metrics data from checkpoint #2323

Merged
merged 2 commits into from
Sep 26, 2024

Conversation

blaz-r
Copy link
Contributor

@blaz-r blaz-r commented Sep 25, 2024

📝 Description

✨ Changes

Select what type of change your PR is:

  • 🐞 Bug fix (non-breaking change which fixes an issue)
  • 🔨 Refactor (non-breaking change which refactors the code base)
  • 🚀 New feature (non-breaking change which adds functionality)
  • 💥 Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • 📚 Documentation update
  • 🔒 Security update

✅ Checklist

Before you submit your pull request, please make sure you have completed the following steps:

  • 📋 I have summarized my changes in the CHANGELOG and followed the guidelines for my type of change (skip for minor changes, documentation updates, and test enhancements).
  • 📚 I have made the necessary updates to the documentation (if applicable).
  • 🧪 I have written tests that support my changes and prove that my fix is effective or my feature works (if applicable).

For more information about code review checklists, see the Code Review Checklist.

Copy link

codecov bot commented Sep 25, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.95%. Comparing base (983ec58) to head (636f399).
Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #2323      +/-   ##
==========================================
+ Coverage   80.93%   80.95%   +0.01%     
==========================================
  Files         247      247              
  Lines       10901    10899       -2     
==========================================
  Hits         8823     8823              
+ Misses       2078     2076       -2     
Flag Coverage Δ
80.95% <100.00%> (+0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@samet-akcay samet-akcay enabled auto-merge (squash) September 26, 2024 07:52
@samet-akcay samet-akcay merged commit f473df8 into openvinotoolkit:main Sep 26, 2024
4 checks passed
samet-akcay added a commit that referenced this pull request Oct 16, 2024
* Update timm requirement from <=1.0.7,>=1.0.7 to >=1.0.7,<=1.0.9 (#2274)

* Update timm requirement from <=1.0.7,>=1.0.7 to >=1.0.7,<=1.0.9

Updates the requirements on [timm](https://github.com/huggingface/pytorch-image-models) to permit the latest version.
- [Release notes](https://github.com/huggingface/pytorch-image-models/releases)
- [Commits](huggingface/pytorch-image-models@v1.0.7...v1.0.9)

---
updated-dependencies:
- dependency-name: timm
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update pyproject.toml

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Samet Akcay <[email protected]>

* 🐞Update `setuptools` requirement for PEP 660 support (#2320)

Update setup tools

Signed-off-by: Samet Akcay <[email protected]>

* Fix transforms for draem, dsr and rkde (#2324)

Signed-off-by: Blaz Rolih <[email protected]>

* Add check before loading metrics data from checkpoint (#2323)

Add check before loading from checkpoint

Signed-off-by: Blaz Rolih <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* Add PIMO (#2329)

* PIMO (#1726)

* update

Signed-off-by: jpcbertoldo <[email protected]>

* test binclf curves numpy and numba and fixes

Signed-off-by: jpcbertoldo <[email protected]>

* correct som docstrings

Signed-off-by: jpcbertoldo <[email protected]>

* torch interface and tests

Signed-off-by: jpcbertoldo <[email protected]>

* torch interface and tests

Signed-off-by: jpcbertoldo <[email protected]>

* constants regrouped in dataclass as class vars

Signed-off-by: jpcbertoldo <[email protected]>

* result class was unneccesary for per_image_binclf_curve

Signed-off-by: jpcbertoldo <[email protected]>

* factorize function _get_threshs_minmax_linspace

Signed-off-by: jpcbertoldo <[email protected]>

* small docs fixes

Signed-off-by: jpcbertoldo <[email protected]>

* add pimo numpy version and test

Signed-off-by: jpcbertoldo <[email protected]>

* move validation

Signed-off-by: jpcbertoldo <[email protected]>

* add `shared_fpr_metric` option

Signed-off-by: jpcbertoldo <[email protected]>

* add pimo torch functional version and test

Signed-off-by: jpcbertoldo <[email protected]>

* add torchmetrics interface and test

Signed-off-by: jpcbertoldo <[email protected]>

* renames and put things in init

Signed-off-by: jpcbertoldo <[email protected]>

* validate inputs in result objects

Signed-off-by: jpcbertoldo <[email protected]>

* result objects to from dict and tests

Signed-off-by: jpcbertoldo <[email protected]>

* add save and load methods to result objects and test

Signed-off-by: jpcbertoldo <[email protected]>

* refactor validations and minor changes

Signed-off-by: jpcbertoldo <[email protected]>

* test result objects' properties

Signed-off-by: jpcbertoldo <[email protected]>

* minor refactors

Signed-off-by: jpcbertoldo <[email protected]>

* add missing docstrings

Signed-off-by: jpcbertoldo <[email protected]>

* minore vocabulary fix for consistency

Signed-off-by: jpcbertoldo <[email protected]>

* add per image scores statistics and test it

Signed-off-by: jpcbertoldo <[email protected]>

* refactor constants notation

Signed-off-by: jpcbertoldo <[email protected]>

* add stats tests and test it

Signed-off-by: jpcbertoldo <[email protected]>

* change the meaning of AUPIMO.num_thresh

Signed-off-by: jpcbertoldo <[email protected]>

* interface to format pairwise test results

Signed-off-by: jpcbertoldo <[email protected]>

* improve doc

Signed-off-by: jpcbertoldo <[email protected]>

* add optional `paths` to result objects and some minor fixes and refactors

Signed-off-by: jpcbertoldo <[email protected]>

* remove frozen from dataclasses and some done todos

Signed-off-by: jpcbertoldo <[email protected]>

* review headers

Signed-off-by: jpcbertoldo <[email protected]>

* doc modifs

Signed-off-by: jpcbertoldo <[email protected]>

* refactor `score_less_than_thresh` in `_binclf_one_curve_python`

Signed-off-by: jpcbertoldo <[email protected]>

* correct license comments

Signed-off-by: jpcbertoldo <[email protected]>

* fix doc

Signed-off-by: jpcbertoldo <[email protected]>

* numba as extra requirement

Signed-off-by: jpcbertoldo <[email protected]>

* refactor copyrights from jpcbertoldo

Signed-off-by: jpcbertoldo <[email protected]>

* remove from __future__ import annotations

Signed-off-by: jpcbertoldo <[email protected]>

* refactor validations names

Signed-off-by: jpcbertoldo <[email protected]>

* dedupe file path validation

Signed-off-by: jpcbertoldo <[email protected]>

* fix tests

Signed-off-by: jpcbertoldo <[email protected]>

* Add todo

Signed-off-by: jpcbertoldo <[email protected]>

* refactor enums

Signed-off-by: jpcbertoldo <[email protected]>

* only logger.warning

Signed-off-by: jpcbertoldo <[email protected]>

* refactor test imports

Signed-off-by: jpcbertoldo <[email protected]>

* refactor docs

Signed-off-by: jpcbertoldo <[email protected]>

* refactor some docs

Signed-off-by: jpcbertoldo <[email protected]>

* correct pre commit errors

Signed-off-by: jpcbertoldo <[email protected]>

* remove author tag

Signed-off-by: jpcbertoldo <[email protected]>

* add thrid party program

Signed-off-by: jpcbertoldo <[email protected]>

* Update src/anomalib/metrics/per_image/pimo.py

* move HAS_NUMBA

Signed-off-by: jpcbertoldo <[email protected]>

* remove PIMOSharedFPRMetric

Signed-off-by: jpcbertoldo <[email protected]>

* make torchmetrics compute avg by dft

Signed-off-by: jpcbertoldo <[email protected]>

* pre-commit hooks corrections

Signed-off-by: jpcbertoldo <[email protected]>

* correct numpy.trapezoid

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* 🗑️ Remove numba (#2313)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* add third-party-programs.txt

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* 🗑️ Remove unused methods (#2315)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* PIMO: Port Numpy → Torch (#2316)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <[email protected]>

* replace numpy with torch

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* 🔨Refactor methods across files (#2321)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <[email protected]>

* replace numpy with torch

Signed-off-by: Ashwin Vaidya <[email protected]>

* refactor code

Signed-off-by: Ashwin Vaidya <[email protected]>

* refactor
move functional inside update
remove path from the metric

* Add changes from comments

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* Remove model to model comparison (#2325)

* rename to pimo

Signed-off-by: Ashwin Vaidya <[email protected]>

* minor refactor

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove model to model comparison

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix test

Signed-off-by: Ashwin Vaidya <[email protected]>

* PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* Minor refactor

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* Remove unused enums

Signed-off-by: Ashwin Vaidya <[email protected]>

* update doc strings

Signed-off-by: Ashwin Vaidya <[email protected]>

* update param names

Signed-off-by: Ashwin Vaidya <[email protected]>

* add aupimo basic usage tutorial notebook (#2330)

* add aupimo basic usage tutorial notebook

Signed-off-by: jpcbertoldo <[email protected]>

* update scipy import

Signed-off-by: jpcbertoldo <[email protected]>

* add cite us

Signed-off-by: jpcbertoldo <[email protected]>

* minor

Signed-off-by: jpcbertoldo <[email protected]>

* modify texts and add illustration

Signed-off-by: jpcbertoldo <[email protected]>

* udpate working dir

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>
Signed-off-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Joao P C Bertoldo <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* Makes batch size dynamic (#2339)

Made batch dimension of ONNX export dynamic when specifying input shape.

* Add pimo tutorial advanced i (fixed) (#2336)

* uset all padim features to make it deterministic

Signed-off-by: jpcbertoldo <[email protected]>

* add aupimo notebook advanced i

Signed-off-by: jpcbertoldo <[email protected]>

* update readme

Signed-off-by: jpcbertoldo <[email protected]>

* modify changelog

Signed-off-by: jpcbertoldo <[email protected]>

* correct readme

Signed-off-by: jpcbertoldo <[email protected]>

* correct again

Signed-off-by: jpcbertoldo <[email protected]>

* minor corrections

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>

* Pimo tutorials/02 advanced ii (#2347)

* uset all padim features to make it deterministic

Signed-off-by: jpcbertoldo <[email protected]>

* add aupimo notebook advanced i

Signed-off-by: jpcbertoldo <[email protected]>

* update readme

Signed-off-by: jpcbertoldo <[email protected]>

* modify changelog

Signed-off-by: jpcbertoldo <[email protected]>

* correct readme

Signed-off-by: jpcbertoldo <[email protected]>

* correct again

Signed-off-by: jpcbertoldo <[email protected]>

* minor corrections

Signed-off-by: jpcbertoldo <[email protected]>

* add aupimo notebook advanced ii (pimo curve and integration bounds)

Signed-off-by: jpcbertoldo <[email protected]>

* fix links

Signed-off-by: jpcbertoldo <[email protected]>

* correct change log

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>

* Create epic.yaml

* 🔨 Update the issue templates (#2363)

* Update epic.yaml

* Update epic.yaml

* Update epic.yaml

* Update epic.yaml

* Update task.yaml

* Create user_story.yaml

* Update epic.yaml

* Pimo tutorials/03 advanced iii (#2348)

* add aupimo notebook advanced iii (aupimo score of a random model)

Signed-off-by: jpcbertoldo <[email protected]>

* add cite us

Signed-off-by: jpcbertoldo <[email protected]>

* update notebooks readme

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* 🔨 Deprecate try import and replace it with Lightning's package_available (#2373)

Replace try_import with lightnings package_available function

Signed-off-by: Samet Akcay <[email protected]>

* Refactor folder3d to avoid complex-structure (C901) issue (#2185)

* Refactored-make_folder3d_dataset-ruff-error-C901 (#1926)

Signed-off-by: sahusiddharth <[email protected]>

* Simplify folder 3d dataset (#2184)

---------

Signed-off-by: sahusiddharth <[email protected]>
Co-authored-by: Siddharth Sahu <[email protected]>

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Samet Akcay <[email protected]>
Signed-off-by: Blaz Rolih <[email protected]>
Signed-off-by: jpcbertoldo <[email protected]>
Signed-off-by: Ashwin Vaidya <[email protected]>
Signed-off-by: sahusiddharth <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Blaž Rolih <[email protected]>
Co-authored-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Joao P C Bertoldo <[email protected]>
Co-authored-by: Marcus Pertlwieser <[email protected]>
Co-authored-by: Siddharth Sahu <[email protected]>
samet-akcay added a commit that referenced this pull request Nov 15, 2024
* Update timm requirement from <=1.0.7,>=1.0.7 to >=1.0.7,<=1.0.9 (#2274)

* Update timm requirement from <=1.0.7,>=1.0.7 to >=1.0.7,<=1.0.9

Updates the requirements on [timm](https://github.com/huggingface/pytorch-image-models) to permit the latest version.
- [Release notes](https://github.com/huggingface/pytorch-image-models/releases)
- [Commits](huggingface/pytorch-image-models@v1.0.7...v1.0.9)

---
updated-dependencies:
- dependency-name: timm
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>

* Update pyproject.toml

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Samet Akcay <[email protected]>

* 🐞Update `setuptools` requirement for PEP 660 support (#2320)

Update setup tools

Signed-off-by: Samet Akcay <[email protected]>

* Fix transforms for draem, dsr and rkde (#2324)

Signed-off-by: Blaz Rolih <[email protected]>

* Add check before loading metrics data from checkpoint (#2323)

Add check before loading from checkpoint

Signed-off-by: Blaz Rolih <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* Add PIMO (#2329)

* PIMO (#1726)

* update

Signed-off-by: jpcbertoldo <[email protected]>

* test binclf curves numpy and numba and fixes

Signed-off-by: jpcbertoldo <[email protected]>

* correct som docstrings

Signed-off-by: jpcbertoldo <[email protected]>

* torch interface and tests

Signed-off-by: jpcbertoldo <[email protected]>

* torch interface and tests

Signed-off-by: jpcbertoldo <[email protected]>

* constants regrouped in dataclass as class vars

Signed-off-by: jpcbertoldo <[email protected]>

* result class was unneccesary for per_image_binclf_curve

Signed-off-by: jpcbertoldo <[email protected]>

* factorize function _get_threshs_minmax_linspace

Signed-off-by: jpcbertoldo <[email protected]>

* small docs fixes

Signed-off-by: jpcbertoldo <[email protected]>

* add pimo numpy version and test

Signed-off-by: jpcbertoldo <[email protected]>

* move validation

Signed-off-by: jpcbertoldo <[email protected]>

* add `shared_fpr_metric` option

Signed-off-by: jpcbertoldo <[email protected]>

* add pimo torch functional version and test

Signed-off-by: jpcbertoldo <[email protected]>

* add torchmetrics interface and test

Signed-off-by: jpcbertoldo <[email protected]>

* renames and put things in init

Signed-off-by: jpcbertoldo <[email protected]>

* validate inputs in result objects

Signed-off-by: jpcbertoldo <[email protected]>

* result objects to from dict and tests

Signed-off-by: jpcbertoldo <[email protected]>

* add save and load methods to result objects and test

Signed-off-by: jpcbertoldo <[email protected]>

* refactor validations and minor changes

Signed-off-by: jpcbertoldo <[email protected]>

* test result objects' properties

Signed-off-by: jpcbertoldo <[email protected]>

* minor refactors

Signed-off-by: jpcbertoldo <[email protected]>

* add missing docstrings

Signed-off-by: jpcbertoldo <[email protected]>

* minore vocabulary fix for consistency

Signed-off-by: jpcbertoldo <[email protected]>

* add per image scores statistics and test it

Signed-off-by: jpcbertoldo <[email protected]>

* refactor constants notation

Signed-off-by: jpcbertoldo <[email protected]>

* add stats tests and test it

Signed-off-by: jpcbertoldo <[email protected]>

* change the meaning of AUPIMO.num_thresh

Signed-off-by: jpcbertoldo <[email protected]>

* interface to format pairwise test results

Signed-off-by: jpcbertoldo <[email protected]>

* improve doc

Signed-off-by: jpcbertoldo <[email protected]>

* add optional `paths` to result objects and some minor fixes and refactors

Signed-off-by: jpcbertoldo <[email protected]>

* remove frozen from dataclasses and some done todos

Signed-off-by: jpcbertoldo <[email protected]>

* review headers

Signed-off-by: jpcbertoldo <[email protected]>

* doc modifs

Signed-off-by: jpcbertoldo <[email protected]>

* refactor `score_less_than_thresh` in `_binclf_one_curve_python`

Signed-off-by: jpcbertoldo <[email protected]>

* correct license comments

Signed-off-by: jpcbertoldo <[email protected]>

* fix doc

Signed-off-by: jpcbertoldo <[email protected]>

* numba as extra requirement

Signed-off-by: jpcbertoldo <[email protected]>

* refactor copyrights from jpcbertoldo

Signed-off-by: jpcbertoldo <[email protected]>

* remove from __future__ import annotations

Signed-off-by: jpcbertoldo <[email protected]>

* refactor validations names

Signed-off-by: jpcbertoldo <[email protected]>

* dedupe file path validation

Signed-off-by: jpcbertoldo <[email protected]>

* fix tests

Signed-off-by: jpcbertoldo <[email protected]>

* Add todo

Signed-off-by: jpcbertoldo <[email protected]>

* refactor enums

Signed-off-by: jpcbertoldo <[email protected]>

* only logger.warning

Signed-off-by: jpcbertoldo <[email protected]>

* refactor test imports

Signed-off-by: jpcbertoldo <[email protected]>

* refactor docs

Signed-off-by: jpcbertoldo <[email protected]>

* refactor some docs

Signed-off-by: jpcbertoldo <[email protected]>

* correct pre commit errors

Signed-off-by: jpcbertoldo <[email protected]>

* remove author tag

Signed-off-by: jpcbertoldo <[email protected]>

* add thrid party program

Signed-off-by: jpcbertoldo <[email protected]>

* Update src/anomalib/metrics/per_image/pimo.py

* move HAS_NUMBA

Signed-off-by: jpcbertoldo <[email protected]>

* remove PIMOSharedFPRMetric

Signed-off-by: jpcbertoldo <[email protected]>

* make torchmetrics compute avg by dft

Signed-off-by: jpcbertoldo <[email protected]>

* pre-commit hooks corrections

Signed-off-by: jpcbertoldo <[email protected]>

* correct numpy.trapezoid

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* 🗑️ Remove numba (#2313)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* add third-party-programs.txt

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* 🗑️ Remove unused methods (#2315)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* PIMO: Port Numpy → Torch (#2316)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <[email protected]>

* replace numpy with torch

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* 🔨Refactor methods across files (#2321)

* remove numba

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix pre-commit checks

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove all unused methods

Signed-off-by: Ashwin Vaidya <[email protected]>

* replace numpy with torch

Signed-off-by: Ashwin Vaidya <[email protected]>

* refactor code

Signed-off-by: Ashwin Vaidya <[email protected]>

* refactor
move functional inside update
remove path from the metric

* Add changes from comments

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* Remove model to model comparison (#2325)

* rename to pimo

Signed-off-by: Ashwin Vaidya <[email protected]>

* minor refactor

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove model to model comparison

Signed-off-by: Ashwin Vaidya <[email protected]>

* fix test

Signed-off-by: Ashwin Vaidya <[email protected]>

* PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* Minor refactor

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* Remove unused enums

Signed-off-by: Ashwin Vaidya <[email protected]>

* update doc strings

Signed-off-by: Ashwin Vaidya <[email protected]>

* update param names

Signed-off-by: Ashwin Vaidya <[email protected]>

* add aupimo basic usage tutorial notebook (#2330)

* add aupimo basic usage tutorial notebook

Signed-off-by: jpcbertoldo <[email protected]>

* update scipy import

Signed-off-by: jpcbertoldo <[email protected]>

* add cite us

Signed-off-by: jpcbertoldo <[email protected]>

* minor

Signed-off-by: jpcbertoldo <[email protected]>

* modify texts and add illustration

Signed-off-by: jpcbertoldo <[email protected]>

* udpate working dir

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>
Signed-off-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Joao P C Bertoldo <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* Makes batch size dynamic (#2339)

Made batch dimension of ONNX export dynamic when specifying input shape.

* Add pimo tutorial advanced i (fixed) (#2336)

* uset all padim features to make it deterministic

Signed-off-by: jpcbertoldo <[email protected]>

* add aupimo notebook advanced i

Signed-off-by: jpcbertoldo <[email protected]>

* update readme

Signed-off-by: jpcbertoldo <[email protected]>

* modify changelog

Signed-off-by: jpcbertoldo <[email protected]>

* correct readme

Signed-off-by: jpcbertoldo <[email protected]>

* correct again

Signed-off-by: jpcbertoldo <[email protected]>

* minor corrections

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>

* Pimo tutorials/02 advanced ii (#2347)

* uset all padim features to make it deterministic

Signed-off-by: jpcbertoldo <[email protected]>

* add aupimo notebook advanced i

Signed-off-by: jpcbertoldo <[email protected]>

* update readme

Signed-off-by: jpcbertoldo <[email protected]>

* modify changelog

Signed-off-by: jpcbertoldo <[email protected]>

* correct readme

Signed-off-by: jpcbertoldo <[email protected]>

* correct again

Signed-off-by: jpcbertoldo <[email protected]>

* minor corrections

Signed-off-by: jpcbertoldo <[email protected]>

* add aupimo notebook advanced ii (pimo curve and integration bounds)

Signed-off-by: jpcbertoldo <[email protected]>

* fix links

Signed-off-by: jpcbertoldo <[email protected]>

* correct change log

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>

* Create epic.yaml

* 🔨 Update the issue templates (#2363)

* Update epic.yaml

* Update epic.yaml

* Update epic.yaml

* Update epic.yaml

* Update task.yaml

* Create user_story.yaml

* Update epic.yaml

* Pimo tutorials/03 advanced iii (#2348)

* add aupimo notebook advanced iii (aupimo score of a random model)

Signed-off-by: jpcbertoldo <[email protected]>

* add cite us

Signed-off-by: jpcbertoldo <[email protected]>

* update notebooks readme

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* 🔨 Deprecate try import and replace it with Lightning's package_available (#2373)

Replace try_import with lightnings package_available function

Signed-off-by: Samet Akcay <[email protected]>

* Refactor folder3d to avoid complex-structure (C901) issue (#2185)

* Refactored-make_folder3d_dataset-ruff-error-C901 (#1926)

Signed-off-by: sahusiddharth <[email protected]>

* Simplify folder 3d dataset (#2184)

---------

Signed-off-by: sahusiddharth <[email protected]>
Co-authored-by: Siddharth Sahu <[email protected]>

* 🚀 Add datumaro annotation dataloader (#2377)

* Add datumaro annotation dataloader

Signed-off-by: Ashwin Vaidya <[email protected]>

* Update changelog

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add examples

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* Pimo tutorials/04 advanced iv  (#2352)

* add notebook 701e_aupimo_advanced_iv on load/save and statistical comparisons

Signed-off-by: jpcbertoldo <[email protected]>

* make `AUPIMOResult.num_thresholds` optional

Signed-off-by: jpcbertoldo <[email protected]>

* add aupimo notebook advanced iv (load/save and statistical tests)

Signed-off-by: jpcbertoldo <[email protected]>

* simplify cite us and mention intal

Signed-off-by: jpcbertoldo <[email protected]>

* fix readme

Signed-off-by: jpcbertoldo <[email protected]>

---------

Signed-off-by: jpcbertoldo <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* 🐞 Defer OpenVINO import to avoid unnecessary warnings (#2385)

* Fix openvino import issue

Signed-off-by: Samet Akcay <[email protected]>

* Fix pre-commit issues

Signed-off-by: Samet Akcay <[email protected]>

---------

Signed-off-by: Samet Akcay <[email protected]>

* 🚀 Add VLM based Anomaly Model (#2344)

* [Draft] Llm on (#2165)

* Add TaskType Explanation

Signed-off-by: Bepitic <[email protected]>

* Add llm model

Signed-off-by: Bepitic <[email protected]>

* add ollama

Signed-off-by: Bepitic <[email protected]>

* better description for descr in title

Signed-off-by: Bepitic <[email protected]>

* add text of llm into imageResult visualization

* add text of llm into imageResult visualization

Signed-off-by: Bepitic <[email protected]>

* latest changes

Signed-off-by: Bepitic <[email protected]>

* add wip llava/llava_next

Signed-off-by: Bepitic <[email protected]>

* add init

Signed-off-by: Bepitic <[email protected]>

* add text of llm into imageResult visualization

Signed-off-by: Bepitic <[email protected]>

* latest changes

Signed-off-by: Bepitic <[email protected]>

* upd Lint

Signed-off-by: Bepitic <[email protected]>

* fix visualization with description

Signed-off-by: Bepitic <[email protected]>

* show the images every batch

Signed-off-by: Bepitic <[email protected]>

* fix docstring and error management

Signed-off-by: Bepitic <[email protected]>

* Add compatibility for TaskType.EXPLANATION.

Signed-off-by: Bepitic <[email protected]>

* Remove, show in the engine-Visualization.

* fix visualization and llm openai multishot.

* fix Circular import problem

* Add HugginFace To LLavaNext

Signed-off-by: Bepitic <[email protected]>

---------

Signed-off-by: Bepitic <[email protected]>

* 🔨 Scaffold for refactor (#2340)

* initial scafold

Signed-off-by: Ashwin Vaidya <[email protected]>

* Apply PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* rename dir

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add ChatGPT (#2341)

* initial scafold

Signed-off-by: Ashwin Vaidya <[email protected]>

* Apply PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* rename dir

Signed-off-by: Ashwin Vaidya <[email protected]>

* delete llm_ollama

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add ChatGPT

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add ChatGPT

Signed-off-by: Ashwin Vaidya <[email protected]>

* Remove LLM model

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add Huggingface (#2343)

* initial scafold

Signed-off-by: Ashwin Vaidya <[email protected]>

* Apply PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* rename dir

Signed-off-by: Ashwin Vaidya <[email protected]>

* delete llm_ollama

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add ChatGPT

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add ChatGPT

Signed-off-by: Ashwin Vaidya <[email protected]>

* Remove LLM model

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add transformers

Signed-off-by: Ashwin Vaidya <[email protected]>

* Remove llava

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* 🔨 Minor Refactor (#2345)

Refactor

Signed-off-by: Ashwin Vaidya <[email protected]>

* undo changes

Signed-off-by: Ashwin Vaidya <[email protected]>

* undo changes

Signed-off-by: Ashwin Vaidya <[email protected]>

* undo changes to image.py

Signed-off-by: Ashwin Vaidya <[email protected]>

* Add explanation visualizer (#2351)

* Add explanation visualizer

Signed-off-by: Ashwin Vaidya <[email protected]>

* bug-fix

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* 🔨 Allow setting API keys from env (#2353)

Allow setting API keys from env

Signed-off-by: Ashwin Vaidya <[email protected]>

* 🧪 Add tests (#2355)

* Add tests

Signed-off-by: Ashwin Vaidya <[email protected]>

* remove explanation task type

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Ashwin Vaidya <[email protected]>

* minor fixes

Signed-off-by: Ashwin Vaidya <[email protected]>

* Update changelog

Signed-off-by: Ashwin Vaidya <[email protected]>

* Fix tests

Signed-off-by: Ashwin Vaidya <[email protected]>

* Address PR comments

Signed-off-by: Ashwin Vaidya <[email protected]>

* update name

Signed-off-by: Ashwin Vaidya <[email protected]>

* Update src/anomalib/models/image/vlm_ad/lightning_model.py

Co-authored-by: Samet Akcay <[email protected]>

* update name

Signed-off-by: Ashwin Vaidya <[email protected]>

---------

Signed-off-by: Bepitic <[email protected]>
Signed-off-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Paco <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* Add ensembling methods for tiling to Anomalib (#1226)

* Fixed broken links in readme

* Fixed inference command in readme

* Add tiling for ensemble

* Add tests for tiling for ensemble

* Moved ensemble tiler to separate file

* Modify padim config for ensemble

* Add tiling to dataset

* Revert changes to train

* Add tiling to collate fn

* Fix tiling in collate

* Change val. function to protected

* Add tile number logic

* Move collate fn to separate file

* Update tests for tiler

* Add training loop for ensemble

* Add model input size setup

* Move ens config to separate file

* Revert mvtec modifications

* Remove unused imports in mvtec

* Add batch adjustment to untiling

* Add predict step to ensemble

* Add comment and docstring to tile joining function

* Move tile joining to separate function

* Add joining for all tiled data

* Add joining for all box data

* Refactor pred. joining as modular class

* Fix box joining

* Add label and score joining

* Add ensemble visualization

* Add end of predict hook

* Add metric computation

* Fix metric thresholds

* Add removal of individual visualization

* Add demo1 notebook

* Add docstrings and cleanup

* Add memory benchmark

* Add modular class for storing predictions

* Add metric to separate class

* Refactor to support prediction data class

* Rename predictions class

* Add filesystem predictions class

* Add resized predictions class

* Fix joiner for classification task

* Add page peak to memory benchmark

* Add global stats calculation

* Add docstrings to stats calculation

* Refactor joiner for pipeline

* Refactor stats into pipeline

* Refactor metrics as pipeline block

* Refactor visualization as pipeline block

* Refactor postprocessing into a pipeline

* Add normalization and thresholding on joined predictions

* Refactor tiler to accept config file

* Add smoothing of tile joins.

* Refactor ensemble datamodule preparation

* Remove unused changes in dataloader

* Fix metric configuration

* Fix box coordinates in joining

* Add ensemble callbacks preparation function

* Fix box prediction bug in postprocess

* Add ensemble params to config

* Refactor postprocessing.

* Refactor post-processing

* Refactor predictions

* Code cleanup

* Optimize prediction storage

* Make join smoothing configurable

* Cleanup before PR

* Fix stats pipeline

* Fix logging strings

* Fix memory benchmark

* Fix tiler issues

* Fix import issues

* Fix naming in metrics and visualization

* Fix cyclic import

* Make logging lazy

* Refactor tiler tests

* Added collate tiling tests

* Added ensemble helper functions tests

* Refactor for dummy ensemble config

* Refactor for dummy base config

* Add tests for prediction storage

* Add tests for prediction joiner

* Add tests for visualization

* Fix small issues in tests

* Add metrics test

* Add post-processing tests

* Fix tiler to work with different instance

* Move seed setting inside train loop

* Fix pipeline stats bug

* Rename ensemble config fixture

* Add pipeline tests

* Fix config in pipeline tests

* Add training script test

* Fix types and docstrings

* Move and rename to tiled_ensemble

* Fix bug in label joining.

* Remove memory benchmark

* Cleanup files

* Fix metrics setup

* Rename collate function

* Add license to test files

* Rename fixtures

* Add more comments to tiled ensemble training

* Add start of training log message

* Refactor tiler to have explicit arguments

* Refactor pred. storage to have explicit arguments

* Refactor metrics to have explicit arguments

* Refactor visualization to have explicit arguments

* Refactor post-processing to have explicit arguments

* Sort imports

* Add test ensemble script

* Fix join smoothing bug

* Add more documentation to doc-strings

* Remove unused import

* Add brief tiled ensemble documentation

* Update typehints

* Make training args more clear

* Revert addition of no threshold option.

* Refactor normalization and threshold config

* Remove tiled ensemble from docs index

* Add comments to clarify parts of ensemble config

* Improve ensemble config comments

* Add num_tiles attribute to tiler.

* Fix metrics process docstring

* Fix visualization bug and cover with test

* Replace strings with enum

* Improve comments in joiner.

* Fix bug when model doesn't have anomaly maps.

* Improve docstrings (types, clarify).

* Fix visualization tests

* Fix dict membership checks

* Add saving of ensemble config file

* Update test script args

* Cover test script with tests

* Update export warning

* Fix case when no test or val data

* Improve documentation images

* Add images for documentation

* Add codacy suggestion

* Refactor joiner to single class

* Refactor storage names and config

* Update normalization and threshold stage names

* Add transforms independent input size to models

Signed-off-by: blaz-r <[email protected]>

* Make collate function a datamodule attribute

Signed-off-by: blaz-r <[email protected]>

* Refactor tiled ensemble train into pipeline step

Signed-off-by: blaz-r <[email protected]>

* Refactor tiled ensemble prediction into pipeline step

Signed-off-by: blaz-r <[email protected]>

* Refactor tiled ensemble merging into pipeline step

Signed-off-by: blaz-r <[email protected]>

* Refactor tiled ensemble seam smoothing into pipeline step

Signed-off-by: blaz-r <[email protected]>

* Refactor tiled stats calculation into pipeline step

Signed-off-by: blaz-r <[email protected]>

* Fix ckpt loading when predicting on test set.

Signed-off-by: blaz-r <[email protected]>

* Add logging and add tqdm to pipeline steps.

Signed-off-by: blaz-r <[email protected]>

* Refactor normalization pipeline step

Signed-off-by: blaz-r <[email protected]>

* Refactor thresholding into new pipeline job

* Fix transforms issue when predicting with dataloader

* Add visualization as new pipeline step

* Add metrics as new pipeline step

* Format the code and address some lint problems

Signed-off-by: Blaz Rolih <[email protected]>

* Add code to skip test if test split is none

Signed-off-by: Blaz Rolih <[email protected]>

* Add accelerator to metrics and smoothing

Signed-off-by: Blaz Rolih <[email protected]>

* Make threshold acq helper function and add to threshold to metrics

Signed-off-by: Blaz Rolih <[email protected]>

* Make a separate test pipeline

Signed-off-by: Blaz Rolih <[email protected]>

* Restructure tiled ensemble files into directories

Signed-off-by: Blaz Rolih <[email protected]>

* Pipeline code cleanup

Signed-off-by: Blaz Rolih <[email protected]>

* Remove old tiled ensemble files

Signed-off-by: blaz-r <[email protected]>

* Remove old post processing files

Signed-off-by: blaz-r <[email protected]>

* Fix sigma value read in smoothing

Signed-off-by: blaz-r <[email protected]>

* Update stats calc and normalization

Signed-off-by: blaz-r <[email protected]>

* Update args naming convention

Signed-off-by: blaz-r <[email protected]>

* Refactor code for nice config

Signed-off-by: blaz-r <[email protected]>

* Update docs structure for new system

Signed-off-by: blaz-r <[email protected]>

* Cleanup train code

Signed-off-by: blaz-r <[email protected]>

* Fix test script args

Signed-off-by: blaz-r <[email protected]>

* Update box merging

Signed-off-by: blaz-r <[email protected]>

* Refactor helper function tests

Signed-off-by: blaz-r <[email protected]>

* Small changes in helper and engine

Signed-off-by: blaz-r <[email protected]>

* Refactor merging tests

Signed-off-by: blaz-r <[email protected]>

* Refactor tiling tests

Signed-off-by: blaz-r <[email protected]>

* Refactor metrics test

Signed-off-by: blaz-r <[email protected]>

* Add support for different threshold methods

Signed-off-by: blaz-r <[email protected]>

* Format tests

Signed-off-by: blaz-r <[email protected]>

* Change test to predict

Signed-off-by: blaz-r <[email protected]>

* Refactor stats calculation tests

Signed-off-by: blaz-r <[email protected]>

* Refactor prediction data tests

Signed-off-by: blaz-r <[email protected]>

* Update metrics tests

Signed-off-by: blaz-r <[email protected]>

* Move metrics tests to components

Signed-off-by: blaz-r <[email protected]>

* Refactor seam smoothing tests

Signed-off-by: blaz-r <[email protected]>

* Refactor normalization tests

Signed-off-by: blaz-r <[email protected]>

* Move mock stats to conftest

Signed-off-by: blaz-r <[email protected]>

* Fix typehints for generator

Signed-off-by: blaz-r <[email protected]>

* Refactor threshold tests

Signed-off-by: blaz-r <[email protected]>

* Temporarily disable box minmax

Signed-off-by: blaz-r <[email protected]>

* Add tiled ensemble integration test

Signed-off-by: blaz-r <[email protected]>

* Fix normalization tests and add additional merging test

Signed-off-by: blaz-r <[email protected]>

* Add tile collater tests

Signed-off-by: blaz-r <[email protected]>

* Change dataset in tests to dummy

Signed-off-by: blaz-r <[email protected]>

* Format and fix linter errors

Signed-off-by: blaz-r <[email protected]>

* Format and some cleanup

Signed-off-by: blaz-r <[email protected]>

* Rename predict to eval

Signed-off-by: blaz-r <[email protected]>

* Update docs for refactored version of code

Signed-off-by: blaz-r <[email protected]>

* Cleanup the docs

Signed-off-by: blaz-r <[email protected]>

* Update ensemble engine

Signed-off-by: blaz-r <[email protected]>

* Remove boxes from pipelines and tests

Signed-off-by: blaz-r <[email protected]>

* Fix TODO comment issue

Signed-off-by: blaz-r <[email protected]>

* Fix unused model in ens. engine

Signed-off-by: blaz-r <[email protected]>

* Fix path case in test

Signed-off-by: blaz-r <[email protected]>

* Change temporary dir to project_path

Signed-off-by: blaz-r <[email protected]>

* Change mvtec to MVTec in test path

Signed-off-by: Blaz Rolih <[email protected]>

---------

Signed-off-by: blaz-r <[email protected]>
Signed-off-by: Blaz Rolih <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* 📚 Add training from a checkpoint example (#2389)

* Add training from a checkpoint example

Signed-off-by: Samet Akcay <[email protected]>

* Replace patchcore example with efficient-ad

Signed-off-by: Samet Akcay <[email protected]>

---------

Signed-off-by: Samet Akcay <[email protected]>

* Export experiment duration in seconds in CSV. (#2392)

* Export experiment duration in seconds in CSV.

Signed-off-by: Weilin Xu <[email protected]>

* Update CHANGELOG

Signed-off-by: Weilin Xu <[email protected]>

* Log fit and test durations separately.

Signed-off-by: Weilin Xu <[email protected]>

---------

Signed-off-by: Weilin Xu <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* Make single GPU benchmarking 5x more efficient (#2390)

* Use SerialRunner if only one CUDA device is available.

Signed-off-by: Weilin Xu <[email protected]>

* Resolve PLR6201.

Signed-off-by: Weilin Xu <[email protected]>

* Update CHANGELOG.

Signed-off-by: Weilin Xu <[email protected]>

* Keep the same logging level in benchmarking.

Signed-off-by: Weilin Xu <[email protected]>

---------

Signed-off-by: Weilin Xu <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* 🐞 Fix installation package issues (#2395)

* Update the coverage settings

Signed-off-by: Samet Akcay <[email protected]>

* Remove VlmAd's relative import

Signed-off-by: Samet Akcay <[email protected]>

* Revert relative imports

Signed-off-by: Samet Akcay <[email protected]>

* Add type checking

Signed-off-by: Samet Akcay <[email protected]>

---------

Signed-off-by: Samet Akcay <[email protected]>

* Export the flattened config in benchmark CSV. (#2391)

* Export the flattened config in benchmark CSV.

Signed-off-by: Weilin Xu <[email protected]>

* Update CHANGELOG

Signed-off-by: Weilin Xu <[email protected]>

* Reuse the existing flatten_dict().

Signed-off-by: Weilin Xu <[email protected]>

---------

Signed-off-by: Weilin Xu <[email protected]>
Co-authored-by: Samet Akcay <[email protected]>

* `v1.2.0` Release (#2397)

Prepare v1.2.0 release (#2396)

* Update changelog



* Update the version in __init__



---------

Signed-off-by: Samet Akcay <[email protected]>

* Bump Anomalib version to `2.0.0dev` in `main` (#2402)

Update __init__.py

* 🐞Replace package_available with module_available (#2407)

* fix datumaro config

* fix edge case in benchmarking pipeline

* fix vlm ad

* remove ensemble tiling in preparation of refactor

---------

Signed-off-by: dependabot[bot] <[email protected]>
Signed-off-by: Samet Akcay <[email protected]>
Signed-off-by: Blaz Rolih <[email protected]>
Signed-off-by: jpcbertoldo <[email protected]>
Signed-off-by: Ashwin Vaidya <[email protected]>
Signed-off-by: sahusiddharth <[email protected]>
Signed-off-by: Bepitic <[email protected]>
Signed-off-by: blaz-r <[email protected]>
Signed-off-by: Weilin Xu <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Samet Akcay <[email protected]>
Co-authored-by: Blaž Rolih <[email protected]>
Co-authored-by: Ashwin Vaidya <[email protected]>
Co-authored-by: Joao P C Bertoldo <[email protected]>
Co-authored-by: Marcus Pertlwieser <[email protected]>
Co-authored-by: Siddharth Sahu <[email protected]>
Co-authored-by: Paco <[email protected]>
Co-authored-by: Weilin Xu <[email protected]>
Co-authored-by: Harim Kang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Checkpoint loading fails when model doesn't use normalization
3 participants