Skip to content

Commit

Permalink
Added metrics for 2024.5 nightly (#2954)
Browse files Browse the repository at this point in the history
### Changes

as stated in the title
This PR introduced changes in accuracy:
openvinotoolkit/openvino#26147
but it's coming from float error:

![image](https://github.com/user-attachments/assets/e819a14b-3c74-4568-beef-7b43737128a6)

### Reason for changes

As result validation with openvino nightly failed:

![image](https://github.com/user-attachments/assets/23fe22dd-6ec2-4577-98df-f26732c7962a)

### Related tickets

150613
151260


### Tests

- [x] job/NNCF/job/manual/job/post_training_weight_compression/163/
  • Loading branch information
ljaljushkin authored Sep 9, 2024
1 parent 67d6386 commit a75b47a
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions tests/post_training/data/wc_reference_data_2024.5.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
tinyllama_data_aware_awq_stateful_backend_OV:
metric_value: 0.85571
num_int4: 94
num_int8: 124
tinyllama_data_aware_awq_scale_estimation_backend_OV:
metric_value: 0.86355
num_int4: 94
num_int8: 124
tinyllama_data_aware_awq_scale_estimation_stateful_backend_OV:
metric_value: 0.86355
num_int4: 94
num_int8: 124
tinyllama_data_aware_gptq_backend_OV:
metric_value: 0.81664
num_int4: 94
num_int8: 124
metrics_xfail_reason: "Issue-148819"
tinyllama_scale_estimation_per_channel_backend_OV:
metric_value: 0.80798
num_int4: 188
num_int8: 124
metrics_xfail_reason: "Issue-148819"

0 comments on commit a75b47a

Please sign in to comment.