[CI] Fix benchmark on gpu (#1706) #3753
benchmarks.yml
on: push
CPU Pytest benchmark
5m 10s
GPU Pytest benchmark
9m 42s
Annotations
3 errors and 7 warnings
GPU Pytest benchmark
WARNING: Running pip as the 'root' user can result in broken permissions and conflicting behaviour with the system package manager. It is recommended to use a virtual environment instead: https://pip.pypa.io/warnings/venv
|
GPU Pytest benchmark
WARNING: The directory '/github/home/.cache/pip' or its parent directory is not owned or is not writable by the current user. The cache has been disabled. Check the permissions and owner of that directory. If executing pip with sudo, you should use sudo's -H flag.
|
GPU Pytest benchmark
# :warning: **Performance Alert** :warning:
Possible performance regression was detected for benchmark **'GPU Benchmark Results'**.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold `2`.
| Benchmark suite | Current: c2edf357d10ae93f18eaf06df7f19212949ee30e | Previous: 5cac16a0bc1e1265a1ff6b5e923f859f2fc3929e | Ratio |
|-|-|-|-|
| `benchmarks/test_collectors_benchmark.py::test_sync` | `9.810059463167097` iter/sec (`stddev: 0.002041292288897499`) | `28.532412789722958` iter/sec (`stddev: 0.0026846866550320116`) | `2.91` |
| `benchmarks/test_collectors_benchmark.py::test_async` | `10.080410924621184` iter/sec (`stddev: 0.09701922146618137`) | `30.9133477332784` iter/sec (`stddev: 0.0311860928756896`) | `3.07` |
| `benchmarks/test_envs_benchmark.py::test_simple` | `1.1489713303686049` iter/sec (`stddev: 0.03784047056231992`) | `2.3423070056657287` iter/sec (`stddev: 0.03146140112651748`) | `2.04` |
| `benchmarks/test_envs_benchmark.py::test_parallel` | `0.4121471142355259` iter/sec (`stddev: 0.033077600972760635`) | `0.9163986086644761` iter/sec (`stddev: 0.03835037655176077`) | `2.22` |
| `benchmarks/test_objectives_benchmarks.py::test_values[td1_return_estimate-False-False]` | `18.737783606872945` iter/sec (`stddev: 0.0006650968771473066`) | `38.14853728413215` iter/sec (`stddev: 0.00014240161117313105`) | `2.04` |
| `benchmarks/test_objectives_benchmarks.py::test_values[td_lambda_return_estimate-True-False]` | `11.782401705119526` iter/sec (`stddev: 0.00029902150016902155`) | `27.50430573580675` iter/sec (`stddev: 0.00031334395849292625`) | `2.33` |
| `benchmarks/test_objectives_benchmarks.py::test_gae_speed[generalized_advantage_estimate-False-1-512]` | `43.328464861657416` iter/sec (`stddev: 0.00008755735906681894`) | `123.99011192517025` iter/sec (`stddev: 0.000029043194140820502`) | `2.86` |
This comment was automatically generated by [workflow](https://github.com/pytorch/rl/actions?query=workflow%3AContinuous%20Benchmark) using [github-action-benchmark](https://github.com/marketplace/actions/continuous-benchmark).
CC: @vmoens
Comment was generated at https://github.com/pytorch/rl/commit/c2edf357d10ae93f18eaf06df7f19212949ee30e#commitcomment-133068059
|
GPU Pytest benchmark
Performance alert! Previous value was 28.532412789722958 and current value is 9.810059463167097. It is 2.908485203056201x worse than previous exceeding a ratio threshold 2
|
GPU Pytest benchmark
Performance alert! Previous value was 30.9133477332784 and current value is 10.080410924621184. It is 3.066675353260969x worse than previous exceeding a ratio threshold 2
|
GPU Pytest benchmark
Performance alert! Previous value was 2.3423070056657287 and current value is 1.1489713303686049. It is 2.038612229701403x worse than previous exceeding a ratio threshold 2
|
GPU Pytest benchmark
Performance alert! Previous value was 0.9163986086644761 and current value is 0.4121471142355259. It is 2.2234745240525697x worse than previous exceeding a ratio threshold 2
|
GPU Pytest benchmark
Performance alert! Previous value was 38.14853728413215 and current value is 18.737783606872945. It is 2.0359151372705266x worse than previous exceeding a ratio threshold 2
|
GPU Pytest benchmark
Performance alert! Previous value was 27.50430573580675 and current value is 11.782401705119526. It is 2.334354779624935x worse than previous exceeding a ratio threshold 2
|
GPU Pytest benchmark
Performance alert! Previous value was 123.99011192517025 and current value is 43.328464861657416. It is 2.861631777656x worse than previous exceeding a ratio threshold 2
|