[Feature] Minari datasets

[vmoens]

Inegrates Minari in our dataset hub

Usage:

from torchrl.data.datasets.minari import MinariExperienceReplay

data = MinariExperienceReplay("door-human-v1", batch_size=32)
for sample in data:
    print(sample)
    break

which prints

TensorDict(
    fields={
        action: Tensor(shape=torch.Size([32, 28]), device=cpu, dtype=torch.float32, is_shared=False),
        index: Tensor(shape=torch.Size([32]), device=cpu, dtype=torch.int64, is_shared=False),
        info: TensorDict(
            fields={
                success: Tensor(shape=torch.Size([32]), device=cpu, dtype=torch.bool, is_shared=False)},
            batch_size=torch.Size([32]),
            device=None,
            is_shared=False),
        next: TensorDict(
            fields={
                observation: Tensor(shape=torch.Size([32, 39]), device=cpu, dtype=torch.float64, is_shared=False),
                reward: Tensor(shape=torch.Size([32, 1]), device=cpu, dtype=torch.float64, is_shared=False),
                state: TensorDict(
                    fields={
                        door_body_pos: Tensor(shape=torch.Size([32, 3]), device=cpu, dtype=torch.float64, is_shared=False),
                        qpos: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False),
                        qvel: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False)},
                    batch_size=torch.Size([32]),
                    device=None,
                    is_shared=False),
                terminated: Tensor(shape=torch.Size([32, 1]), device=cpu, dtype=torch.bool, is_shared=False),
                truncated: Tensor(shape=torch.Size([32, 1]), device=cpu, dtype=torch.bool, is_shared=False)},
            batch_size=torch.Size([32]),
            device=None,
            is_shared=False),
        observation: Tensor(shape=torch.Size([32, 39]), device=cpu, dtype=torch.float64, is_shared=False),
        state: TensorDict(
            fields={
                door_body_pos: Tensor(shape=torch.Size([32, 3]), device=cpu, dtype=torch.float64, is_shared=False),
                qpos: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False),
                qvel: Tensor(shape=torch.Size([32, 30]), device=cpu, dtype=torch.float64, is_shared=False)},
            batch_size=torch.Size([32]),
            device=None,
            is_shared=False)},
    batch_size=torch.Size([32]),
    device=None,
    is_shared=False)

TODO:

• Tests across several datasets
• Decide what to do with text data
• integrate metadata in the buffer class (and figure out how to save them without recurring to pickle)
  The plan would be to save things as a json after converting the specs to torchrl format

cc @rodrigodelazcano @WillDudley @younik @BY571

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/rl/1721

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ You can merge normally! (17 Unrelated Failures)

As of commit 0090fe0 with merge base d545364 ():

FLAKY - The following jobs failed but were likely due to flakiness present on trunk:

• Brax Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)
• Examples Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
• Olddeps Tests on Linux / unittests (3.8, 11.6) / linux-job (gh)
• Optdeps Tests on Linux / tests (3.9, 12.1) / linux-job (gh)
• PettingZoo Tests on Linux / unittests / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.10) / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.11) / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.8) / linux-job (gh)
• Unit-tests on Linux CPU / tests (3.9) / linux-job (gh)
• Unit-tests on Linux GPU / tests (3.8, 12.1) / linux-job (gh)
• Unit-tests on Linux GPU, latest stable release / tests (3.8, 11.8) / linux-job (gh)
• Unit-tests on MacOS CPU / tests (3.11) / macos-job (gh)
• Unit-tests on MacOS CPU / tests (3.8) / macos-job (gh)
• Unit-tests on Windows CPU / unittests / windows-job (gh)
• Vmas Tests on Linux / unittests (3.9, 12.1) / linux-job (gh)

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• Habitat Tests on Linux / tests (3.9, 11.6) / linux-job (gh)
• Unit-tests on Windows GPU / unittests / windows-job (gh)

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of CPU Benchmark Tests

Total Benchmarks: 89. Improved: $\large\color{#35bf28}7$. Worsened: $\large\color{#d91a1a}3$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	64.0120ms	62.7584ms	15.9341 Ops/s	15.0018 Ops/s	$\textbf{\color{#35bf28}+6.21\%}$
test_sync	40.3020ms	34.0208ms	29.3938 Ops/s	26.9923 Ops/s	$\textbf{\color{#35bf28}+8.90\%}$
test_async	58.1312ms	32.6184ms	30.6576 Ops/s	30.5066 Ops/s	$\color{#35bf28}+0.49\%$
test_simple	0.4861s	0.4319s	2.3156 Ops/s	2.2391 Ops/s	$\color{#35bf28}+3.41\%$
test_transformed	0.6426s	0.5934s	1.6853 Ops/s	1.6404 Ops/s	$\color{#35bf28}+2.74\%$
test_serial	1.3772s	1.3355s	0.7488 Ops/s	0.7355 Ops/s	$\color{#35bf28}+1.81\%$
test_parallel	1.3705s	1.2991s	0.7698 Ops/s	0.7843 Ops/s	$\color{#d91a1a}-1.85\%$
test_step_mdp_speed[True-True-True-True-True]	0.1881ms	22.7169μs	44.0202 KOps/s	44.1618 KOps/s	$\color{#d91a1a}-0.32\%$
test_step_mdp_speed[True-True-True-True-False]	80.6730μs	13.4385μs	74.4133 KOps/s	73.5029 KOps/s	$\color{#35bf28}+1.24\%$
test_step_mdp_speed[True-True-True-False-True]	41.6380μs	13.8629μs	72.1348 KOps/s	71.4287 KOps/s	$\color{#35bf28}+0.99\%$
test_step_mdp_speed[True-True-True-False-False]	56.7350μs	8.1495μs	122.7076 KOps/s	120.6534 KOps/s	$\color{#35bf28}+1.70\%$
test_step_mdp_speed[True-True-False-True-True]	59.7320μs	23.9453μs	41.7619 KOps/s	41.1156 KOps/s	$\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-True-False-True-False]	51.1560μs	14.8894μs	67.1618 KOps/s	66.8581 KOps/s	$\color{#35bf28}+0.45\%$
test_step_mdp_speed[True-True-False-False-True]	47.3990μs	15.1311μs	66.0892 KOps/s	64.2442 KOps/s	$\color{#35bf28}+2.87\%$
test_step_mdp_speed[True-True-False-False-False]	35.4770μs	9.4989μs	105.2758 KOps/s	103.3998 KOps/s	$\color{#35bf28}+1.81\%$
test_step_mdp_speed[True-False-True-True-True]	89.0870μs	25.1894μs	39.6993 KOps/s	38.6878 KOps/s	$\color{#35bf28}+2.61\%$
test_step_mdp_speed[True-False-True-True-False]	51.0960μs	16.0424μs	62.3349 KOps/s	60.6123 KOps/s	$\color{#35bf28}+2.84\%$
test_step_mdp_speed[True-False-True-False-True]	55.5640μs	15.2138μs	65.7298 KOps/s	64.3426 KOps/s	$\color{#35bf28}+2.16\%$
test_step_mdp_speed[True-False-True-False-False]	39.7450μs	9.5446μs	104.7718 KOps/s	99.5583 KOps/s	$\textbf{\color{#35bf28}+5.24\%}$
test_step_mdp_speed[True-False-False-True-True]	86.7130μs	26.4480μs	37.8101 KOps/s	37.0769 KOps/s	$\color{#35bf28}+1.98\%$
test_step_mdp_speed[True-False-False-True-False]	48.5110μs	17.2359μs	58.0183 KOps/s	57.1005 KOps/s	$\color{#35bf28}+1.61\%$
test_step_mdp_speed[True-False-False-False-True]	41.3480μs	16.0476μs	62.3146 KOps/s	61.2224 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[True-False-False-False-False]	44.0730μs	10.5574μs	94.7204 KOps/s	93.0646 KOps/s	$\color{#35bf28}+1.78\%$
test_step_mdp_speed[False-True-True-True-True]	54.1120μs	25.2365μs	39.6252 KOps/s	38.8743 KOps/s	$\color{#35bf28}+1.93\%$
test_step_mdp_speed[False-True-True-True-False]	61.3950μs	15.8600μs	63.0516 KOps/s	61.4092 KOps/s	$\color{#35bf28}+2.67\%$
test_step_mdp_speed[False-True-True-False-True]	53.2200μs	17.0784μs	58.5537 KOps/s	57.4952 KOps/s	$\color{#35bf28}+1.84\%$
test_step_mdp_speed[False-True-True-False-False]	34.3640μs	10.5691μs	94.6157 KOps/s	93.4061 KOps/s	$\color{#35bf28}+1.29\%$
test_step_mdp_speed[False-True-False-True-True]	64.3810μs	26.0916μs	38.3265 KOps/s	37.4620 KOps/s	$\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-True-False-True-False]	0.1141ms	17.7674μs	56.2830 KOps/s	56.6058 KOps/s	$\color{#d91a1a}-0.57\%$
test_step_mdp_speed[False-True-False-False-True]	46.3670μs	18.2345μs	54.8410 KOps/s	53.8008 KOps/s	$\color{#35bf28}+1.93\%$
test_step_mdp_speed[False-True-False-False-False]	36.2380μs	11.7749μs	84.9262 KOps/s	83.4207 KOps/s	$\color{#35bf28}+1.80\%$
test_step_mdp_speed[False-False-True-True-True]	67.3670μs	27.5541μs	36.2923 KOps/s	35.8412 KOps/s	$\color{#35bf28}+1.26\%$
test_step_mdp_speed[False-False-True-True-False]	53.4300μs	18.5630μs	53.8706 KOps/s	52.2579 KOps/s	$\color{#35bf28}+3.09\%$
test_step_mdp_speed[False-False-True-False-True]	46.0260μs	18.2639μs	54.7529 KOps/s	53.7326 KOps/s	$\color{#35bf28}+1.90\%$
test_step_mdp_speed[False-False-True-False-False]	77.9760μs	12.0160μs	83.2224 KOps/s	81.3431 KOps/s	$\color{#35bf28}+2.31\%$
test_step_mdp_speed[False-False-False-True-True]	62.5280μs	28.5652μs	35.0076 KOps/s	34.6084 KOps/s	$\color{#35bf28}+1.15\%$
test_step_mdp_speed[False-False-False-True-False]	3.5814ms	19.6839μs	50.8028 KOps/s	50.2767 KOps/s	$\color{#35bf28}+1.05\%$
test_step_mdp_speed[False-False-False-False-True]	56.9070μs	19.3138μs	51.7763 KOps/s	51.2777 KOps/s	$\color{#35bf28}+0.97\%$
test_step_mdp_speed[False-False-False-False-False]	91.9220μs	12.8918μs	77.5685 KOps/s	76.8348 KOps/s	$\color{#35bf28}+0.95\%$
test_values[generalized_advantage_estimate-True-True]	12.5958ms	12.1275ms	82.4570 Ops/s	82.8755 Ops/s	$\color{#d91a1a}-0.50\%$
test_values[vec_generalized_advantage_estimate-True-True]	29.6932ms	27.0544ms	36.9626 Ops/s	37.1271 Ops/s	$\color{#d91a1a}-0.44\%$
test_values[td0_return_estimate-False-False]	0.2503ms	0.1846ms	5.4176 KOps/s	5.2306 KOps/s	$\color{#35bf28}+3.57\%$
test_values[td1_return_estimate-False-False]	27.4010ms	26.1068ms	38.3041 Ops/s	38.7151 Ops/s	$\color{#d91a1a}-1.06\%$
test_values[vec_td1_return_estimate-False-False]	59.1113ms	29.1509ms	34.3043 Ops/s	37.0920 Ops/s	$\textbf{\color{#d91a1a}-7.52\%}$
test_values[td_lambda_return_estimate-True-False]	39.5949ms	36.5558ms	27.3554 Ops/s	27.3570 Ops/s	$-0.01\%$
test_values[vec_td_lambda_return_estimate-True-False]	36.7762ms	27.3424ms	36.5732 Ops/s	36.9070 Ops/s	$\color{#d91a1a}-0.90\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	8.5144ms	8.2838ms	120.7169 Ops/s	125.5552 Ops/s	$\color{#d91a1a}-3.85\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	10.1785ms	2.0282ms	493.0470 Ops/s	540.3755 Ops/s	$\textbf{\color{#d91a1a}-8.76\%}$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.5712ms	0.4335ms	2.3067 KOps/s	2.2438 KOps/s	$\color{#35bf28}+2.80\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	48.8926ms	41.5851ms	24.0471 Ops/s	24.0453 Ops/s	$+0.01\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	11.1551ms	2.5934ms	385.6003 Ops/s	391.0177 Ops/s	$\color{#d91a1a}-1.39\%$
test_dqn_speed	9.6636ms	1.6306ms	613.2897 Ops/s	598.0734 Ops/s	$\color{#35bf28}+2.54\%$
test_ddpg_speed	70.5358ms	3.8886ms	257.1637 Ops/s	268.4252 Ops/s	$\color{#d91a1a}-4.20\%$
test_sac_speed	19.6605ms	10.3811ms	96.3293 Ops/s	95.2308 Ops/s	$\color{#35bf28}+1.15\%$
test_redq_speed	28.0531ms	19.7165ms	50.7190 Ops/s	46.0802 Ops/s	$\textbf{\color{#35bf28}+10.07\%}$
test_redq_deprec_speed	24.3397ms	15.6511ms	63.8933 Ops/s	63.4842 Ops/s	$\color{#35bf28}+0.64\%$
test_td3_speed	18.3716ms	10.6273ms	94.0970 Ops/s	92.7044 Ops/s	$\color{#35bf28}+1.50\%$
test_cql_speed	46.8275ms	38.8169ms	25.7620 Ops/s	25.6659 Ops/s	$\color{#35bf28}+0.37\%$
test_a2c_speed	16.3290ms	8.3596ms	119.6234 Ops/s	122.1672 Ops/s	$\color{#d91a1a}-2.08\%$
test_ppo_speed	17.6756ms	8.6614ms	115.4554 Ops/s	117.8676 Ops/s	$\color{#d91a1a}-2.05\%$
test_reinforce_speed	16.2194ms	7.4543ms	134.1499 Ops/s	136.7534 Ops/s	$\color{#d91a1a}-1.90\%$
test_iql_speed	43.8759ms	34.6873ms	28.8290 Ops/s	28.6919 Ops/s	$\color{#35bf28}+0.48\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	2.3748ms	1.9093ms	523.7546 Ops/s	521.8334 Ops/s	$\color{#35bf28}+0.37\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.2206ms	2.0948ms	477.3676 Ops/s	489.9670 Ops/s	$\color{#d91a1a}-2.57\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.4099ms	2.0925ms	477.8936 Ops/s	481.8153 Ops/s	$\color{#d91a1a}-0.81\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.8770ms	1.9289ms	518.4215 Ops/s	519.1230 Ops/s	$\color{#d91a1a}-0.14\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.5604ms	2.0896ms	478.5696 Ops/s	488.7607 Ops/s	$\color{#d91a1a}-2.09\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.4691ms	2.1041ms	475.2620 Ops/s	486.4268 Ops/s	$\color{#d91a1a}-2.30\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.0791ms	1.9008ms	526.1034 Ops/s	511.2014 Ops/s	$\color{#35bf28}+2.92\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	3.2073ms	2.0655ms	484.1512 Ops/s	494.3654 Ops/s	$\color{#d91a1a}-2.07\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.0001ms	2.0953ms	477.2640 Ops/s	479.2652 Ops/s	$\color{#d91a1a}-0.42\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.9333ms	2.0725ms	482.5188 Ops/s	511.9925 Ops/s	$\textbf{\color{#d91a1a}-5.76\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.2948ms	2.1232ms	470.9800 Ops/s	428.7532 Ops/s	$\textbf{\color{#35bf28}+9.85\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.8066ms	2.0766ms	481.5624 Ops/s	484.2339 Ops/s	$\color{#d91a1a}-0.55\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.5995ms	1.9028ms	525.5400 Ops/s	516.0127 Ops/s	$\color{#35bf28}+1.85\%$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.0803ms	2.1048ms	475.1009 Ops/s	442.4543 Ops/s	$\textbf{\color{#35bf28}+7.38\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.6382ms	2.0841ms	479.8162 Ops/s	479.6149 Ops/s	$\color{#35bf28}+0.04\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.4320ms	1.9658ms	508.6939 Ops/s	515.9570 Ops/s	$\color{#d91a1a}-1.41\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	2.1186ms	2.0377ms	490.7511 Ops/s	438.2522 Ops/s	$\textbf{\color{#35bf28}+11.98\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.3723ms	2.1190ms	471.9191 Ops/s	476.7122 Ops/s	$\color{#d91a1a}-1.01\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.1648s	17.6102ms	56.7852 Ops/s	55.3291 Ops/s	$\color{#35bf28}+2.63\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1068s	16.3464ms	61.1755 Ops/s	60.2557 Ops/s	$\color{#35bf28}+1.53\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1176s	16.6174ms	60.1778 Ops/s	60.6476 Ops/s	$\color{#d91a1a}-0.77\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1155s	16.8098ms	59.4891 Ops/s	60.2389 Ops/s	$\color{#d91a1a}-1.24\%$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1174s	16.9969ms	58.8344 Ops/s	60.8407 Ops/s	$\color{#d91a1a}-3.30\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1148s	16.9554ms	58.9784 Ops/s	60.1354 Ops/s	$\color{#d91a1a}-1.92\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1209s	17.2115ms	58.1008 Ops/s	60.1386 Ops/s	$\color{#d91a1a}-3.39\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1149s	16.9357ms	59.0468 Ops/s	59.5426 Ops/s	$\color{#d91a1a}-0.83\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1174s	16.8949ms	59.1896 Ops/s	59.8852 Ops/s	$\color{#d91a1a}-1.16\%$

[github-actions]

$\color{#D29922}\textsf{\Large⚠\kern{0.2cm}\normalsize Warning}$ Result of GPU Benchmark Tests

Total Benchmarks: 92. Improved: $\large\color{#35bf28}3$. Worsened: $\large\color{#d91a1a}6$.

Expand to view detailed results

Name	Max	Mean	Ops	Ops on Repo HEAD	Change
test_single	0.1229s	0.1216s	8.2261 Ops/s	8.2255 Ops/s	$+0.01\%$
test_sync	0.1033s	0.1028s	9.7268 Ops/s	9.7094 Ops/s	$\color{#35bf28}+0.18\%$
test_async	0.2804s	0.1009s	9.9148 Ops/s	10.0190 Ops/s	$\color{#d91a1a}-1.04\%$
test_single_pixels	0.1452s	0.1445s	6.9195 Ops/s	6.9528 Ops/s	$\color{#d91a1a}-0.48\%$
test_sync_pixels	96.9551ms	95.6214ms	10.4579 Ops/s	10.5817 Ops/s	$\color{#d91a1a}-1.17\%$
test_async_pixels	0.2439s	91.0323ms	10.9851 Ops/s	10.9647 Ops/s	$\color{#35bf28}+0.19\%$
test_simple	0.9726s	0.9168s	1.0907 Ops/s	1.1390 Ops/s	$\color{#d91a1a}-4.24\%$
test_transformed	1.2370s	1.1712s	0.8538 Ops/s	0.8880 Ops/s	$\color{#d91a1a}-3.85\%$
test_serial	2.6121s	2.5108s	0.3983 Ops/s	0.4085 Ops/s	$\color{#d91a1a}-2.50\%$
test_parallel	2.5613s	2.4985s	0.4002 Ops/s	0.4008 Ops/s	$\color{#d91a1a}-0.15\%$
test_step_mdp_speed[True-True-True-True-True]	73.0510μs	35.1923μs	28.4153 KOps/s	27.7517 KOps/s	$\color{#35bf28}+2.39\%$
test_step_mdp_speed[True-True-True-True-False]	46.7910μs	20.6009μs	48.5416 KOps/s	47.5548 KOps/s	$\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-True-True-False-True]	41.7510μs	21.0448μs	47.5176 KOps/s	47.7728 KOps/s	$\color{#d91a1a}-0.53\%$
test_step_mdp_speed[True-True-True-False-False]	28.5700μs	12.1157μs	82.5375 KOps/s	80.6843 KOps/s	$\color{#35bf28}+2.30\%$
test_step_mdp_speed[True-True-False-True-True]	79.7310μs	37.1144μs	26.9437 KOps/s	26.5774 KOps/s	$\color{#35bf28}+1.38\%$
test_step_mdp_speed[True-True-False-True-False]	48.4510μs	22.2547μs	44.9343 KOps/s	43.8160 KOps/s	$\color{#35bf28}+2.55\%$
test_step_mdp_speed[True-True-False-False-True]	42.5510μs	22.4583μs	44.5271 KOps/s	44.2858 KOps/s	$\color{#35bf28}+0.54\%$
test_step_mdp_speed[True-True-False-False-False]	33.4610μs	13.9100μs	71.8906 KOps/s	70.7420 KOps/s	$\color{#35bf28}+1.62\%$
test_step_mdp_speed[True-False-True-True-True]	66.5110μs	39.6265μs	25.2356 KOps/s	25.1582 KOps/s	$\color{#35bf28}+0.31\%$
test_step_mdp_speed[True-False-True-True-False]	47.9210μs	24.6450μs	40.5762 KOps/s	40.5101 KOps/s	$\color{#35bf28}+0.16\%$
test_step_mdp_speed[True-False-True-False-True]	40.3510μs	22.6878μs	44.0766 KOps/s	43.3957 KOps/s	$\color{#35bf28}+1.57\%$
test_step_mdp_speed[True-False-True-False-False]	30.3200μs	13.9199μs	71.8395 KOps/s	70.2144 KOps/s	$\color{#35bf28}+2.31\%$
test_step_mdp_speed[True-False-False-True-True]	66.3810μs	40.5062μs	24.6876 KOps/s	24.5046 KOps/s	$\color{#35bf28}+0.75\%$
test_step_mdp_speed[True-False-False-True-False]	42.1300μs	26.0529μs	38.3834 KOps/s	37.5996 KOps/s	$\color{#35bf28}+2.08\%$
test_step_mdp_speed[True-False-False-False-True]	42.2210μs	24.6554μs	40.5590 KOps/s	41.1845 KOps/s	$\color{#d91a1a}-1.52\%$
test_step_mdp_speed[True-False-False-False-False]	33.6110μs	15.9010μs	62.8890 KOps/s	62.5256 KOps/s	$\color{#35bf28}+0.58\%$
test_step_mdp_speed[False-True-True-True-True]	70.0110μs	38.9052μs	25.7035 KOps/s	25.1498 KOps/s	$\color{#35bf28}+2.20\%$
test_step_mdp_speed[False-True-True-True-False]	49.8700μs	24.4048μs	40.9755 KOps/s	39.6258 KOps/s	$\color{#35bf28}+3.41\%$
test_step_mdp_speed[False-True-True-False-True]	53.7410μs	26.6862μs	37.4726 KOps/s	37.1666 KOps/s	$\color{#35bf28}+0.82\%$
test_step_mdp_speed[False-True-True-False-False]	50.2100μs	15.9876μs	62.5485 KOps/s	61.5377 KOps/s	$\color{#35bf28}+1.64\%$
test_step_mdp_speed[False-True-False-True-True]	78.0620μs	42.0881μs	23.7597 KOps/s	24.1305 KOps/s	$\color{#d91a1a}-1.54\%$
test_step_mdp_speed[False-True-False-True-False]	67.6910μs	26.5237μs	37.7022 KOps/s	37.0614 KOps/s	$\color{#35bf28}+1.73\%$
test_step_mdp_speed[False-True-False-False-True]	47.7100μs	29.0031μs	34.4790 KOps/s	34.6041 KOps/s	$\color{#d91a1a}-0.36\%$
test_step_mdp_speed[False-True-False-False-False]	75.8010μs	17.7349μs	56.3859 KOps/s	55.4869 KOps/s	$\color{#35bf28}+1.62\%$
test_step_mdp_speed[False-False-True-True-True]	62.9510μs	42.7804μs	23.3752 KOps/s	23.0472 KOps/s	$\color{#35bf28}+1.42\%$
test_step_mdp_speed[False-False-True-True-False]	45.3700μs	28.0964μs	35.5917 KOps/s	34.9351 KOps/s	$\color{#35bf28}+1.88\%$
test_step_mdp_speed[False-False-True-False-True]	71.6410μs	28.8953μs	34.6077 KOps/s	35.0415 KOps/s	$\color{#d91a1a}-1.24\%$
test_step_mdp_speed[False-False-True-False-False]	42.1100μs	17.4232μs	57.3949 KOps/s	55.8001 KOps/s	$\color{#35bf28}+2.86\%$
test_step_mdp_speed[False-False-False-True-True]	85.2120μs	44.0494μs	22.7018 KOps/s	22.7311 KOps/s	$\color{#d91a1a}-0.13\%$
test_step_mdp_speed[False-False-False-True-False]	56.8410μs	30.0912μs	33.2323 KOps/s	32.5202 KOps/s	$\color{#35bf28}+2.19\%$
test_step_mdp_speed[False-False-False-False-True]	72.1010μs	30.1872μs	33.1266 KOps/s	32.8009 KOps/s	$\color{#35bf28}+0.99\%$
test_step_mdp_speed[False-False-False-False-False]	41.0210μs	19.6134μs	50.9855 KOps/s	50.4653 KOps/s	$\color{#35bf28}+1.03\%$
test_values[generalized_advantage_estimate-True-True]	25.5640ms	25.1359ms	39.7838 Ops/s	40.1876 Ops/s	$\color{#d91a1a}-1.00\%$
test_values[vec_generalized_advantage_estimate-True-True]	86.7363ms	3.2932ms	303.6552 Ops/s	304.9340 Ops/s	$\color{#d91a1a}-0.42\%$
test_values[td0_return_estimate-False-False]	0.1028ms	67.5450μs	14.8049 KOps/s	15.1913 KOps/s	$\color{#d91a1a}-2.54\%$
test_values[td1_return_estimate-False-False]	57.3057ms	55.7678ms	17.9315 Ops/s	18.4608 Ops/s	$\color{#d91a1a}-2.87\%$
test_values[vec_td1_return_estimate-False-False]	2.0377ms	1.7458ms	572.8072 Ops/s	580.5355 Ops/s	$\color{#d91a1a}-1.33\%$
test_values[td_lambda_return_estimate-True-False]	90.7970ms	89.3277ms	11.1947 Ops/s	10.9251 Ops/s	$\color{#35bf28}+2.47\%$
test_values[vec_td_lambda_return_estimate-True-False]	2.0122ms	1.7337ms	576.7972 Ops/s	581.9462 Ops/s	$\color{#d91a1a}-0.88\%$
test_gae_speed[generalized_advantage_estimate-False-1-512]	24.5635ms	23.9340ms	41.7815 Ops/s	40.5597 Ops/s	$\color{#35bf28}+3.01\%$
test_gae_speed[vec_generalized_advantage_estimate-True-1-512]	0.8816ms	0.7215ms	1.3860 KOps/s	1.3931 KOps/s	$\color{#d91a1a}-0.51\%$
test_gae_speed[vec_generalized_advantage_estimate-False-1-512]	0.7499ms	0.6801ms	1.4704 KOps/s	1.4821 KOps/s	$\color{#d91a1a}-0.79\%$
test_gae_speed[vec_generalized_advantage_estimate-True-32-512]	1.5480ms	1.4759ms	677.5608 Ops/s	679.9627 Ops/s	$\color{#d91a1a}-0.35\%$
test_gae_speed[vec_generalized_advantage_estimate-False-32-512]	0.9578ms	0.7083ms	1.4117 KOps/s	1.4211 KOps/s	$\color{#d91a1a}-0.66\%$
test_dqn_speed	7.9640ms	1.4676ms	681.3864 Ops/s	684.9125 Ops/s	$\color{#d91a1a}-0.51\%$
test_ddpg_speed	4.7597ms	3.3303ms	300.2736 Ops/s	298.5705 Ops/s	$\color{#35bf28}+0.57\%$
test_sac_speed	10.0790ms	9.2935ms	107.6023 Ops/s	108.2735 Ops/s	$\color{#d91a1a}-0.62\%$
test_redq_speed	17.5870ms	16.5784ms	60.3194 Ops/s	59.3600 Ops/s	$\color{#35bf28}+1.62\%$
test_redq_deprec_speed	14.4330ms	13.1036ms	76.3147 Ops/s	76.3024 Ops/s	$\color{#35bf28}+0.02\%$
test_td3_speed	18.4127ms	9.5057ms	105.1996 Ops/s	105.6919 Ops/s	$\color{#d91a1a}-0.47\%$
test_cql_speed	34.1241ms	32.5671ms	30.7059 Ops/s	31.1298 Ops/s	$\color{#d91a1a}-1.36\%$
test_a2c_speed	0.1071s	8.0567ms	124.1205 Ops/s	136.3454 Ops/s	$\textbf{\color{#d91a1a}-8.97\%}$
test_ppo_speed	8.8128ms	7.5893ms	131.7653 Ops/s	130.8808 Ops/s	$\color{#35bf28}+0.68\%$
test_reinforce_speed	7.7075ms	6.2932ms	158.9020 Ops/s	157.2707 Ops/s	$\color{#35bf28}+1.04\%$
test_iql_speed	28.9283ms	27.3499ms	36.5632 Ops/s	36.2667 Ops/s	$\color{#35bf28}+0.82\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	3.2172ms	2.4876ms	401.9867 Ops/s	396.2246 Ops/s	$\color{#35bf28}+1.45\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.9329ms	2.6990ms	370.5080 Ops/s	334.3932 Ops/s	$\textbf{\color{#35bf28}+10.80\%}$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.8494ms	2.6937ms	371.2401 Ops/s	371.1459 Ops/s	$\color{#35bf28}+0.03\%$
test_sample_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	2.7054ms	2.4762ms	403.8496 Ops/s	399.8230 Ops/s	$\color{#35bf28}+1.01\%$
test_sample_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	4.3842ms	2.6841ms	372.5587 Ops/s	371.2138 Ops/s	$\color{#35bf28}+0.36\%$
test_sample_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	3.6805ms	2.6823ms	372.8187 Ops/s	372.2062 Ops/s	$\color{#35bf28}+0.16\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	2.7445ms	2.5063ms	398.9937 Ops/s	396.4230 Ops/s	$\color{#35bf28}+0.65\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	4.0297ms	2.6781ms	373.3933 Ops/s	370.6536 Ops/s	$\color{#35bf28}+0.74\%$
test_sample_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	3.7911ms	2.6942ms	371.1641 Ops/s	372.3999 Ops/s	$\color{#d91a1a}-0.33\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-4000]	0.1921s	3.0060ms	332.6664 Ops/s	395.4346 Ops/s	$\textbf{\color{#d91a1a}-15.87\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-10000]	3.5578ms	2.6946ms	371.1058 Ops/s	371.6257 Ops/s	$\color{#d91a1a}-0.14\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-10000]	3.9798ms	2.7006ms	370.2944 Ops/s	373.9180 Ops/s	$\color{#d91a1a}-0.97\%$
test_iterate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-4000]	0.1917s	2.9931ms	334.1045 Ops/s	395.5867 Ops/s	$\textbf{\color{#d91a1a}-15.54\%}$
test_iterate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-10000]	3.5369ms	2.6830ms	372.7184 Ops/s	371.3249 Ops/s	$\color{#35bf28}+0.38\%$
test_iterate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-10000]	4.1961ms	2.6917ms	371.5112 Ops/s	372.9928 Ops/s	$\color{#d91a1a}-0.40\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-4000]	0.1913s	3.0077ms	332.4755 Ops/s	396.4501 Ops/s	$\textbf{\color{#d91a1a}-16.14\%}$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-10000]	4.0278ms	2.6931ms	371.3173 Ops/s	371.6405 Ops/s	$\color{#d91a1a}-0.09\%$
test_iterate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-10000]	4.0700ms	2.6782ms	373.3855 Ops/s	371.5540 Ops/s	$\color{#35bf28}+0.49\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-RandomSampler-400]	0.2323s	17.4910ms	57.1723 Ops/s	51.5188 Ops/s	$\textbf{\color{#35bf28}+10.97\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-RandomSampler-400]	0.1268s	17.6035ms	56.8070 Ops/s	56.2724 Ops/s	$\color{#35bf28}+0.95\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-RandomSampler-400]	0.1280s	17.5385ms	57.0173 Ops/s	56.5135 Ops/s	$\color{#35bf28}+0.89\%$
test_populate_rb[TensorDictReplayBuffer-ListStorage-SamplerWithoutReplacement-400]	0.1279s	17.5181ms	57.0840 Ops/s	64.6269 Ops/s	$\textbf{\color{#d91a1a}-11.67\%}$
test_populate_rb[TensorDictReplayBuffer-LazyMemmapStorage-SamplerWithoutReplacement-400]	0.1260s	17.5854ms	56.8655 Ops/s	56.3542 Ops/s	$\color{#35bf28}+0.91\%$
test_populate_rb[TensorDictReplayBuffer-LazyTensorStorage-SamplerWithoutReplacement-400]	0.1264s	15.2863ms	65.4180 Ops/s	56.5237 Ops/s	$\textbf{\color{#35bf28}+15.74\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-ListStorage-None-400]	0.1282s	17.5775ms	56.8909 Ops/s	56.1992 Ops/s	$\color{#35bf28}+1.23\%$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyMemmapStorage-None-400]	0.1260s	17.4850ms	57.1918 Ops/s	64.8546 Ops/s	$\textbf{\color{#d91a1a}-11.82\%}$
test_populate_rb[TensorDictPrioritizedReplayBuffer-LazyTensorStorage-None-400]	0.1265s	17.6209ms	56.7508 Ops/s	56.5349 Ops/s	$\color{#35bf28}+0.38\%$