COMBO performance

[mamengyiyi]

Hi, I run COMBO on v2 version d4rl MuJoCo Gym datasets and found that the performances are surprisingly poor. For example, the normalized score after training 1M step of COMBO on walker2d-medium-expert-v2 is -0.18, on halfcheetah-medium-replay-v2 is 35.98 (The results fluctuated between 30-60 in the last few steps). Did I do something wrong?

[tangbotony]

same as you

[takuseno]

We confirmed that three is an issue about rollout termination. For now, COMBO is still an experimental feature.
#101 (comment)

[typoverflow]

Hello,
any updates about this issue? With some experiments using the code from d3rlpy and https://agit.ai/Polixir/OfflineRL/src/branch/master/offlinerl/algo/modelbase/combo.py, I found that for both MOPO and COMBO, there are gaps between the actual performances and what was reported in the original paper.

I'm quite skeptical about the reproducibility of some model-based offline RL algorithms (see tianheyu927/mopo#5). The source code of COMBO is not even released (as far as I know).

[takuseno]

Sorry for the inconvenience. Currently, COMBO support is a low priority because I'm working on Transformer architecture support in nightly branch. I still believe that environmental termination is the key to fix this issue. Once I sort it out, hope I can revisit this...