Experiments: allow setting of list parameters

[sjawhar]

• ❗ I have followed the Contributing to DVC checklist.

• 📖 If this PR requires documentation updates, I have created a separate PR (or issue, at least) in dvc.org and linked it here.

Thank you for the contribution - we'll try to review it as soon as possible. 🙏

Will close #4768

As stated in the issue, I'm sure more work will be needed, but this little tweak was enough to at least allow the experiment to run. Some more work would be required to get parameter diffs to show up nicely in dvc exp show (screenshot of current display is included below, see the change in activation function), but that might be out of scope for this issue.

If there are no major objections, I can go ahead and write up some tests and finish this PR off proper-like.

[pmrowla]

@sjawhar thanks for identifying this issue and for the PR! This looks like a great start, and worked for me in a test project.

Don't worry about cleaning up the exp show display for this PR, we can consider how to address that in the future. For now, lists and dicts being displayed in the string [..., {..}] format is at least consistent with other DVC commands (params diff/metrics diff/etc).

For a test case, a modified version of

dvc/tests/func/experiments/test_experiments.py

Line 46 in 33ece01

def test_new_simple(tmp_dir, scm, dvc, mocker):

should work.

Your test case will just need to generate a params.yaml file containing a small list, and when running the stage it would look something like

dvc.experiments.run(stage.addressing, params=["foo.<list_index>...=modified_value"])

The test stage we use just copies a params into a metrics file, so after calling experiments.run, the final output metrics file should contain the modified_value from the run command, rather than initial params.yaml value.

[sjawhar]

OK, test added. Thanks for pointing me to the right spot, that made it super easy for me 🙏

[sjawhar]

I've been thinking about this and wondering if there's an opportunity to simplify things and delete a bunch of code by using something like https://github.com/fabiocaccamo/python-benedict. If the existing project params were instantiated as a benedict instance, then you could do stuff like params['foo.bar[0].bar'] = 1 with no fuss:

>>> from benedict import benedict
>>> params = benedict({'foo': { 'bar': [{'baz': 3}, 2]}})
>>> params['foo.bar[0].baz']=1
>>> params
{'foo': {'bar': [{'baz': 1}, 2]}}

[pmrowla]

Thanks for the PR! 🙏

The suggestion regarding benedict is interesting and we could potentially consider using something like that in the future. If you'd like to file a separate feature request for it that would be great