Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

exp show: experiment order randomly changing #9504

Closed
mattseddon opened this issue May 24, 2023 · 3 comments · Fixed by #9660
Closed

exp show: experiment order randomly changing #9504

mattseddon opened this issue May 24, 2023 · 3 comments · Fixed by #9660
Assignees
Labels
A: experiments Related to dvc exp p3-nice-to-have It should be done this or next sprint ui user interface / interaction

Comments

@mattseddon
Copy link
Member

Bug Report

Description

When calling exp show multiple times the order of experiments can change between calls (no sorts applied):

Screen.Recording.2023-05-24.at.12.29.44.pm.mov

Reproduce

  1. queue several experiments
  2. start the queue with multiple workers
  3. monitor exp show

Expected

Experiments order is static unless sorts are applied.

Environment information

Output of dvc doctor:

$ dvc doctor
DVC version: 2.58.1 (pip)
-------------------------
Platform: Python 3.10.6 on macOS-13.3.1-arm64-arm-64bit
Subprojects:
        dvc_data = 0.51.0
        dvc_objects = 0.22.0
        dvc_render = 0.5.0
        dvc_task = 0.2.1
        scmrepo = 1.0.3
Supports:
        http (aiohttp = 3.8.4, aiohttp-retry = 2.8.3),
        https (aiohttp = 3.8.4, aiohttp-retry = 2.8.3),
        s3 (s3fs = 2023.5.0, boto3 = 1.26.76)
Config:
        Global: /Users/mattseddon/Library/Application Support/dvc
        System: /Library/Application Support/dvc
Cache types: reflink, hardlink, symlink
Cache directory: apfs on /dev/disk3s1s1
Caches: local
Remotes: s3
Workspace directory: apfs on /dev/disk3s1s1
Repo: dvc, git
Repo.site_cache_dir: /Library/Caches/dvc/repo/9b2d1355ef26cb2a38a704c54d1f3710

Additional Information (if any):

Noticed while integrating #9432 with the extension (for iterative/vscode-dvc#3436 & iterative/vscode-dvc#3178)

@pmrowla pmrowla added p3-nice-to-have It should be done this or next sprint ui user interface / interaction A: experiments Related to dvc exp labels May 24, 2023
@pmrowla
Copy link
Contributor

pmrowla commented May 24, 2023

Discussed this in slack, the issue is probably due to (lack of) git commit timestamp precision. The default sort is chronological by git timestamp, there may be a way to get more consistent ordering but I don't think it needs to be prioritized

@omesser
Copy link
Contributor

omesser commented May 26, 2023

@mattseddon - tried a very simple fix in: https://github.com/omesser/dvc/tree/stable_exp_sorting
Can you please check if this fixes the issue?

@mattseddon
Copy link
Member Author

@mattseddon - tried a very simple fix in: https://github.com/omesser/dvc/tree/stable_exp_sorting Can you please check if this fixes the issue?

That does seem like it worked. Did you add the commit/name to the default order?

@omesser omesser self-assigned this Jun 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A: experiments Related to dvc exp p3-nice-to-have It should be done this or next sprint ui user interface / interaction
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants