Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

get_more_info(): different seed but same 'train-accuracy', 'train-all-time', 'train-all-time' #54

Open
MU-PING opened this issue Mar 19, 2023 · 0 comments

Comments

@MU-PING
Copy link

MU-PING commented Mar 19, 2023

Why do different trials have same "train-accuracy", "train-per-time" and "train-all-time", it's not intuitive.

Code

print("Trial-1--------------------------------------")
print(self.api.get_more_info(index=arch, dataset='cifar10', hp='200', is_random=777), "\n")
print("Trial-2--------------------------------------")
print(self.api.get_more_info(index=arch, dataset='cifar10', hp='200', is_random=888), "\n")
print("Trial-3--------------------------------------")
print(self.api.get_more_info(index=arch, dataset='cifar10', hp='200', is_random=999), "\n")

Output

Trial-1--------------------------------------
{'train-loss': 0.0019287928102910518, 'train-accuracy': 99.986, 'train-per-time': 28.7974933385849, 'train-all-time': 5759.49866771698, 'comment': 'In this dict, train-loss/accuracy/time is the metric on the train+valid sets of CIFAR-10. The test-loss/accuracy/time is the performance of the CIFAR-10 test set after training on the train+valid sets by 200 epochs. The per-time and total-time indicate the per epoch and total time costs, respectively.', 'test-loss': 0.273262268447876, 'test-accuracy': 94.26, 'test-per-time': 1.8184241397040222, 'test-all-time': 363.68482794080444}

Trial-2--------------------------------------
{'train-loss': 0.0019176186656206847, 'train-accuracy': 99.986, 'train-per-time': 28.7974933385849, 'train-all-time': 5759.49866771698, 'comment': 'In this dict, train-loss/accuracy/time is the metric on the train+valid sets of CIFAR-10. The test-loss/accuracy/time is the performance of the CIFAR-10 test set after training on the train+valid sets by 200 epochs. The per-time and total-time indicate the per epoch and total time costs, respectively.', 'test-loss': 0.25168872060775754, 'test-accuracy': 94.68, 'test-per-time': 1.8184241397040222, 'test-all-time': 363.68482794080444}

Trial-3--------------------------------------
{'train-loss': 0.0018634130948781966, 'train-accuracy': 99.986, 'train-per-time': 28.7974933385849, 'train-all-time': 5759.49866771698, 'comment': 'In this dict, train-loss/accuracy/time is the metric on the train+valid sets of CIFAR-10. The test-loss/accuracy/time is the performance of the CIFAR-10 test set after training on the train+valid sets by 200 epochs. The per-time and total-time indicate the per epoch and total time costs, respectively.', 'test-loss': 0.26081861381530763, 'test-accuracy': 94.18, 'test-per-time': 1.8184241397040222, 'test-all-time': 363.68482794080444}
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant