-
Notifications
You must be signed in to change notification settings - Fork 108
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Fix the sample job file and execution commands #1651
Fix the sample job file and execution commands #1651
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you update the commit to include a description of why we need these changes? Thanks!
@@ -200,21 +200,21 @@ Then you construct a fio job file for your initial sequential tests, and this wi | |||
|
|||
[global] | |||
# size of each FIO file (in MB) | |||
size=$@ | |||
size=1GiB |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This seems incorrect, given the comment on line 199.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The size is not filled as well.
# do not use gettimeofday, much too expensive for KVM guests | ||
clocksource=clock_gettime | ||
# give fio workers some time to get launched on remote hosts | ||
startdelay=5 | ||
# files accessed by fio are in this directory | ||
directory=$@ | ||
directory=/mnt/fio/files |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure about this change either, see our basic job templates:
- https://github.com/distributed-system-analysis/pbench/blob/master/agent/bench-scripts/templates/fio.job
- https://github.com/distributed-system-analysis/pbench/blob/master/agent/bench-scripts/templates/fio-shared-fs.job
The fio-shared-fs.job
template uses directory
with the $target
environment variable reference that allows us to pass that from the command line.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the command line I've used, this parameter is not getting filled. Also the pdsh commands above assumes the directory path is fixed.
# write fio latency logs in /var/tmp with "fio" prefix | ||
write_lat_log=/var/tmp/fio | ||
write_lat_log=fio |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to change this file name and location without also changing the comment?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
With the absolute path provided, the /var/lib/pbench-agent/fio__2020.05.08T08.24.48/1-write-4KiB/sample1/result.json is not created.
Logs:
running fio job: /var/lib/pbench-agent/fio__2020.05.08T08.24.48/1-write-4096KiB/fio.job (sample1)
Warning: ALREADY_ENABLED: '8765:tcp' already in 'public'
Warning: ALREADY_ENABLED: '8765:tcp' already in 'public'
fio job complete
fio-postprocess: could not find any result files to process, exiting
agent/bench-scripts/pbench-fio.md
Outdated
# write histogram record once every 10 seconds | ||
log_hist_msec=10 | ||
#log_hist_msec=10 |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why do we need to comment out lines 215 and 217?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These options were throwing errors. I did not investigate further.
--output-format=json fio-sequential.job | ||
pbench-fio --client-file=vms.list --pre-iteration-script=drop-cache.sh \ | ||
-t write,read -b 4,128,1024 -d /mnt/fio/files --numjobs=32 \ | ||
--job-file fio-sequential.job |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why make all these changes when the comments at line 234 and 240 don't match the new changes?
If the original line was supposed to work with the base fio
program itself, then we should fix it. If it is intended to be run with pbench-fio
wrapper as this new change suggests, we should consider updating the text surrounding the command to reflect the intention.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
fio doesn't take the parameters --client-file, --pre-iteration-script.
And pbench-fio does not take the parameters --output-format=json, --max-jobs --rw. So as such this command will neither work for fio nor pbench-fio. I'll change the texts around it.
@karthiksundaravel, I have not forgotten about this request. We are working on getting |
The sample job file and the fio / pbench-fio commands needs change in order to be usable. The parameters size, directory were not automatically filled for the commands given. The path for write_lat_log shall not include '/var/tmp/' similar to https://github.com/distributed-system-analysis/pbench/blob/master/agent/bench-scripts/templates/fio.job
Closing for now, tracking in PR #3254 for possible future work. |
No description provided.