Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix the sample job file and execution commands #1651

Conversation

karthiksundaravel
Copy link

No description provided.

Copy link
Member

@portante portante left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you update the commit to include a description of why we need these changes? Thanks!

@@ -200,21 +200,21 @@ Then you construct a fio job file for your initial sequential tests, and this wi

[global]
# size of each FIO file (in MB)
size=$@
size=1GiB
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems incorrect, given the comment on line 199.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The size is not filled as well.

# do not use gettimeofday, much too expensive for KVM guests
clocksource=clock_gettime
# give fio workers some time to get launched on remote hosts
startdelay=5
# files accessed by fio are in this directory
directory=$@
directory=/mnt/fio/files
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure about this change either, see our basic job templates:

The fio-shared-fs.job template uses directory with the $target environment variable reference that allows us to pass that from the command line.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the command line I've used, this parameter is not getting filled. Also the pdsh commands above assumes the directory path is fixed.

# write fio latency logs in /var/tmp with "fio" prefix
write_lat_log=/var/tmp/fio
write_lat_log=fio
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to change this file name and location without also changing the comment?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the absolute path provided, the /var/lib/pbench-agent/fio__2020.05.08T08.24.48/1-write-4KiB/sample1/result.json is not created.
Logs:
running fio job: /var/lib/pbench-agent/fio__2020.05.08T08.24.48/1-write-4096KiB/fio.job (sample1)
Warning: ALREADY_ENABLED: '8765:tcp' already in 'public'
Warning: ALREADY_ENABLED: '8765:tcp' already in 'public'
fio job complete
fio-postprocess: could not find any result files to process, exiting

# write histogram record once every 10 seconds
log_hist_msec=10
#log_hist_msec=10
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need to comment out lines 215 and 217?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These options were throwing errors. I did not investigate further.

--output-format=json fio-sequential.job
pbench-fio --client-file=vms.list --pre-iteration-script=drop-cache.sh \
-t write,read -b 4,128,1024 -d /mnt/fio/files --numjobs=32 \
--job-file fio-sequential.job
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why make all these changes when the comments at line 234 and 240 don't match the new changes?

If the original line was supposed to work with the base fio program itself, then we should fix it. If it is intended to be run with pbench-fio wrapper as this new change suggests, we should consider updating the text surrounding the command to reflect the intention.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fio doesn't take the parameters --client-file, --pre-iteration-script.
And pbench-fio does not take the parameters --output-format=json, --max-jobs --rw. So as such this command will neither work for fio nor pbench-fio. I'll change the texts around it.

@portante portante added Agent Documentation enhancement fio pbench-fio benchmark related labels May 8, 2020
@portante
Copy link
Member

@karthiksundaravel, I have not forgotten about this request. We are working on getting master deployed to our staging and production nodes, so reviewing this is in the queue, but probably won't get attention until Friday or Monday next week. Thanks for you patience!

@portante portante added this to the v0.71 milestone Aug 15, 2020
@portante portante self-assigned this Oct 1, 2020
@portante portante modified the milestones: v0.71, v0.72 Nov 21, 2020
@portante portante changed the base branch from master to main January 28, 2021 22:11
@portante portante marked this pull request as draft January 28, 2021 22:14
@portante portante modified the milestones: v0.72, v0.73 Sep 14, 2022
The sample job file and the fio / pbench-fio commands needs change in order to be usable.
The parameters size, directory were not automatically filled for the commands given.
The path for write_lat_log shall not include '/var/tmp/' similar to https://github.com/distributed-system-analysis/pbench/blob/master/agent/bench-scripts/templates/fio.job
@portante
Copy link
Member

Closing for now, tracking in PR #3254 for possible future work.

@portante portante closed this Feb 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

2 participants