Profiling workflow on workflow_dispatch
#1012
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This was referenced Jun 29, 2023
tamuri
approved these changes
Aug 14, 2023
No problem - I've swapped it to |
- .gitignore will ignore profiling outputs - dev.in and dev.txt list pyinstrument as a requirement - MANIFEST.in ignores the profiling/ directory
willGraham01
force-pushed
the
wgraham/pyinstrument-profiling-ci
branch
from
921fdb5 to
71c6ea5
Compare
|
@tamuri This OK to be merged in? (I don't have merge authority to |
|
Yes, fine to merge - I've given you permissions. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Short Description
Provides the "skeleton" for implementation of profile tracking of runs across the repository history (#686).
Adds the
run_profiling.ymlworkflow which profiles thescale_run.pyscript and exports the resulting.pyisessionfile to theTLOmodel-profilingrepository - repo is under the UCL organisation so everyone should be able to see it, but I can fix that if this isn't the case. Workflow runs on self-hosted runners and uses atoxenvironment to run the profiling script.Preview the profiling site here
Full Changes
Adds
.github/workflows/run_profiling.ymlworkflow.year/month/day/HHMM/{files.out}on theTLOmodel-profilingrepository. Deployment targets theresultsbranch (so that in future we can trigger rendering onpushes to this branch).src/scripts/profilingdirectory:scale_run.pyhas the "main" refactored so that it can be imported from the profiling script_paths.pyfile to fix some important paths to avoid filesystem errors_parameters.pyfile to reliably the track the parameters that should be passed to the models that we are profiling. NOTE: For the purposes of testing new features, the profiling only runs a month-long simulation. This can be edited once we're happy with the workflow and how the outputs are presented/moved to the output repo.run_profiling.py; new script that can be run from the command-line. Profiles thescale_runmodel and writes the outputs to either/both HTML and JSON.vscode's linter has also zealously run over a couple of other files likeshared.py, which is bloating the diff.