Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Eliminate the use of temporary files in the vx_config library #2691

Closed
8 of 21 tasks
JohnHalleyGotway opened this issue Sep 25, 2023 · 1 comment · Fixed by #2692 or #2695
Closed
8 of 21 tasks

Eliminate the use of temporary files in the vx_config library #2691

JohnHalleyGotway opened this issue Sep 25, 2023 · 1 comment · Fixed by #2692 or #2695
Assignees
Labels
component: code optimization Code optimization issue priority: high High Priority requestor: NOAA/other NOAA Laboratory, not otherwise specified type: enhancement Improve something that it is currently doing

Comments

@JohnHalleyGotway
Copy link
Collaborator

Describe the Enhancement

Please see #2690 which is the parent issue derived from the dtcenter/METplus#2364 discussion.

The vx_config library is one major source of the use of temp files in MET. They are used in two ways:

  1. Each config file read() function reads the input line-by-line. Each line is passed through a utility function to recursively interpret any environment variables referenced. The output is written to a temp file and subsequently read back in.
  2. The logic for parsing thresholds is also handled in by the config file code. The read_string() function is called to do so. It writes the threshold string to be parsed to a temp file prior to calling the read() function to handle it... which uses its own temp file! So at least 2 temp files are used for each threshold string!

This issue is to leverage the std::stringstream class to eliminate the use of temporary files in the vx_config library. While the I/O difference may not be obvious when writing to the local /tmp directory, it will vastly improve performance on the lustre file system used by many HPC's, including WCOSS2 and jet.

Time Estimate

1 day

Sub-Issues

Consider breaking the enhancement down into sub-issues.

Relevant Deadlines

List relevant project deadlines here or state NONE.

Funding Source

Define the source of funding and account keys here or state NONE.

Define the Metadata

Assignee

  • Select engineer(s) or no engineer required
  • Select scientist(s) or no scientist required

Labels

  • Review default alert labels
  • Select component(s)
  • Select priority
  • Select requestor(s)

Milestone and Projects

  • Select Milestone as the next official version or Backlog of Development Ideas
  • For the next official version, select the MET-X.Y.Z Development project

Define Related Issue(s)

Consider the impact to the other METplus components.

Enhancement Checklist

See the METplus Workflow for details.

  • Complete the issue definition above, including the Time Estimate and Funding Source.
  • Fork this repository or create a branch of develop.
    Branch name: feature_<Issue Number>_<Description>
  • Complete the development and test your changes.
  • Add/update log messages for easier debugging.
  • Add/update unit tests.
  • Add/update documentation.
  • Push local changes to GitHub.
  • Submit a pull request to merge into develop.
    Pull request: feature <Issue Number> <Description>
  • Define the pull request metadata, as permissions allow.
    Select: Reviewer(s) and Development issue
    Select: Milestone as the next official version
    Select: MET-X.Y.Z Development project for development toward the next official release
  • Iterate until the reviewer(s) accept and merge your changes.
  • Delete your fork or branch.
  • Close this issue.
@JohnHalleyGotway JohnHalleyGotway added type: enhancement Improve something that it is currently doing requestor: NOAA/other NOAA Laboratory, not otherwise specified component: code optimization Code optimization issue priority: high High Priority labels Sep 25, 2023
@JohnHalleyGotway JohnHalleyGotway added this to the MET 12.0.0 milestone Sep 25, 2023
@JohnHalleyGotway JohnHalleyGotway self-assigned this Sep 25, 2023
@JohnHalleyGotway JohnHalleyGotway changed the title Eliminate the use of temporary files in the vx_config library Eliminate the use of temporary files in the vx_config library Sep 25, 2023
@JohnHalleyGotway JohnHalleyGotway moved this from 🔖 Ready to 🏗 In progress in MET-12.0.0 Development Sep 25, 2023
@JohnHalleyGotway JohnHalleyGotway moved this from 🏗 In progress to 👀 In review in MET-12.0.0 Development Sep 25, 2023
@JohnHalleyGotway JohnHalleyGotway linked a pull request Sep 27, 2023 that will close this issue
15 tasks
@JohnHalleyGotway
Copy link
Collaborator Author

Note that PR #2692 merged a fix into the develop branch for this. However, I'm going to keep this issue open since it sounds like we want to provide the same fix in the main_v11.1 branch.

@JohnHalleyGotway JohnHalleyGotway moved this from 👀 In review to ✅ Done in MET-12.0.0 Development Sep 27, 2023
@JohnHalleyGotway JohnHalleyGotway linked a pull request Sep 28, 2023 that will close this issue
15 tasks
@JohnHalleyGotway JohnHalleyGotway moved this from ✅ Done to 👀 In review in MET-12.0.0 Development Sep 28, 2023
@github-project-automation github-project-automation bot moved this from 🔖 Ready to ✅ Done in Coordinated METplus-5.1 Support Oct 2, 2023
@github-project-automation github-project-automation bot moved this from 👀 In review to ✅ Done in MET-12.0.0 Development Oct 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
component: code optimization Code optimization issue priority: high High Priority requestor: NOAA/other NOAA Laboratory, not otherwise specified type: enhancement Improve something that it is currently doing
Projects
Status: 🏁 Done
1 participant