Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Avoid recompiling executable if already compiled in previous tests #69

Open
wants to merge 6 commits into
base: main
Choose a base branch
from

Conversation

RemiLehe
Copy link
Contributor

@RemiLehe RemiLehe commented Jul 27, 2020

Preliminary note

This PR re-opens PR #68, and incorporates several modifications:

  • an option was added (avoid_recompiling, deactivated by default) in order to enable or disable the feature implemented in this PR.
  • the code does not assume anymore that a set of compilation options results in a unique executable name.
    The description below (which comes from PR#68) has been edited accordingly.

Overview

By default, when performing several tests with regtest.py, the executable is recompiled from scratch for every test. The compilation time can sometimes dominate the time taken by the tests.

This PR proposes the avoid re-compiling the executable, if it was already compiled with the exact same options in a previous test. We are currently using this feature in order to same time in the Travis CI tests for WarpX.

In practice, this is done by:

  • Creating a new folder PreviouslyCompiled, where compiled executable are stored in unique sub-directory (unique to a given a set of compilation options).
  • When running a new tests, comparing compiling options to those used in previous tests. If they match, the previous executable is retrieved from the corresponding unique sub-forder, in the PreviouslyCompiled folder.

In order to obtain the string that contains compiling options, parts of the function build_c was extracted as a separate function. In addition, for convenience, the name of the executable is now stored as an attribute in the Test class. (This allows the name of executable to be easily retrieved in the case where the executable is not recompiled from scratch.)

This is done by:
- Storing compiled executable into a new "Build" folder
- When running a new tests, comparing compiling options to those used
in previous tests; it they match, the previous executable is retrieved
from the Build folder.

Note: this assumes that each executable name is unique ; i.e. that
two sets of compiling options cannot produce the same executable name.

In order to obtain the string that contains compiling options, part
of the function `build_c` was extracted as a separate function
@RemiLehe RemiLehe changed the title [WIP] Avoid recompiling executable if already compiled in previous tests Avoid recompiling executable if already compiled in previous tests Jul 27, 2020
Copy link
Member

@atmyers atmyers left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good to me - @zingale what do you think?

@zingale
Copy link
Member

zingale commented Feb 22, 2021

I'll test this now

@zingale
Copy link
Member

zingale commented Feb 23, 2021

our suite fails if we try to use the new option:
http://groot.astro.sunysb.edu/Microphysics/test-suite/gfortran/2021-02-22-003/index.html

@ax3l
Copy link
Member

ax3l commented Sep 7, 2021

Re: hackathon - this feature is probably the most central one for why we forked. We would love to use mainline again if we can figure this out in a hackathon :)

@zingale
Copy link
Member

zingale commented Sep 7, 2021

agreed, we need to find a way to do this without breaking other codes (e.g. Castro)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants