Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Run some Sanity Checks for PyAF 5.0 #224

Closed
antoinecarme opened this issue Mar 1, 2023 · 5 comments
Closed

Run some Sanity Checks for PyAF 5.0 #224

antoinecarme opened this issue Mar 1, 2023 · 5 comments
Assignees
@antoinecarme
Labels
class:enhancement priority:high topic:modeling_quality topic:signal_transformation

Comments

@antoinecarme
Copy link
Owner

Following #45

Need to run some profiling checks to review the current state of PyAF 5.0.

A check for signal transformations stability and invariance is welcome.

To be included in the next PyAF release (5.0, expected on 2023-07-14).
@antoinecarme
Copy link
Owner Author

Invariance tests.

  1. Check that for the negative of a signal, PyAF produces the negative of the model given by the original signal. Not always the case, and that's normal.
  2. The same for these signals : -S, S+1, S*2. simple changes in the signal should not alter much the forecasts.
  3. Double check the forecast components: trends, cycles, AR etc.
  4. Double check the performance measures (MAPE(S+1) != MAPE(S)).

@antoinecarme
Copy link
Owner Author

For profiling, there are a lot of unnecessary precautions (np.clip(x, 0, None) for log(x), etc). These can sometimes be relaxed.

antoinecarme added a commit that referenced this issue Mar 1, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Add some transformation sta…
4c21d4f 
…bility tests.
antoinecarme added a commit that referenced this issue Mar 1, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Added a generic column for …
fc1fe7b 
…the transformed signal in the model forecasts dataframe. Kolmogorov rocks.
@antoinecarme antoinecarme self-assigned this Mar 2, 2023
@antoinecarme
Copy link
Owner Author

Computation time for long term models.

@antoinecarme
Copy link
Owner Author

Memory footprint for long term models.

This was referenced Mar 4, 2023
Closed
Closed
@antoinecarme
Copy link
Owner Author

Closing.

antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Memory profiling. Refactore…
413ab5a 
…d perf computations.
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Memory profiling. Refactore…
e3204a2 
…d perf computations.
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Memory profiling. Refactore…
c71eccb 
…d perf computations.
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Memory profiling. Refactore…
3c5046b 
…d perf computations.
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Memory profiling. Refactore…
c777fb9 
…d perf computations.
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Memory profiling. Force Gar…
0232cdf 
…bage collection. Refactored perf computations.
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Memory profiling. Force Gar…
d3ec975 
…bage collection. Refactored perf computations. Updated this notebook
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Removed a warning.
c79e002
antoinecarme added a commit that referenced this issue Mar 15, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Updated these logs.
94578c5
antoinecarme added a commit that referenced this issue Mar 25, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Refactored model selection…
7f20cd6 
…. No need to transfer the whole pytorch/xgboost models to perform model selection. xeon-phi side-effect
antoinecarme added a commit that referenced this issue Mar 25, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Refactored model selection…
af843ed 
…. No need to transfer the whole pytorch/xgboost models to perform model selection. xeon-phi side-effect
antoinecarme added a commit that referenced this issue Apr 13, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Profiling. Speed up Time c…
d0395c2 
…olumn manipulations.
antoinecarme added a commit that referenced this issue Apr 13, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Profiling. Display more de…
049dcc5 
…tailed infor about test executions (Time, Memory, CPU usage).
antoinecarme added a commit that referenced this issue Apr 13, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Profiling. Display more de…
3c723c2 
…tailed infor about test executions (Time, Memory, CPU usage).
antoinecarme added a commit that referenced this issue Apr 13, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Profiling. Display more de…
9eb5103 
…tailed infor about test executions (Time, Memory, CPU usage).
antoinecarme added a commit that referenced this issue Apr 13, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Profiling. Avoid unnecessa…
dadcecd 
…ry column order changes.
antoinecarme added a commit that referenced this issue Apr 13, 2023
@antoinecarme
Run some Sanity Checks for PyAF 5.0 #224. Profiling. Avoid unnecessa…
0dfb553 
…ry column order changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@antoinecarme antoinecarme

Labels
class:enhancement priority:high topic:modeling_quality topic:signal_transformation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@antoinecarme