Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Modularised and tested MATLAB code (ustar_cp) #35

Draft
wants to merge 80 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
80 commits
Select commit Hold shift + click to select a range
7fbbc04
Added us_arc_sample data by upload
j-emberton Aug 16, 2024
6238426
First definition of contest.py with fixtures
j-emberton Aug 16, 2024
2a333f7
multiple changes
j-emberton Aug 16, 2024
6f2f925
added test case files
j-emberton Aug 16, 2024
ac16932
requested fixes plus setup matlab refactor files
j-emberton Aug 20, 2024
0dfa92f
first regression and unit tests for launch.m
j-emberton Aug 22, 2024
726dfe4
improved unit tests docstrings
j-emberton Aug 22, 2024
98227d9
removed verbose tag from pytest output as unnecessary
j-emberton Aug 22, 2024
750086f
updated project dependencies and added pytest cov
j-emberton Aug 22, 2024
181e45e
Changed test artifact folder structure. Reviewed how test artifact di…
j-emberton Aug 22, 2024
937e71c
added test data
j-emberton Aug 22, 2024
bbe0b0c
Merge pull request #20 from Cambridge-ICCS/sample_data_test_infrastru…
dorchard Aug 22, 2024
4040859
first regression and unit tests for launch.m
j-emberton Aug 22, 2024
c4e5e06
improved unit tests docstrings
j-emberton Aug 22, 2024
fd5f9fd
removed verbose tag from pytest output as unnecessary
j-emberton Aug 22, 2024
f6e5ca9
updated project dependencies and added pytest cov
j-emberton Aug 22, 2024
a5989a6
Changed test artifact folder structure. Reviewed how test artifact di…
j-emberton Aug 22, 2024
e0d1212
added test readme
j-emberton Aug 22, 2024
24c15c5
Add large files with Git LFS
j-emberton Aug 22, 2024
4b02f3e
Update tests/conftest.py
j-emberton Sep 9, 2024
32ea5af
extra unit test on keyword behaviour
dorchard Sep 9, 2024
bb9a11d
Update tests/unit_tests/test_ustar_cp/test_launch.py
dorchard Sep 9, 2024
4a8c365
Update tests/conftest.py
j-emberton Sep 9, 2024
2c2411b
Updated missing file test comment on return value
j-emberton Sep 9, 2024
edd4068
Added check for declaration of 0 files found
j-emberton Sep 9, 2024
d3b32b8
Merge pull request #25 from Cambridge-ICCS/further_test_launch.m
dorchard Sep 9, 2024
d21ed8c
added assertion for std out when empty input
j-emberton Sep 9, 2024
4560550
adjusted docstring
j-emberton Sep 9, 2024
df4f694
removed unnecessary cases
j-emberton Sep 10, 2024
32e4a7b
updated list of regression test cases
j-emberton Sep 10, 2024
ab3c0e3
updated list of regression test cases
j-emberton Sep 10, 2024
27d2514
remove lfs
j-emberton Sep 10, 2024
3430224
added search for 'report'
j-emberton Sep 10, 2024
49c30e3
re added US_ARc after accidental deletion
j-emberton Sep 10, 2024
917cd4e
added check for output file contents
j-emberton Sep 10, 2024
fb86c87
merged in refactor_and_test_launch
j-emberton Sep 10, 2024
45f69bc
fix conflict
j-emberton Sep 10, 2024
393da43
added test case selction logic summary
j-emberton Sep 10, 2024
9786955
Updated folder comparison test failure message
j-emberton Sep 10, 2024
79b02e9
Updated folder comparison test failure message
j-emberton Sep 10, 2024
43ce656
Better explanation of site ID
j-emberton Sep 10, 2024
2296433
Removed surplus sites
j-emberton Sep 10, 2024
d617ec1
Merge pull request #22 from Cambridge-ICCS/Add_test_cases
j-emberton Sep 10, 2024
84e06aa
Account for new lines in txt strip
j-emberton Sep 10, 2024
e4b3608
Merge pull request #21 from Cambridge-ICCS/refactor_and_test_launch
j-emberton Sep 10, 2024
52a51cd
Create test_cpdBootstrapUStarTh4Season20100901
j-emberton Sep 11, 2024
bb73328
Added chatGPT generated tests
j-emberton Sep 11, 2024
fd76cfa
convert mock_data to a fixture
tztsai Sep 11, 2024
ef90782
convert mock_data to a fixture
tztsai Sep 11, 2024
69d385f
added first test artifacts
j-emberton Sep 16, 2024
eb60a12
Merge branch '29-generate-unit-tests-for-cpdbootstrap' of github.com:…
j-emberton Sep 16, 2024
2cf07e1
first pass at new function etst based on testcases
j-emberton Sep 18, 2024
c11d2fd
Add test_cpdEvaluateUStarTh4Season and test_cpdFindChangePoint
tztsai Sep 20, 2024
d5953ea
add t.csv
tztsai Sep 23, 2024
3c5a9ed
fix bugs including unsupported nonscalar struct output, invalid paths…
tztsai Sep 23, 2024
5807f22
Fix bugs: read_csv, CpdBootstrap testcases, cpd values
tztsai Sep 25, 2024
da7a385
remove matlab_engine and migrate changes from ustar_cp to ustar_cp_re…
tztsai Sep 26, 2024
d55aff0
fixes
j-emberton Sep 29, 2024
1f09a26
moved fixtures and functions into conftest
j-emberton Oct 10, 2024
0f0686f
tidied up tests
j-emberton Oct 11, 2024
3bbca66
initial modularisation
j-emberton Oct 14, 2024
ad86606
deleted surplus code
j-emberton Oct 14, 2024
86931f8
remove test_context.py and append unit test to context.py
tztsai Oct 17, 2024
00420eb
fix setup_Stats
tztsai Oct 17, 2024
21015b5
fix setup_Stats input args
tztsai Oct 17, 2024
53ecf9e
Update tests/unit_tests/test_ustar_cp/test_cpdBootstrapUStarTh4Season…
tztsai Oct 17, 2024
7e13346
remove additional arg in cpdBoostrap in launch.m
tztsai Oct 17, 2024
1efa4c1
add pytest-xdist to requirements
dorchard Oct 17, 2024
3aad58b
Merge pull request #30 from Cambridge-ICCS/29-generate-unit-tests-for…
dorchard Oct 17, 2024
8fd8806
Merge branch '37-modularise-cpdBoostrap' of github.com:Cambridge-ICCS…
j-emberton Oct 17, 2024
5964dab
updated setup stats
j-emberton Oct 17, 2024
d259198
simplified calulation of nt
j-emberton Oct 21, 2024
634cb41
removed redundant tests
j-emberton Oct 21, 2024
dacfc1c
reworked setup_stats and tests
j-emberton Oct 21, 2024
65aaafb
reworked array comparison helper function
j-emberton Oct 21, 2024
4d83e33
Change matlab_engine scope to 'session'
isaacaka Oct 22, 2024
69afcbb
Merge pull request #41 from Cambridge-ICCS/change_matlab_engine_scope
dorchard Oct 24, 2024
44bfb80
setup stats reset to original code logic
j-emberton Oct 28, 2024
1e4f989
Merge pull request #38 from Cambridge-ICCS/37-modularise-cpdBoostrap
dorchard Oct 30, 2024
0d546dd
Merge pull request #43 from Cambridge-ICCS/29-generate-unit-tests-for…
j-emberton Nov 5, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@

2 changes: 1 addition & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ tramp


# csvs
*.csv
#*.csv

# mac files
*.DS_Store
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -143,4 +143,3 @@
end; % for iBoot

end; % if ntNee>=ntN;

2 changes: 1 addition & 1 deletion oneflux_steps/ustar_cp/cpdEvaluateUStarTh4Season20100901.m
Original file line number Diff line number Diff line change
Expand Up @@ -178,7 +178,7 @@
end; % for iStrata

end; % for iSeason

% ========================================================================
% ========================================================================

Expand Down
Binary file not shown.
6 changes: 6 additions & 0 deletions oneflux_steps/ustar_cp_refactor_wip/bin/MCR.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
http://www.mathworks.com/products/compiler/mcr/

$ export MCR_HOME="/home/gilberto/bin/MATLAB_Compiler_Runtime/v717" && export MCR_JRE="/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/sys/java/jre/glnxa64/jre/lib/amd64" && export LD_LIBRARY_PATH="/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/runtime/glnxa64:/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/bin/glnxa64:/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/sys/os/glnxa64:/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/sys/java/jre/glnxa64/jre/lib/amd64/server:/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/sys/java/jre/glnxa64/jre/lib/amd64/client:/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/sys/java/jre/glnxa64/jre/lib/amd64"

$ export XAPPLRESDIR=/home/gilberto/bin/MATLAB_Compiler_Runtime/v717/X11/app-defaults

2 changes: 2 additions & 0 deletions oneflux_steps/ustar_cp_refactor_wip/bin/README_MCR.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
Download MCR (latest version used for USTAR_CP: 2018a)
http://www.mathworks.com/products/compiler/mcr/
Binary file added oneflux_steps/ustar_cp_refactor_wip/bin/ustar_cp
Binary file not shown.
Binary file not shown.
Binary file not shown.
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
launch Executable

1. Prerequisites for Deployment

Verify that version 9.4 (R2018a) of the MATLAB Runtime is installed.
If not, you can run the MATLAB Runtime installer.
To find its location, enter

>>mcrinstaller

at the MATLAB prompt.

Alternatively, download and install the Linux version of the MATLAB Runtime for R2018a
from the following link on the MathWorks website:

http://www.mathworks.com/products/compiler/mcr/index.html

For more information about the MATLAB Runtime and the MATLAB Runtime installer, see
Package and Distribute in the MATLAB Compiler documentation
in the MathWorks Documentation Center.

2. Files to Deploy and Package

Files to Package for Standalone
================================
-launch
-run_launch.sh (shell script for temporarily setting environment variables and executing
the application)
-to run the shell script, type

./run_launch.sh <mcr_directory> <argument_list>

at Linux or Mac command prompt. <mcr_directory> is the directory
where version 9.4 of the MATLAB Runtime is installed or the directory where
MATLAB is installed on the machine. <argument_list> is all the
arguments you want to pass to your application. For example,

If you have version 9.4 of the MATLAB Runtime installed in
/mathworks/home/application/v94, run the shell script as:

./run_launch.sh /mathworks/home/application/v94

If you have MATLAB installed in /mathworks/devel/application/matlab,
run the shell script as:

./run_launch.sh /mathworks/devel/application/matlab
-MCRInstaller.zip
Note: if end users are unable to download the MATLAB Runtime using the
instructions in the previous section, include it when building your
component by clicking the "Runtime included in package" link in the
Deployment Tool.
-This readme file



3. Definitions

For information on deployment terminology, go to
http://www.mathworks.com/help and select MATLAB Compiler >
Getting Started > About Application Deployment >
Deployment Product Terms in the MathWorks Documentation
Center.

4. Appendix

A. Linux systems:
In the following directions, replace MR/v94 by the directory on the target machine where
MATLAB is installed, or MR by the directory where the MATLAB Runtime is installed.

(1) Set the environment variable XAPPLRESDIR to this value:

MR/v94/X11/app-defaults


(2) If the environment variable LD_LIBRARY_PATH is undefined, set it to the following:

MR/v94/runtime/glnxa64:MR/v94/bin/glnxa64:MR/v94/sys/os/glnxa64:MR/v94/sys/opengl/lib/glnxa64

If it is defined, set it to the following:

${LD_LIBRARY_PATH}:MR/v94/runtime/glnxa64:MR/v94/bin/glnxa64:MR/v94/sys/os/glnxa64:MR/v94/sys/opengl/lib/glnxa64

For more detailed information about setting the MATLAB Runtime paths, see Package and
Distribute in the MATLAB Compiler documentation in the MathWorks Documentation Center.



NOTE: To make these changes persistent after logout on Linux
or Mac machines, modify the .cshrc file to include this
setenv command.
NOTE: The environment variable syntax utilizes forward
slashes (/), delimited by colons (:).
NOTE: When deploying standalone applications, you can
run the shell script file run_launch.sh
instead of setting environment variables. See
section 2 "Files to Deploy and Package".






Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
35000 35010
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/sh
# script for execution of deployed applications
#
# Sets up the MATLAB Runtime environment for the current $ARCH and executes
# the specified command.
#
exe_name=$0
exe_dir=`dirname "$0"`
echo "------------------------------------------"
if [ "x$1" = "x" ]; then
echo Usage:
echo $0 \<deployedMCRroot\> args
else
echo Setting up environment variables
MCRROOT="$1"
echo ---
LD_LIBRARY_PATH=.:${MCRROOT}/runtime/glnxa64 ;
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/bin/glnxa64 ;
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/os/glnxa64;
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/opengl/lib/glnxa64;
export LD_LIBRARY_PATH;
echo LD_LIBRARY_PATH is ${LD_LIBRARY_PATH};
shift 1
args=
while [ $# -gt 0 ]; do
token=$1
args="${args} \"${token}\""
shift
done
eval "\"${exe_dir}/ustar_cp_linux64_2018a\"" $args
fi
exit

Binary file not shown.
Binary file not shown.
33 changes: 33 additions & 0 deletions oneflux_steps/ustar_cp_refactor_wip/bin/ustar_cp_run.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
#!/bin/sh
# script for execution of deployed applications
#
# Sets up the MATLAB Runtime environment for the current $ARCH and executes
# the specified command.
#
exe_name=$0
exe_dir=`dirname "$0"`
echo "------------------------------------------"
if [ "x$1" = "x" ]; then
echo Usage:
echo $0 \<deployedMCRroot\> args
else
echo Setting up environment variables
MCRROOT="$1"
echo ---
LD_LIBRARY_PATH=.:${MCRROOT}/runtime/glnxa64 ;
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/bin/glnxa64 ;
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/os/glnxa64;
LD_LIBRARY_PATH=${LD_LIBRARY_PATH}:${MCRROOT}/sys/opengl/lib/glnxa64;
export LD_LIBRARY_PATH;
echo LD_LIBRARY_PATH is ${LD_LIBRARY_PATH};
shift 1
args=
while [ $# -gt 0 ]; do
token=$1
args="${args} \"${token}\""
shift
done
eval "\"${exe_dir}/ustar_cp\"" $args
fi
exit

Binary file not shown.
Loading