Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Port the old apsim SoilTemp2 model #4080

Open
hol430 opened this issue Aug 12, 2019 · 33 comments · May be fixed by #9197
Open

Port the old apsim SoilTemp2 model #4080

hol430 opened this issue Aug 12, 2019 · 33 comments · May be fixed by #9197

Comments

@hol430
Copy link
Contributor

hol430 commented Aug 12, 2019

SoilTemp2 is a numerical solution of soil temperature changes. It is a port of the original model in Classic and is based on the solution in Campbell's "Soil Physics with Basic". There have been various reports that this model performed better than the older CERES soil temperature model. The proposal is that once SoilTemp2 is ported, the CERES model would be removed.

@sarahcleary
Copy link
Contributor

Comment from @yashvirchauhan: I have been using Soiltemp in APSIM classic for quite a while. Tthe major difference in soil temperature simulated by Soiltemp and that simulated by soilN module is when soil is covered by the crop canopy. Soil N based simulated temperature is much higher (>2-5oC) indicating it is not able correctly simulate soil temp when radiation reaching soil is reduced - if I correctly remember. Soil temperature is used in predicting aflatoxin risk in peanuts and the values predicted by soil temp were more realistic.

@sarahcleary
Copy link
Contributor

@sarchontoulis to provide Validation data

@HamishBrownPFR
Copy link
Contributor

@hol353 I have some data that I would be keen to have used in the soil validation and I think will be very usefull.

  1. Soil temperature at 2, 15, 30, 90, 120, 150 and 180 cm, heat flux at 15cm, and crop radiation interception over entire crop duration's under barley, wheat, oats, fodder beet and potato with 6 treatments (and 4 reps for each crop) giving differences in canopy cover patterns
  2. Soil temperature at 7.5, 22.5, 37.5 and 52.5 cm in lysimeters with 0, 30 and 50% stones in the top soil layer in factorial combination with gravels at 30 or 60 cm.

These two sets should provide a wide range of soil temperature profiles and fluxes

  1. Soil temperature at 1.5 cm depth under wheat and barley crops for a range of sowing dates. It is going to be important to have the surface temperature well validated for soil temp so we can look to use it for driving early crop development.

I am happy to pull datasets and simulations together if you want to include them

@hol353
Copy link
Contributor

hol353 commented Nov 12, 2019

Yep that would be great. The more validation data we have the better.

@sno036
Copy link
Contributor

sno036 commented Nov 12, 2019

Sounds good to me. The WWEPP validation is looking pretty good at the moment.

@zur003
Copy link

zur003 commented May 20, 2020

@sno036 - Are you actively working on this? I may have a bit of time in the next few weeks to have a go with it, if it's a priority.

@hol353
Copy link
Contributor

hol353 commented May 20, 2020

No one at AgResearch is working on SoilTemp at the moment. Yes, it is a priority. It seems to be working - just needs some tests and validation data.

@zur003
Copy link

zur003 commented May 20, 2020

OK. I was thinking along the lines of also refactoring the code so it fits a bit better with ApsimX conventions. It currently reflects a bit of its Fortran->VB->C# history in the variable naming and handling of arrays. Or should that be retained a while longer for comparison purposes?

@hol430
Copy link
Contributor Author

hol430 commented May 20, 2020

I think the only reason we haven't refactored it already is that we wanted to wait for tests, to be certain that we didn't accidentally change its behaviour while making the cosmetic changes.

@hol353
Copy link
Contributor

hol353 commented May 20, 2020

I think we refactor SoilTemp2. We are going to validate it anyway. Go for it!

@zur003
Copy link

zur003 commented May 20, 2020

@hol353 - A quick questions about the model: One of the "outputs" is named Thr_profile, but I'm a bit unclear what it is, and would like to provide a better name. It appears to be the soil temperature profile at 5:00 A.M., but I can find no documentation or comments to verify that. Another "output" property provides daily minimum temperatures through the profile, so is this thing needed (or used)?

@hol353
Copy link
Contributor

hol353 commented May 20, 2020

I've got no idea. @sno036 ?

@zur003
Copy link

zur003 commented May 24, 2020

@hol353 @sno036 - A couple of the grower groups in the Monaro and NSW Southern Highlands have established a network of soil moisture probes (see https://www.soilmoistureprobes.com.au/ for a list of locations. I have access to the full set of data, which the groups regard as public, as funding for the probes came from federal grants. The data include soil temperature records for many of the sites. Should I look into using these data for validation of soil water and temperature models? Note that these are pasture sites, not cropping sites. Probe depths are normally 10, 20, 40, 60, 80, 100 cm.

@zur003
Copy link

zur003 commented May 25, 2020

The SoilTemperature model in the Prototypes folder currently doesn't work as expected because the Report model is configured with reporting variables like:

[Soil].SoilTemperature.AverageSoilTemp[1] as SoilTemperature@1
[Soil].SoilTemperature.AverageSoilTemp[2] as SoilTemperature@25
[Soil].SoilTemperature.AverageSoilTemp[3] as SoilTemperature@75

The "@" character, however, is not parsed as part of the output alias. Would there be any objections if I allowed the"@" character to appear in alias names? Is this particular case, it seems to be a useful notation.

@hol353
Copy link
Contributor

hol353 commented May 25, 2020

No objection - go for it.

@sno036
Copy link
Contributor

sno036 commented May 25, 2020

Thanks for this and for the refactoring. I don't think I'll be back to the doco and validation for a while. The first (and perhaps primary) validation I was intending was a comparison against the analytical solution.

@zur003 - if you pull from my SoilTemperature branch you will see "soil temp.docx" which contains the original documentation.

See also the analytical solution from @rcichota in this zip file.
SoilTemperature-AnalyticalSolution.zip

I started working on this using an old publication of his and ran into difficulties. Rogerio's reply was as below but I've not been able to get back to this since then (end of March):

Hi Val,
I had a good look at the formulas, they all seem right, but as we suspected the devil was in the details. In a nutshell rounding errors. The actual explanation is way more complex (or there is a collection of small ones).
One thing that I believe makes the problem much bigger is the format of the equations, which use frequency and phase constant instead of periods and lags. Once I switch the way to look at the functions I realise that I several small errors compounded to make those weird graphs. Because of the small numbers it doesn’t seem to make much of a difference if the frequency is 0.0000727 or 0.000072722… but it makes all the difference!
While at it, I re-wrote the equations to use period and lag, they are in the word doc attached and both functions with the parameters better explained (I hope) in the excel spreadsheet.
Hope this helps, and happy to run through this and show the effect of rounding in the calculations.
Cheers

@zur003
Copy link

zur003 commented May 26, 2020

@sno036 - I'm not sure where to find your "SoilTemperature" Branch. Is that that SoilTemperatureValidation branch of this repository? If so, I don't see a "soil temp.docx" file, although I do see an encrypted file "ObservedFACTS.7z" which might contain it.

@sno036
Copy link
Contributor

sno036 commented May 26, 2020

@zur003 - the branch is at https://github.com/sno036/ApsimX/tree/SoilTemperature/Prototypes/SoilTemperature

Here is the document - if GitHob lets be add it!
soil temp.docx

Here is the zip file (which I had not added to the branch I see)
[SoilTemperature-AnalyticalSolution.zip]
(https://github.com/APSIMInitiative/ApsimX/files/4679904/SoilTemperature-AnalyticalSolution.zip)

@zur003
Copy link

zur003 commented Jun 5, 2020 via email

@sno036
Copy link
Contributor

sno036 commented Jun 5, 2020

Hi Val,
That all makes sense, but I do have a question for you. What is the relationship of SoilTemp and
SoilTemp2 in ASPIMClassic? At first glance, SoilTemp2 appears to be a VB port of SoilTemp’s
Fortran, but the two models give rather different results. Is this by design?
Cheers,
Eric Zurcher

HI Eric,
The impetus for porting from the Fortran was that we wanted to add more detail to the boundary conditions (a atmospheric stability correction). John Hargreaves came to NZ for a stint and did the port. He first did a straight translation and the VB and Fortran versions have the same results. Then the stability corrections were added and they did change the results.. Hope this helps.

@zur003
Copy link

zur003 commented Jun 5, 2020 via email

@sno036
Copy link
Contributor

sno036 commented Aug 4, 2020

@sarahcleary - to add as the top comment. SoilTemp2 is a numerical solution of soil temperature changes. It is a port of the original model in Classic and is based on the solution in Campbell's "Soil Physics with Basic". There have been various reports that this model performed better than the older CERES soil temperature model. The proposal is that once SoilTemp2 is ported, the CERES model would be removed.

@hol353
Copy link
Contributor

hol353 commented Aug 25, 2020

I have created an 'Experiment' in SoilTemperature.apsimx that is the beginning of a test of the model against the analytical solution in the spreadsheet (which I've committed) as specified by @sno036 above.

  • The simulation reads dummy weather data from AnalyticalWeather.xlsx.

  • The soil physical node has a uniform profile specified.

  • The water balance component has been replaced by a manager script (DummyWaterBalance) that ensures water content (a user specified parameter) does not change during the simulation.

  • A 'LayerStructure' component can be added to the soil to get 1cm layers. There is a 'LayerStructure' component in the Wagga simulation that can be copied/pasted.

@sno036
Copy link
Contributor

sno036 commented Jun 6, 2022

@peter-devoil - this models needs SWIM first. Getting closer on that - Sep/Oct probably and then can think about this.

@sno036
Copy link
Contributor

sno036 commented Oct 3, 2022

Note #7417 about initialisation needs

@Chloe0317
Copy link
Contributor

@sno036 @hol353 @HamishBrownPFR When reviewing the soil temperature model, I suggested that we could extract soil texture information from the currently available digital soil mapping products, rather than using a default texture for all simulations. Attached is an R script I have developed to extract soil sand, silt, and clay content within Australia from the latest Soil and Landscape Grid of Australia DSM product, and from ISRIC SoilGrids for other parts of the world. The script reports these values in the native mapped depth intervals of the GlobalSoilMap project (0-5, 5-15, 15-30, 30-60, 60-100, and 100-200 cm).

Although it may be too labour intensive to incorporate this into APSIM Next Gen at the moment, I believe this code could be useful for obtaining a more realistic initialisation of soil texture, and consequently, the soil temperature model. I am sharing it here for anyone who might find it helpful, though not sure how widely this can reach.

I'm also wondering if you will see value e.g., to link this in the documentation for the time being so that if users would like to use the code to initialise their soil texture they can easily find it.

https://github.com/Chloe0317/dsm/blob/main/SoilTextureExtraction.R

@hol353
Copy link
Contributor

hol353 commented Aug 4, 2024

Thanks for this @Chloe0317. Yes, there is merit in your suggestion. There are quite a few data sources that could be used to in-fill a lot of soil parameters, not just particle size information. Can you create a separate GitHub Issue for this because your suggestion probably shouldn't hold up the release of the Soil Temperature model and we don't want to lose your idea?

@Chloe0317
Copy link
Contributor

Thanks @hol353, I'll open a new issue as I agree that this should not affect the release of the soil temperature model and rather, serve as a complementary tool and can be applied more widely for other parameters.

@yashvirchauhan
Copy link

yashvirchauhan commented Aug 5, 2024 via email

@Chloe0317
Copy link
Contributor

Hi Yash,

I have started a new issue as per Dean's suggestion as there are potentially more applications for it. I agree using mapped soil parameters can be another source of error, though they will probably still provide more value when certain attributes are un-measured and present a quantitative way of model initialisation. The mapped products have respective uncertainty assessments such as prediction confidence intervals that can be accessed and for SoilGrids the actual soil profile datasets used to produce the maps can also be accessed from the ISRIC portal. One can thus account for such uncertainty and their propagation in the modelling process through statistical means. Other mapped soil parameters available include e.g., DUL and CLL, newly mapped in 2021. I think their prediction accuracies were quite reasonable and will be useful for regional/national scale modelling.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Under Development
Development

Successfully merging a pull request may close this issue.

10 participants