-
Notifications
You must be signed in to change notification settings - Fork 3
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Update scripts for GEBCO 2024 #265
Conversation
recipes/earth_gebco.recipe
Outdated
# SRC_TITLE=GEBCO_Earth_Relief_2023 | ||
# SRC_REF="GEBCO_Compilation_Group,_2023_GEBCO_2023_Grid" | ||
# SRC_DOI="https://doi.org/10.5285/f98b053b-0cbc-6c23-e053-6c86abc0af7b" | ||
# SRC_FILE=ftp://ftp.soest.hawaii.edu/pwessel/GEBCO/GEBCO_2024.nc |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This can't be right anymore
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I will revise the recipe when I finished processing it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I am not sure what to do with that.
I understand that Paul put the grid there for a backup. Maybe we could use another place to put the data (like /export/gmtserver/gmt
) or not save the data anywhere and use the original link (https://www.bodc.ac.uk/data/open_download/gebco/gebco_2024/zip/)
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We should use the original link if possible.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree. The file from the original link is a zip. So, maybe this is an issue? Should we:
- Use the original file (and at to the script a command to unzip it).
- Save the unzip file in another directory in the server (as Paul did).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, but we should add the file to Paul's FTP but I don't know if we can. I think not.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I thought it was already on Paul's FTP. If not, then I think we can't upload anything to it.
Can SRC_FILE
be a local path?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I usually get the original file, unzip it and move it to the staging directory.
I think that the SRC_FILE
only is needed is the file doesn't exist. I think this is the part of the script that do that:
# 5. Determine if this source is an URL and if we need to download it first
is_url=$(echo ${SRC_FILE} | grep -c :)
if [ $is_url ]; then # Data source is an URL
if [ ! -f ${SRC_BASENAME} ]; then # Must download first
echo "srv_downsampler_grid.sh: Must download original source ${SRC_FILE}"
curl -k ${SRC_FILE} --output ${SRC_BASENAME}
fi
SRC_ORIG=${SRC_FILE}
SRC_FILE=${SRC_BASENAME}
fi
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Maybe we could use the zip file directly and unzip it as we do with the emag2 recipe.
# SRC_FILE=https://www.ngdc.noaa.gov/geomag/data/EMAG2/EMAG2_V3_20170530/EMAG2_V3_20170530.zip
# SRC_PROCESS="rm -f EMAG2_V3_20170530.nc; unzip -n EMAG2_V3_20170530.zip"
I wonder if there is any good reason why Paul didn't do it and preferred to upload the data to the server.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think you can decide what the best way is.
I have updated the gebco and gebcosi in the candidate server. |
For ther record, now for the gebco grid we get the zip file from the official site. The zip file has also two pdf. We only extract the nc grid. It won't overwrite an existing file. |
Feel free to check and test the new recipes. I will merge this tonight (if I don't get any comment). |
GEBCO grid was updated to 2024 version at GenericMappingTools/gmtserver-admin#265
GEBCO grid was updated to 2024 version at GenericMappingTools/gmtserver-admin#265
No description provided.