Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update scripts for GEBCO 2024 #265

Merged
merged 5 commits into from
Aug 13, 2024
Merged

Update scripts for GEBCO 2024 #265

merged 5 commits into from
Aug 13, 2024

Conversation

Esteban82
Copy link
Member

No description provided.

@Esteban82 Esteban82 self-assigned this Jul 25, 2024
# SRC_TITLE=GEBCO_Earth_Relief_2023
# SRC_REF="GEBCO_Compilation_Group,_2023_GEBCO_2023_Grid"
# SRC_DOI="https://doi.org/10.5285/f98b053b-0cbc-6c23-e053-6c86abc0af7b"
# SRC_FILE=ftp://ftp.soest.hawaii.edu/pwessel/GEBCO/GEBCO_2024.nc
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can't be right anymore

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I will revise the recipe when I finished processing it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am not sure what to do with that.

I understand that Paul put the grid there for a backup. Maybe we could use another place to put the data (like /export/gmtserver/gmt) or not save the data anywhere and use the original link (https://www.bodc.ac.uk/data/open_download/gebco/gebco_2024/zip/)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use the original link if possible.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. The file from the original link is a zip. So, maybe this is an issue? Should we:

  1. Use the original file (and at to the script a command to unzip it).
  2. Save the unzip file in another directory in the server (as Paul did).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, but we should add the file to Paul's FTP but I don't know if we can. I think not.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it was already on Paul's FTP. If not, then I think we can't upload anything to it.

Can SRC_FILE be a local path?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I usually get the original file, unzip it and move it to the staging directory.

I think that the SRC_FILE only is needed is the file doesn't exist. I think this is the part of the script that do that:

# 5. Determine if this source is an URL and if we need to download it first
is_url=$(echo ${SRC_FILE} | grep -c :)
if [ $is_url ]; then	# Data source is an URL
	if [ ! -f ${SRC_BASENAME} ]; then # Must download first
		echo "srv_downsampler_grid.sh: Must download original source ${SRC_FILE}"
		curl -k ${SRC_FILE} --output ${SRC_BASENAME}
	fi
	SRC_ORIG=${SRC_FILE}
	SRC_FILE=${SRC_BASENAME}
fi

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe we could use the zip file directly and unzip it as we do with the emag2 recipe.

# SRC_FILE=https://www.ngdc.noaa.gov/geomag/data/EMAG2/EMAG2_V3_20170530/EMAG2_V3_20170530.zip
# SRC_PROCESS="rm -f EMAG2_V3_20170530.nc; unzip -n EMAG2_V3_20170530.zip"

I wonder if there is any good reason why Paul didn't do it and preferred to upload the data to the server.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can decide what the best way is.

@Esteban82
Copy link
Member Author

I have updated the gebco and gebcosi in the candidate server.

@Esteban82 Esteban82 mentioned this pull request Aug 9, 2024
@Esteban82 Esteban82 changed the title WIP update scripts for GEBCO 2024 pdate scripts for GEBCO 2024 Aug 9, 2024
@Esteban82 Esteban82 changed the title pdate scripts for GEBCO 2024 Update scripts for GEBCO 2024 Aug 9, 2024
@Esteban82 Esteban82 marked this pull request as ready for review August 9, 2024 01:08
@Esteban82
Copy link
Member Author

For ther record, now for the gebco grid we get the zip file from the official site.

The zip file has also two pdf. We only extract the nc grid. It won't overwrite an existing file.

@Esteban82
Copy link
Member Author

Feel free to check and test the new recipes. I will merge this tonight (if I don't get any comment).

@Esteban82 Esteban82 merged commit e16841a into master Aug 13, 2024
1 check passed
@Esteban82 Esteban82 deleted the GEBCO_2024 branch August 13, 2024 01:43
weiji14 added a commit to GenericMappingTools/pygmt that referenced this pull request Aug 13, 2024
weiji14 added a commit to GenericMappingTools/pygmt that referenced this pull request Aug 13, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants