Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make links to Cloud Console open in a new page #476

Merged
merged 75 commits into from
Aug 15, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
75 commits
Select commit Hold shift + click to select a range
6af1ba7
integrate custom docs with new UI
kmoscoe Jul 11, 2024
cd0fe76
more edits
kmoscoe Jul 11, 2024
c45d61b
use website wording for intro
kmoscoe Jul 15, 2024
f5cbbca
fix numbering in table
kmoscoe Jul 15, 2024
90c57e7
Merge branch 'master' into custom_dc
kmoscoe Jul 15, 2024
57ca62f
rename and some edits
kmoscoe Jul 15, 2024
2277cb1
Merge branch 'custom_dc' of https://github.com/kmoscoe/docsite into c…
kmoscoe Jul 15, 2024
fb33722
rename manage_repo file, per Bo
kmoscoe Jul 16, 2024
0ebdd67
Merge.
kmoscoe Jul 17, 2024
e3148c4
merge
kmoscoe Jul 17, 2024
ee5f580
Merge branch 'datacommonsorg:master' into master
kmoscoe Jul 22, 2024
5993fb7
Merge branch 'master' of https://github.com/datacommonsorg/docsite
kmoscoe Jul 22, 2024
61ce06d
Merge branch 'custom_dc'
kmoscoe Jul 22, 2024
2b37137
Merge branch 'datacommonsorg:master' into master
kmoscoe Jul 22, 2024
37f3e87
formatting edits
kmoscoe Jul 23, 2024
d83db88
updates per Keyur's feedback
kmoscoe Jul 24, 2024
03b906f
Fix typos
kmoscoe Jul 24, 2024
03538af
fix nav order
kmoscoe Jul 24, 2024
d7fb58f
fix link to API key request form
kmoscoe Jul 24, 2024
c746a7e
update form link
kmoscoe Jul 24, 2024
d5e04af
update key request form and output dir env var
kmoscoe Jul 24, 2024
0183130
Merge branch 'master' into custom_dc
kmoscoe Jul 25, 2024
b6f53cf
Revert to gerund
kmoscoe Jul 26, 2024
e538860
Merge branch 'master' into master
kmoscoe Jul 26, 2024
d368ba4
new troubleshooting entry
kmoscoe Jul 31, 2024
db99da0
Merge branch 'datacommonsorg:master' into master
kmoscoe Jul 31, 2024
535962c
Merge branch 'custom_dc' of https://github.com/kmoscoe/docsite into c…
kmoscoe Jul 31, 2024
fd071e9
Merge branch 'master' of https://github.com/kmoscoe/docsite into cust…
kmoscoe Jul 31, 2024
771ccca
fix typo
kmoscoe Jul 31, 2024
7f2bccf
Merge branch 'datacommonsorg:master' into master
kmoscoe Aug 1, 2024
7286ff2
Merge branch 'master' of https://github.com/kmoscoe/docsite into cust…
kmoscoe Aug 5, 2024
6e61dac
new data container procedures
kmoscoe Aug 5, 2024
24ea3c4
more work
kmoscoe Aug 5, 2024
03b9786
Merge branch 'datacommonsorg:master' into master
kmoscoe Aug 6, 2024
534461b
Merge branch 'master' of https://github.com/kmoscoe/docsite into cust…
kmoscoe Aug 6, 2024
d4d81c8
more work
kmoscoe Aug 6, 2024
5d52e4b
Merge branch 'datacommonsorg:master' into master
kmoscoe Aug 6, 2024
8054ac7
Merge branch 'master' of https://github.com/kmoscoe/docsite into cust…
kmoscoe Aug 6, 2024
166e7e1
complete data draft
kmoscoe Aug 6, 2024
079fe62
more changes
kmoscoe Aug 7, 2024
5ccaba5
more changes
kmoscoe Aug 8, 2024
6d80bcc
more revisions
kmoscoe Aug 8, 2024
21917fa
update troubleshooting doc etc.
kmoscoe Aug 9, 2024
586973a
new version of diagrams
kmoscoe Aug 12, 2024
c207f6b
remove data loading problems troubleshooting entry; can't reproduce
kmoscoe Aug 12, 2024
6fa91d3
revert title change
kmoscoe Aug 12, 2024
9cde771
add example for not mixing entity types
kmoscoe Aug 12, 2024
83bbaf2
changes from Keyur
kmoscoe Aug 12, 2024
86800ba
add screenshots for GCP, and related changes
kmoscoe Aug 13, 2024
1a23559
fixed one image
kmoscoe Aug 13, 2024
b4e6bd4
added screenshots for Cloud Run service
kmoscoe Aug 13, 2024
a68f8b4
resize images
kmoscoe Aug 13, 2024
05b1894
more changes from Keyur
kmoscoe Aug 13, 2024
c606037
fix a tiny error
kmoscoe Aug 13, 2024
9bb0355
delete unused images
kmoscoe Aug 13, 2024
c497e03
Merge branch 'datacommonsorg:master' into master
kmoscoe Aug 13, 2024
914bc76
Merge branch 'datacommonsorg:master' into custom_dc
kmoscoe Aug 13, 2024
75e9b41
Merge branch 'datacommonsorg:master' into master
kmoscoe Aug 13, 2024
edcb5bc
fix missing dash
kmoscoe Aug 13, 2024
e8ca73e
update services container name
kmoscoe Aug 14, 2024
cab4484
Merge branch 'datacommonsorg:master' into master
kmoscoe Aug 14, 2024
529b734
Merge branch 'master' into custom_dc
kmoscoe Aug 14, 2024
740a1e6
Merge branch 'custom_dc' of https://github.com/kmoscoe/docsite into c…
kmoscoe Aug 14, 2024
9bf96d7
Merge branch 'master' of https://github.com/kmoscoe/docsite into cust…
kmoscoe Aug 14, 2024
ae46ea2
update build file
kmoscoe Aug 14, 2024
4ce0fb9
adjust build command
kmoscoe Aug 14, 2024
68a3f51
Revert "adjust build command"
kmoscoe Aug 14, 2024
d8f1ab7
update docker file
kmoscoe Aug 14, 2024
d67d280
more fixes
kmoscoe Aug 14, 2024
15a0a18
one last fix
kmoscoe Aug 14, 2024
5547194
Merge branch 'datacommonsorg:master' into master
kmoscoe Aug 15, 2024
0f3a1df
Merge branch 'datacommonsorg:master' into custom_dc
kmoscoe Aug 15, 2024
8c79911
merge
kmoscoe Aug 15, 2024
437948c
make links to Cloud Console open in a new page
kmoscoe Aug 15, 2024
f225640
Merge branch 'custom_dc' of https://github.com/kmoscoe/docsite into c…
kmoscoe Aug 15, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions custom_dc/data_cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ Alternatively, if you have a very large data set, you may find it faster to stor

## Prerequisites

- A [GCP](https://console.cloud.google.com/welcome) billing account and project.
- A [GCP](https://console.cloud.google.com/welcome){: target="_blank"} billing account and project.
- Optional: Install the [gcloud CLI](https://cloud.google.com/sdk/docs/install-sdk).

## One-time setup steps {#setup}
Expand All @@ -36,7 +36,7 @@ While you are testing, you can start with a single Google Cloud region; to be cl

This stores the CSV and JSON files that you will upload whenever your data changes. It also stores generated files files in a `datacommons` subdirectory when you run the data management job.

1. Go to [https://console.cloud.google.com/storage/browser](https://console.cloud.google.com/storage/browser) for your project.
1. Go to [https://console.cloud.google.com/storage/browser](https://console.cloud.google.com/storage/browser){: target="_blank"} for your project.
1. Next to **Buckets**, click **Create**.
1. Enter a name for this bucket.
1. For the **Location type**, choose the same regional options as for Cloud SQL above.
Expand All @@ -51,7 +51,7 @@ This stores the CSV and JSON files that you will upload whenever your data chang

This stores the data that will be served at run time. The Data Commons data management job will create the SQL tables and populate them when you start the job.

1. Go to https://console.cloud.google.com/sql/instances for your project.
1. Go to [https://console.cloud.google.com/sql/instances](https://console.cloud.google.com/sql/instances){: target="_blank"} for your project.
1. Next to **Instances**, click **Create Instance**.
1. Click **Choose MySQL.**
1. If necessary, enable APIs as directed.
Expand All @@ -69,11 +69,11 @@ This stores the data that will be served at run time. The Data Commons data mana

### Step 4: Create a Google Cloud Run job

Since you won't need to customize the data management container, you can simply run an instance of the released container provided by Data Commons team, at https://console.cloud.google.com.google.com/gcr/images/datcom-ci/global/datacommons-data.
Since you won't need to customize the data management container, you can simply run an instance of the released container provided by Data Commons team, at [https://console.cloud.google.com/gcr/images/datcom-ci/global/datacommons-data](https://console.cloud.google.com/gcr/images/datcom-ci/global/datacommons-data){: target="_blank"}.

See also the [Cloud Run](https://cloud.google.com/run/docs/create-jobs) documentation for links to more information on all the options you may set on your jobs.

1. Go to https://console.cloud.google.com/run/ for your project.
1. Go to [https://console.cloud.google.com/run/](https://console.cloud.google.com/run/){: target="_blank"} for your project.
1. Click **Create job**.
1. In the **Container image URL** field, click **Select** to open the **Select container image** window.
1. Click the **Container Registry** tab.
Expand Down Expand Up @@ -116,7 +116,7 @@ Now set environment variables:

### Step 1: Upload data files to Google Cloud Storage

1. Go to https://console.cloud.google.com/storage/browse and select your custom Data Commons bucket.
1. Go to [https://console.cloud.google.com/storage/browse](https://console.cloud.google.com/storage/browse){: target="_blank"} and select your custom Data Commons bucket.
1. Navigate to the folder you created in the earlier step.
1. Click **Upload Files**, and select all your CSV files and `config.json`.

Expand All @@ -132,7 +132,7 @@ Every time you upload new input CSV or JSON files to Google Cloud Storage, you w

To run the job:

1. Go to https://console.cloud.google.com/run/jobs for your project.
1. Go to [https://console.cloud.google.com/run/jobs](https://console.cloud.google.com/run/jobs){: target="_blank"} for your project.
1. From the list of jobs, click the link of the "datacommons-data" job you created above.
1. Click **Execute**. It will take several minutes for the job to run. You can click the **Logs** tab to view the progress.

Expand All @@ -142,7 +142,7 @@ When it completes, to verify that the data has been loaded correctly, see the ne

To view information about the created tables:

1. Go to [https://console.cloud.google.com/sql/instances](https://console.cloud.google.com/sql/instances) for your project and select the instance you created earlier.
1. Go to [https://console.cloud.google.com/sql/instances](https://console.cloud.google.com/sql/instances){: target="_blank"} for your project and select the instance you created earlier.
1. In the left panel, select **Cloud SQL Studio**.
1. In the **Sign in to SQL Studio** page, from the Database field, select the database you created earlier, e.g. `datacommons`.
1. Enter the user name and password and click **Authenticate**.
Expand Down
8 changes: 4 additions & 4 deletions custom_dc/deploy_cloud.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ You push a locally built Docker image to the [Google Cloud Artifact Registry](ht

## One-time setup: Create a Google Artifact Registry repository

1. Go to [https://console.cloud.google.com/artifacts](https://console.cloud.google.com/artifacts) for your project.
1. Go to [https://console.cloud.google.com/artifacts](https://console.cloud.google.com/artifacts){: target="_blank"} for your project.
1. Click **Create Repository**.
1. Specify a name for the repository.
1. In the **Format** option, select **Docker**.
Expand Down Expand Up @@ -72,7 +72,7 @@ This will take several minutes to upload.

When it completes, verify that the container has been uploaded in the Cloud Console:

1. Go to [https://console.cloud.google.com/artifacts](https://console.cloud.google.com/artifacts) for your project.
1. Go to [https://console.cloud.google.com/artifacts](https://console.cloud.google.com/artifacts){: target="_blank"} for your project.
1. In the list of repositories, click on the one you created above. Under **Repository Details**, you should see the Docker image listed.

## One-time setup: Create a Google Run service
Expand All @@ -81,7 +81,7 @@ This procedure shows you how to create a service and set environment variables u

See also [Deploying to Cloud Run](https://cloud.google.com/run/docs/deploying) for more information on all the options you may set on your service.

1. Go to https://console.cloud.google.com/run/ for your project.
1. Go to [https://console.cloud.google.com/run/](https://console.cloud.google.com/run/){: target="_blank"} for your project.
1. Click **Create service**.
1. Keep the default **Deploy one revision from an existing container image** enabled.
1. In the **Container image URL** field, click **Select** to open the **Select container image** window.
Expand Down Expand Up @@ -124,5 +124,5 @@ Click **Create** to kick off the deployment. Click the **Logs** tab to see the

Every time you make changes to the code and release a new Docker artifact, or rerun the [data management job](/custom_dc/data_cloud.html#run-job), you need to restart the service as well. To do so:

1. Go to the https://console.cloud.google.com/run/ page, click on the service you created above, and click **Edit & Deploy Revision**.
1. Go to the [https://console.cloud.google.com/run/](https://console.cloud.google.com/run/){: target="_blank"} page, click on the service you created above, and click **Edit & Deploy Revision**.
1. Select a new container and click **Deploy**.
2 changes: 1 addition & 1 deletion custom_dc/troubleshooting.md
Original file line number Diff line number Diff line change
Expand Up @@ -80,7 +80,7 @@ In general, whenever you encounter problems with any Google Cloud Run service, c
### "403 Forbidden: Your client does not have permission to get URL / from this server"

This error indicates that your application requires authenticated requests but you have not provided an authentication token. If your site is intended to be public, first check to see that the Cloud Run service is not set up to require authentication:
1. Go to the [Google Cloud Console Cloud Run](https://console.cloud.google.com?run) page for your project.
1. Go to the [Google Cloud Console Cloud Run](https://console.cloud.google.com?run){: target="_blank"} page for your project.
1. From the list of services, select the relevant service and select the **Security** tab.
1. Ensure that you have enabled **Allow unauthenticated invocations** and restart the Cloud Run service.

Expand Down