Add POST requests for examples and fix URL-encoding content (#513)

* integrate custom docs with new UI * more edits * use website wording for intro * fix numbering in table * rename and some edits * rename manage_repo file, per Bo * Merge. * formatting edits * updates per Keyur's feedback * Fix typos * fix nav order * fix link to API key request form * update form link * update key request form and output dir env var * Revert to gerund Though the style guide says to just use imperatives, "get started" just sounds weird. Also this is more consistent with "troubleshooting" * new troubleshooting entry * fix typo * new data container procedures * more work * more work * complete data draft * more changes * more changes * more revisions * update troubleshooting doc etc. * new version of diagrams * remove data loading problems troubleshooting entry; can't reproduce * revert title change * add example for not mixing entity types * changes from Keyur * add screenshots for GCP, and related changes * fixed one image * added screenshots for Cloud Run service * resize images * more changes from Keyur * fix a tiny error * delete unused images * fix missing dash * update build file * adjust build command * Revert "adjust build command" This reverts commit 4ce0fb9. * update docker file * more fixes * one last fix * make links to Cloud Console open in a new page * fixes to quickstart suggested by Prem * one more change * change from Keyur * revise procedure * merge * add brief explanation of data model to quickstart * slight wording tweak * incorporate feedback from Keyur * remove erroneous edit * correct missing text * more work on tasks for finding stuff * merge * update to use env.sample * typo * typo * get file back in head shape * fix file name * add POST requests * fixes from Jennifer
datacommonsorg · Sep 18, 2024 · 32e304c · 32e304c
1 parent 5a670c6
commit 32e304c
Show file tree

Hide file tree

Showing 7 changed files with 169 additions and 77 deletions.
diff --git a/api/index.md b/api/index.md
@@ -1,7 +1,7 @@
 ---
 layout: default
 title: API
-nav_order: 0
+nav_order: 4
 has_children: true
 ---
 

diff --git a/api/rest/v2/index.md b/api/rest/v2/index.md
@@ -1,7 +1,7 @@
 ---
 layout: default
 title: REST (V2)
-nav_order: 10
+nav_order: 5
 parent: API
 has_children: true
 published: true
@@ -200,9 +200,25 @@ To illustrate again using the Argentina example:
 - All cities directly contained in Argentina (dcid: `country/ARG`): `country/ARG<-containedInPlace{typeOf:City}`
 - All cities indirectly contained in Argentina (dcid: `country/ARG`): `country/ARG<-containedInPlace+{typeOf:City}`
 
-## Escape codes for reserved characters in GET requests
+{: #url-encode}
+## URL-encoding reserved characters in GET requests
 
-HTTP GET requests do not allow some of the characters used by Data Commons DCIDs and relation expressions. When sending GET requests, you may need use the [corresponding percent codes](https://en.wikipedia.org/wiki/Percent-encoding){: target="_blank"} for reserved characters. 
+HTTP GET requests do not allow some of the characters used by Data Commons DCIDs and relation expressions. When sending GET requests, you may need to use the [corresponding percent codes](https://en.wikipedia.org/wiki/Percent-encoding){: target="_blank"} for reserved characters. For example, a query string such as the following:
+
+```
+https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId/06&property=<-*
+```
+ should be encoded as:
+
+```
+https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId%2F06&property=%3C-%2A
+```
+
+Although sometimes the original characters may work, it's safest to always encode them.
+
+> **Tip:** Don't URL-encode delimiters between parameters (`&`), separators between parameter names and values  (`=`), or `-`. 
+
+See [https://www.w3schools.com/tags/ref_urlencode.ASP](https://www.w3schools.com/tags/ref_urlencode.ASP){: target="_blank"} for a handy reference.
 
 {: #pagination}
 ## Pagination
@@ -216,7 +232,7 @@ For example, the request:
 
 ```bash
 curl --request GET \
-  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId/06&property=<-*'
+  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId%2F06&property=%3C-%2A'
 ```
 
 will return something like:
@@ -228,15 +244,15 @@ will return something like:
       "arcs": < ... output truncated for brevity ...>
     },
   },
-  "nextToken": "SoME_veRy_L0ng_S+rIng"
+  "nextToken": "SoME_veRy_L0ng_STrIng"
 }
 ```
 
 To get the next set of entries, repeat the previous command and append the `nextToken`:
 
 ```bash
 curl --request GET \
-  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId/06&property=<-*&nextToken=SoME_veRy_L0ng_S+rIng'
+  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId%2F06&property=%3C-%2A&nextToken=SoME_veRy_L0ng_STrIng'
 ```
 
 Similarly for POST requests, this would look like:
@@ -248,6 +264,7 @@ curl -X POST \
 --data '{
   "nodes": "geoId/06",
   "property": "<-*",
-  "nextToken": "SoME_veRy_L0ng_S+rIng"
+  "nextToken": "SoME_veRy_L0ng_STrIng"
 }'
-```
+```
+Don't forget to URL-encode any special characters that appear in the string.
diff --git a/api/rest/v2/node.md b/api/rest/v2/node.md
@@ -1,7 +1,7 @@
 ---
 layout: default
 title: Get node properties
-nav_order: 3
+nav_order: 7
 parent: REST (V2)
 grand_parent: API
 published: true
@@ -104,9 +104,9 @@ The response looks like:
 
 ## Examples
 
-### Example 1: Get all incoming arcs for a given node
+### Example 1: Get all incoming arc labels for a given node
 
-Get all incoming arcs of the node with DCID `geoId/06` by querying all properties with the `<-` symbol. This returns just the property labels.
+Get all incoming arc property labels of the node with DCID `geoId/06` by querying all properties with the `<-` symbol. This returns just the property labels but not the property values.
 
 Parameters:
 {: .example-box-title}
@@ -116,12 +116,21 @@ nodes: "geoId/06"
 property: "<-"
 ```
 
-Request:
+GET Request:
 {: .example-box-title}
 
 ```bash
 curl --request GET --url \
-  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId/06&property=<-'
+  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId%2F06&property=%3C-'
+```
+
+POST Request:
+{: .example-box-title}
+
+```bash
+curl -X POST -H "X-API-Key: AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI" \
+  https://api.datacommons.org/v2/node \
+  -d '{"nodes": ["geoId/06"], "property": "<-"}'
 ```
 
 Response:
@@ -142,7 +151,6 @@ Response:
   }
 }
 ```
-{: .example-box-content .scroll}
 
 ### Example 2: Get one property for a given node
 
@@ -156,12 +164,21 @@ nodes: "dc/03lw9rhpendw5"
 property: "->name"
 ```
 
-Request:
+GET Request:
 {: .example-box-title}
 
 ```bash
 curl --request GET --url \
-  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=dc/03lw9rhpendw5&property=->name'
+  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=dc%2F03lw9rhpendw5&property=-%3Ename'
+```
+
+POST Request:
+{: .example-box-title}
+
+```bash
+curl -X POST -H "X-API-Key: AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI" \
+  https://api.datacommons.org/v2/node \
+  -d '{"nodes": ["dc/03lw9rhpendw5"], "property": "->name"}'
 ```
 
 Response:
@@ -187,7 +204,6 @@ Response:
 ```
 {: .example-box-content .scroll}
 
-
 {: #multiple-properties}
 ### Example 3: Get multiple property values for multiple nodes
 
@@ -203,7 +219,16 @@ nodes: "geoId/06085", "geoId/06087"
 property: "->[name, latitude, longitude]"
 ```
 
-Request:
+GET Request:
+{: .example-box-title}
+
+```bash
+curl --request GET --url \
+  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=geoId%2F06085&nodes=geoId%2F06087&property=-%3E%5Bname,%20latitude,%20longitude%5D'
+
+```
+
+POST Request:
 {: .example-box-title}
 
 ```bash
@@ -310,12 +335,21 @@ nodes: "PowerPlant"
 property: "<-*"
 ```
 
-Request:
+GET Request:
 {: .example-box-title}
 
 ```bash
 curl --request GET --url \
-  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=PowerPlant&property=<-*'
+  'https://api.datacommons.org/v2/node?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&nodes=PowerPlant&property=%3C-%2A'
+```
+
+POST Request:
+{: .example-box-title}
+
+```bash
+curl -X POST -H "X-API-Key: AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI" \
+  https://api.datacommons.org/v2/node \
+  -d '{"nodes": ["PowerPlant"], "property": "<-*"}'
 ```
 
 Response:

diff --git a/api/rest/v2/observation.md b/api/rest/v2/observation.md
@@ -1,7 +1,7 @@
 ---
 layout: default
 title: Get statistical observations
-nav_order: 1
+nav_order: 6
 parent: REST (V2)
 grand_parent: API
 published: true
@@ -38,20 +38,22 @@ X-API-Key: AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI
 JSON data:
 {
   "date": "<var>DATE_EXPRESSION</var>",
-  "variable.dcids": [
+  "variable": {
+    "dcids": [
       "<var>VARIABLE_DCID_1</var>",
       "<var>VARIABLE_DCID_2</var>",
       ...
-    ],
-  "entity.dcids": [
+    ]
+  },
+  "entity": {
+    "dcids":[
       "<var>ENTITY_DCID_1</var>",
       "<var>ENTITY_DCID_2</var>",
       ...
-    ],
-  "entity.expression": <var>ENTITY_EXPRESSION</var>,
-  "select": "variable",
-  "select": "entity",
-  ...
+    ]
+    "expression": "<var>ENTITY_EXPRESSION</var>"
+  },
+  "select": ["date", "entity", "variable", "value"]
 }
 </div>
 
@@ -174,9 +176,6 @@ With `select=date` and `select=value` specified, the response looks like:
 
 Specify `date=LATEST` to get the latest observations and values. In this example, we select the entity by its DCID using `entity.dcids`.
 
-> Note: When sending a GET request, you need to use the following percent codes for reserved characters: 
-- `%2F` for `/`
-
 Parameters:
 {: .example-box-title}
 
@@ -190,13 +189,22 @@ select: "value"
 select: "date"
 ```
 
-Request:
+GET Request:
 {: .example-box-title}
 
 ```bash
 curl --request GET --url \
 'https://api.datacommons.org/v2/observation?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&date=LATEST&variable.dcids=Count_Person&entity.dcids=country%2FUSA&select=entity&select=variable&select=value&select=date'
 ```
+POST Request:
+{: .example-box-title}
+
+```bash
+curl -X POST -H "X-API-Key: AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI" \
+  https://api.datacommons.org/v2/observation \
+  -d '{"date": "LATEST", "variable": { "dcids": ["Count_Person"] }, "entity": { "dcids": ["country/USA"] }, "select": ["entity", "variable", "value", "date"] }'
+```
+
 {: .example-box-content .scroll}
 
 Response:
@@ -279,13 +287,23 @@ select: "value"
 select: "variable"
 ```
 
-Request:
+GET Request:
 {: .example-box-title}
 
 ```bash
 curl --request GET --url \
 'https://api.datacommons.org/v2/observation?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&date=2015&variable.dcids=Count_Person&entity.dcids=country%2FUSA&entity.dcids=geoId%2F06&select=date&select=entity&select=value&select=variable'
 ```
+
+POST Request:
+{: .example-box-title}
+
+```bash
+curl -X POST -H "X-API-Key: AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI" \
+  https://api.datacommons.org/v2/observation \
+  -d '{"date": "2015", "variable": { "dcids": ["Count_Person"] }, "entity": { "dcids": ["country/USA", "geoId/06"] }, "select": ["entity", "variable", "value", "date"] }'
+```
+
 {: .example-box-content .scroll}
 
 Response:
@@ -355,13 +373,6 @@ with date and value for each variable
 ([`Count_Person`](https://datacommons.org/tools/statvar#sv=Count_Person){: target="_blank"}) and
 entity (all counties in California).
 
-> Note: When sending a GET request, you need to use the following escape codes for reserved characters:
-- `%3C` for `<`
-- `%2B` for `+`
-- `%7B` for `{`
-- `%3A` for `:`
-- `%7D` for `}`
-
 Parameters:
 {: .example-box-title}
 
@@ -375,13 +386,23 @@ select: "value"
 select: "variable"
 ```
 
-Request:
+GET Request:
 {: .example-box-title}
 
 ```bash
 curl --request GET --url \
 'https://api.datacommons.org/v2/observation?key=AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI&date=2015&date=LATEST&variable.dcids=Count_Person&entity.expression=geoId%2F06%3C-containedInPlace%2B%7BtypeOf%3ACounty%7D&select=date&select=entity&select=value&select=variable'
 ```
+
+POST Request:
+{: .example-box-title}
+
+```bash
+curl -X POST -H "X-API-Key: AIzaSyCTI4Xz-UW_G2Q2RfknhcfdAnTHq5X5XuI" \
+  https://api.datacommons.org/v2/observation \
+  -d '{"date": "LATEST", "variable": { "dcids": ["Count_Person"] }, "entity": { "expression": "geoId/06<-containedInPlace+{typeOf:County}"}, "select": ["entity", "variable", "value", "date"] }'
+```
+
 {: .example-box-content .scroll}
 
 Response: