Merge branch 'main' into paras/sky-183-azure-transfers-are-broken-on-…

…main

docs/debugging.md

@@ -1,27 +1,36 @@
 # Debugging Tools
-Skyplane has built-in tools for debugging during development.
+Skyplane has built-in tools for debugging during development. 
 
+## Logs 
 
-## Gateway Logs 
+### Client Logs
+When you run a transfer, the client logs will be written to a folder inside of the `/tmp/skyplane/transfer_logs/` directory. 
 
-### Dozzle
-You can view gateway logs at ???.
-
-### Gateway `ssh` 
-You can `ssh` into a gateway with: 
+### Gateway Logs
+Inside the `client.log` file, for each provisioned Skyplane gateway there will be the lines: 
 ```
-skyplane ssh
+[INFO]     Log viewer: http://127.0.0.1:[PORT]/container/[CONTAINER_ID] 
+[INFO]     API: http://127.0.0.1:[PORT]
 ```
-which will list running gateways you can `ssh` into. 
+which correspond to the gateway log viewer and the gateway chunk API. You can view the logs for each gateway by going to the "Log viewer" address for that gateway. 
 
-To view the gateway logs on the instance, you can view the docker logs with: 
+### Chunk API 
+The gateway chunk API allows for status of chunks on each gateway to be queried externally, and is used by the Skyplane client to monitor transfers. 
 ```
-docker logs ??
+    * GET /api/v1/status - returns status of API
+    * GET /api/v1/servers - returns list of running servers
+    * GET /api/v1/chunk_requests - returns list of chunk requests (use {'state': '<state>'} to filter)
+    * GET /api/v1/chunk_requests/<int:chunk_id> - returns chunk request
+    * POST /api/v1/chunk_requests - adds a new chunk request
+    * PUT /api/v1/chunk_requests/<int:chunk_id> - updates chunk request
+    * GET /api/v1/chunk_status_log - returns list of chunk status log entries
 ```
-You can view downloaded chunks in the ?? folder.
-
 
-## Chunk API 
-For a running gateway, you can use HTTP requests to view the chunk state on that gateway. 
+## Gateway `ssh`
+You can `ssh` into a gateway with: 
+```
+skyplane ssh
+```
+which will list available gateways that you can select to ssh into. 
 
-TODO: can we auto-document the chunk API? 
+Once you've ssh-ed into a gateway instance, you can interact with the Skyplane docker image (??).

docs/development_guide.md

@@ -59,4 +59,4 @@ $ skyplane cp s3://... s3://...
 ## Development Tips 
 If testing transfers repeatedly, we recommend using the `--reuse-gateways` to reduce setup time. 
 
-Skyplane has debugging tools to `ssh` into gateway instances, view detailed transfer logs, and query chunk states during transfers. See [Debugging Tools](debugging.md) for more. 
+Skyplane has debugging tools to `ssh` into gateway instances, view detailed transfer logs, and query chunk states during transfers. See [Debugging Tools](debugging.md) for more. 

skyplane/compute/azure/azure_auth.py

@@ -83,9 +83,9 @@ def get_skus(region):
                     valid_skus.append(sku.name)
             return set(valid_skus)
 
-        print("    Querying for SKU availbility in regions")
-        result = do_parallel(get_skus, region_list, spinner=True, spinner_persist=False, desc="Query SKUs")
-
+        result = do_parallel(
+            get_skus, region_list, spinner=True, spinner_persist=False, desc="Query available VM SKUs from each enabled Azure region", n=8
+        )
         region_sku = dict()
         for region, skus in result:
             region_sku[region] = list()