Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New skylark init workflow with better config management #184

Merged
merged 39 commits into from
Mar 16, 2022
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
39 commits
Select commit Hold shift + click to select a range
309b116
Squashed commit of the following:
parasj Mar 8, 2022
dc14afb
skylark gcp ssh (#194)
parasj Mar 8, 2022
954a2f1
Autoload config for GCP cli
parasj Mar 8, 2022
e7821bc
Misc changes
parasj Mar 8, 2022
d69ed6f
Print deprovision before final result log
parasj Mar 8, 2022
1523f83
?
parasj Mar 9, 2022
f38dace
Fix pytype errors
parasj Mar 10, 2022
7adbdf0
Skylark init end to end
parasj Mar 10, 2022
ca19997
Remove quotes from auto azure config
parasj Mar 10, 2022
c412f84
Role assignment
parasj Mar 10, 2022
c4107c9
?~
parasj Mar 10, 2022
94832a2
GCP auth
parasj Mar 14, 2022
d7b1dd5
Merge branch 'main' into dev/paras/better_init_config
parasj Mar 15, 2022
cc1a6a4
Fix service quota query
parasj Mar 15, 2022
84bf734
experiemnt paras
parasj Mar 15, 2022
62cb42e
Catch resource group not available
ShishirPatil Mar 15, 2022
aa11bfa
Merge branch 'dev/paras/better_init_config' of github.com:parasj/skyl…
ShishirPatil Mar 15, 2022
be4bfc2
Merge branch 'main' into dev/paras/better_init_config
parasj Mar 15, 2022
4672219
Fix credential inference
parasj Mar 15, 2022
af7e34a
Regenerate key pairs
parasj Mar 15, 2022
92c0996
Update
parasj Mar 15, 2022
e25caec
Cache credentials to avoid credential errors
parasj Mar 16, 2022
b4431ca
Storage account successfully created
parasj Mar 16, 2022
715d24c
Changing Azure to premium storage
ShishirPatil Mar 16, 2022
98fe25b
AWS and GCP works
parasj Mar 16, 2022
6eeedcc
Make container
parasj Mar 16, 2022
7384d20
Autoflake
parasj Mar 16, 2022
a226c56
Autoflake
parasj Mar 16, 2022
d7e2df8
Remove unused
parasj Mar 16, 2022
90b61c3
Dead code elimination
parasj Mar 16, 2022
688b21b
Cast keyfile to string
parasj Mar 16, 2022
370188a
Unused tests
parasj Mar 16, 2022
79dcafa
Merge branch 'dev/paras/cleanup' into dev/paras/better_init_config
parasj Mar 16, 2022
5aa0493
Partial cleanup from patch
parasj Mar 16, 2022
7ed13b1
Merge branch 'dev/paras/cleanup' into dev/paras/better_init_config
parasj Mar 16, 2022
c12c72b
Merge branch 'main' into dev/paras/better_init_config
parasj Mar 16, 2022
77600f4
Minor cleanup
parasj Mar 16, 2022
51857f7
Fix AzureInterface
parasj Mar 16, 2022
d861212
Fix another Azure misconfiguration
parasj Mar 16, 2022
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions scripts/requirements-gateway.txt
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ azure-identity
azure-mgmt-compute
azure-mgmt-network
azure-mgmt-resource
azure-mgmt-authorization
azure-storage-blob>=12.0.0
boto3
click>=7.1.2
Expand Down
1 change: 1 addition & 0 deletions setup.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
"azure-mgmt-compute",
"azure-mgmt-network",
"azure-mgmt-resource",
"azure-mgmt-authorization",
"azure-storage-blob>=12.0.0",
"boto3",
"click>=7.1.2",
Expand Down
21 changes: 19 additions & 2 deletions skylark/__init__.py
Original file line number Diff line number Diff line change
@@ -1,10 +1,23 @@
import os

from pathlib import Path
from skylark.compute.utils import query_which_cloud

from skylark.config import SkylarkConfig

# paths
skylark_root = Path(__file__).parent.parent
key_root = skylark_root / "data" / "keys"
config_file = skylark_root / "data" / "config.json"
config_root = Path("~/.skylark").expanduser()
config_root.mkdir(exist_ok=True)

if "SKYLARK_CONFIG" in os.environ:
config_path = Path(os.environ["SKYLARK_CONFIG"]).expanduser()
else:
config_path = config_root / "config"

key_root = config_root / "keys"
tmp_log_dir = Path("/tmp/skylark")
tmp_log_dir.mkdir(exist_ok=True)

# header
def print_header():
Expand All @@ -26,3 +39,7 @@ def print_header():
KB = 1024
MB = 1024 * 1024
GB = 1024 * 1024 * 1024
if config_path.exists():
cloud_config = SkylarkConfig.load_config(config_path)
else:
cloud_config = SkylarkConfig()
92 changes: 0 additions & 92 deletions skylark/benchmark/network/latency.py

This file was deleted.

2 changes: 1 addition & 1 deletion skylark/benchmark/network/traceroute.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,7 @@ def main(args):
log_dir.mkdir(exist_ok=True, parents=True)

aws = AWSCloudProvider()
gcp = GCPCloudProvider(args.gcp_project)
gcp = GCPCloudProvider()
aws_instances, gcp_instances = provision(
aws=aws,
gcp=gcp,
Expand Down
Empty file.
102 changes: 0 additions & 102 deletions skylark/benchmark/replicate/benchmark_triangles.py

This file was deleted.

136 changes: 0 additions & 136 deletions skylark/benchmark/replicate/test_direct.py

This file was deleted.

Loading