Code

✨ Migrate autoscaling (⚠️ devops)#3566

Code

Merged

sanderegg merged 155 commits intoITISFoundation:masterITISFoundation/osparc-simcore:masterfrom sanderegg:is3558/migrate_autoscalingsanderegg/osparc-simcore:is3558/migrate_autoscalingCopy head branch name to clipboardNov 27, 2022

+2,906-483

Commits

Commits on Nov 27, 2022

• deprecated parameters to isort

  sanderegg

  committed
  78b98f4

• added asgi-lifespan

  sanderegg

  committed
  4e17d81

• initial test

  sanderegg

  committed
  4974bb3

• define polling interval

  sanderegg

  committed
  b3cf9cb

• create background task

  sanderegg

  committed
  856bff6

• refactor

  sanderegg

  committed
  ef0e795

• test check that background task restarts

  sanderegg

  committed
  e0e712d

• check cancellation

  sanderegg

  committed
  631b1f1

• rename

  sanderegg

  committed
  20666fe

• initial test

  sanderegg

  committed
  1e782ec

• added docker for testing

  sanderegg

  committed
  4232834

• adding calls to mimick script

  sanderegg

  committed
  36e28e5

• renaming

  sanderegg

  committed
  1f87b3c

• fix check on needed resources

  sanderegg

  committed
  6673897

• move fixture up

  sanderegg

  committed
  b7ed79b

• add pytest-helpers-namespace

  sanderegg

  committed
  90a4a4b

• moved helper fct

  sanderegg

  committed
  29dbe88

• Revert "add pytest-helpers-namespace"

  Show description for f90fe57
  sanderegg

  committed
  f90fe57

• make the assertion a fixture

  sanderegg

  committed
  945d4bb

• added psutil for testing

  sanderegg

  committed
  a60c3b8

• test evaluation of cluster resources

  sanderegg

  committed
  2943645

• rename

  sanderegg

  committed
  d1d8997

• test get labelized node resources

  sanderegg

  committed
  fe678cd

• rename

  sanderegg

  committed
  d5d4815

• rename

  sanderegg

  committed
  fbb16a2

• test compute used resources

  sanderegg

  committed
  748dfe1

• fix tests

  sanderegg

  committed
  567fbbb

• cleanup

  sanderegg

  committed
  10f8929

• temp: adding function to get pending resources

  sanderegg

  committed
  4c45862

• more settings

  sanderegg

  committed
  13a3b4d

• added some dedicated error

  sanderegg

  committed
  f494150

• refactor

  sanderegg

  committed
  99cbe9c

• add deepdiff

  sanderegg

  committed
  3d622ba

• reworking calls to get info on swarm

  sanderegg

  committed
  a43cc90

• improve

  sanderegg

  committed
  bb50fbb

• pending service tasks method tested

  sanderegg

  committed
  74bc856

• all tests there

  sanderegg

  committed
  e5395a6

• make the whole faster

  sanderegg

  committed
  c98f735

• ensure container associated with service is really gone

  sanderegg

  committed
  5fafdcc

• typo

  sanderegg

  committed
  de4dcbe

• not ready with aws

  sanderegg

  committed
  8def947

• added moto

  sanderegg

  committed
  1eea6f0

• added boto3 stubs

  sanderegg

  committed
  1d81fcc

• added type hints for ec2

  sanderegg

  committed
  246feea

• fix reqs

  sanderegg

  committed
  7052e92

• adding mock ec2 server

  sanderegg

  committed
  b4ad67f

• moved model

  sanderegg

  committed
  f2836ee

• testing aws utils

  sanderegg

  committed
  0ad7470

• renaming

  sanderegg

  committed
  8d2d403

• ensure cpus are positive numbers

  sanderegg

  committed
  6deb9e5

• actually non negative

  sanderegg

  committed
  918488b

• more tests

  sanderegg

  committed
  96ad5a8

• test best fitting ec2 instance

  sanderegg

  committed
  be2aed5

• code to launch AWS instances is now there

  sanderegg

  committed
  b470829

• set to info

  sanderegg

  committed
  454651c

• use default policy

  sanderegg

  committed
  66f3ee6

• moved fixtures

  sanderegg

  committed
  0a197e9

• upgrade reqs

  sanderegg

  committed
  227ff25

• tests are passing

  sanderegg

  committed
  5008fae

• sonarcloud

  sanderegg

  committed
  3159e72

• use pydantic models

  sanderegg

  committed
  8d70523

• upgrade docker api models

  sanderegg

  committed
  f30288a

• no verify for this file

  sanderegg

  committed
  8ad3586

• fixed warnings

  sanderegg

  committed
  47ea555

• renamed variables

  sanderegg

  committed
  c703cd1

• mypy

  sanderegg

  committed
  e3eaad1

• also typecheck autoscaling

  sanderegg

  committed
  605529a

• ask for 1 instance per task

  sanderegg

  committed
  21d8bc1

• added test for getting resources from a docker task

  sanderegg

  committed
  62358fc

• added docs

  sanderegg

  committed
  0834383

• cleanup

  sanderegg

  committed
  4793fc7

• make it start some instances

  sanderegg

  committed
  94a385c

• cleanup

  sanderegg

  committed
  de59e3b

• typo

  sanderegg

  committed
  c0cddd7

• cleanup

  sanderegg

  committed
  c426ac8

• cleaning up

  sanderegg

  committed
  950e7d6

• cleanup + use redis alpine

  sanderegg

  committed
  37e74cf

• test creation of instance through moto

  sanderegg

  committed
  9ef0d49

• pass env variables

  sanderegg

  committed
  6e8d8bf

• cleanup

  sanderegg

  committed
  c0f2fed

• properly mock settings

  sanderegg

  committed
  285212d

• pass tags correctly

  sanderegg

  committed
  1cccb31

• added placement and resources constraints

  sanderegg

  committed
  ee3bf8a

• typing

  sanderegg

  committed
  c881beb

• added error when too many instances are started

  sanderegg

  committed
  98615d0

• added descriptions and max number of instances

  sanderegg

  committed
  14c8140

• added check when already enough instances are up

  sanderegg

  committed
  5a9f118

• added method to get docker swarm join script

  sanderegg

  committed
  86b4d3e

• fix moto server not reset between tests

  sanderegg

  committed
  ab20c22

• check return code

  sanderegg

  committed
  854ea46

• add some nicer tags

  sanderegg

  committed
  fa05507

• fix check on process code

  sanderegg

  committed
  6c5ebf6

• passing the startup script

  sanderegg

  committed
  25fda73

• re-organizing settings

  sanderegg

  committed
  8158106

• renamed settings

  sanderegg

  committed
  31b35e1

• test cli

  sanderegg

  committed
  54d26ec

• add test for cli/main

  sanderegg

  committed
  03b9fb0

• added httpx

  sanderegg

  committed
  3209edc

• added test for route healthcheck

  sanderegg

  committed
  f4687f0

• allow to disable the autoscaling while doing nothing

  sanderegg

  committed
  30698de

• better message

  sanderegg

  committed
  5e76ecc

• typing

  sanderegg

  committed
  264f7f1

• return the IP addres

  sanderegg

  committed
  0b8b38d

• add some more logs

  sanderegg

  committed
  91c748b

• added function to wait for a node and tag a node

  sanderegg

  committed
  7624b93

• added new ENV to tag new nodes

  sanderegg

  committed
  0b15e30

• return the new dns

  sanderegg

  committed
  076bc1d

• auto tag new node

  sanderegg

  committed
  3797a13

• fix test when there is no cluster

  sanderegg

  committed
  75835e1

• removed useless comments

  sanderegg

  committed
  e536c90

• @GitHK review: add some more info to the doc

  sanderegg

  committed
  52ebc03

• @GitHK review: rename

  sanderegg

  committed
  dcb433c

• @GitHK review: simplify

  sanderegg

  committed
  b034754

• @GitHK review: increase the sleep time

  sanderegg

  committed
  316bd67

• @pcrespov review: rename

  sanderegg

  committed
  f2e39ac

• @mrnicegyu11 review: raise AutoscalingConfigurationError if misconfiguration is detected

  Show description for cd6e14a
  sanderegg

  committed
  cd6e14a

• @mrnicegyu11 review: make docker join call clearer

  sanderegg

  committed
  5ae3fac

• @mrnicegyu11 review: review logs when starting a new instance

  sanderegg

  committed
  3c6c6d4

• @mrnicegyu11 review: use python version from ENV

  sanderegg

  committed
  8a7f61d

• @mrnicegyu11 review: improve logs

  sanderegg

  committed
  d295d4c

• @GitHK review: define what are these 2 variables that were not defined

  sanderegg

  committed
  41babeb

• @mrnicegyu11 review: unbury fixture

  sanderegg

  committed
  570c7f1

• @pcrespov and @mrnicegyu11 reviews

  sanderegg

  committed
  8625bb5

• @pcrespov review: moved background task to service library

  sanderegg

  committed
  41a1b61

• use tenacity for periodic task

  sanderegg

  committed
  6e8240e

• added DockerLabelKey type with constrain

  sanderegg

  committed
  c50f71d

• @mrnicegyu11 review: add constraints on allowed docker labels

  sanderegg

  committed
  8044125

• renaming

  sanderegg

  committed
  d170bba

• new node label used now

  sanderegg

  committed
  2e6a698

• fix name of fct

  sanderegg

  committed
  bc9d7e8

• refactor

  sanderegg

  committed
  b2af9a7

• @mrnicegyu11 review: remove unused variable

  sanderegg

  committed
  50f02ef

• @mrnicegyu11 review: remove .env, useless

  sanderegg

  committed
  607b4a3

• removed all ENV variables such that developers are happy

  sanderegg

  committed
  16f7f9a

• @mrnicegyu11 review: remove comment

  sanderegg

  committed
  19b2e9c

• @mrnicegyu11 review: typo in filename

  sanderegg

  committed
  1673162

• @pcrespov review: use named capture tags

  sanderegg

  committed
  22d35ec

• clean

  sanderegg

  committed
  e481ede

• mypy

  sanderegg

  committed
  91cf3a2

• typos

  sanderegg

  committed
  6392099

• fix the retrial... without exception it does not retry ;)

  sanderegg

  committed
  365b074

• install the docker-cli in the container

  sanderegg

  committed
  32637f2

• pylint

  sanderegg

  committed
  097f30a

• services/autoscaling version: 0.1.1 → 1.0.0

  sanderegg

  committed
  a99d1d5

• remove alpha

  sanderegg

  committed
  7c09d47

• dockerfile improvements

  sanderegg

  committed
  8c61f7f

• make info log to separate from boto3

  sanderegg

  committed
  65dbf3e

• improving logs

  sanderegg

  committed
  5b94b90

• logs

  sanderegg

  committed
  d86497c

• use aioboto3 instead of boto3

  sanderegg

  committed
  788b1e5

• private dns name contains more than I thought

  sanderegg

  committed
  98e0468

• deactivate waiting for status ok

  sanderegg

  committed
  2a00443

• no confusing messages

  sanderegg

  committed
  4577ddd

• check the limits as well when getting the task resources.

  Show description for dd3beb3
  sanderegg

  committed
  dd3beb3

• application config is info

  sanderegg

  committed
  a99ff1b