Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🐛 rabbitmq queue name prefix is unique during the lifecycle of the container #6365

Merged
merged 15 commits into from
Sep 18, 2024
Merged

🐛 rabbitmq queue name prefix is unique during the lifecycle of the container #6365

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
344df55
added unique generation of token that persists during each container run
Sep 16, 2024
d61d002
updated docstring
Sep 16, 2024
1ba4072
rename notes
Sep 16, 2024
e8f5698
TODO: check what else fails
Sep 16, 2024
d06ea65
simplify implementation
Sep 17, 2024
5a95dac
Merge remote-tracking branch 'upstream/master' into pr-osparc-change-…
Sep 17, 2024
8e2c252
revert change
Sep 17, 2024
c75f239
Merge remote-tracking branch 'upstream/master' into pr-osparc-change-…
Sep 17, 2024
556649b
added missing psutil to servicelib repowisde
Sep 17, 2024
fc927c4
revert directro changes
Sep 17, 2024
8fe2964
Merge remote-tracking branch 'upstream/master' into pr-osparc-change-…
Sep 17, 2024
9eb1111
fixed missing types
Sep 17, 2024
3372b00
Merge branch 'master' into pr-osparc-change-rabbitmq-names
GitHK Sep 17, 2024
3da1a85
Merge remote-tracking branch 'upstream/master' into pr-osparc-change-…
Sep 18, 2024
a234e6d
Merge branch 'pr-osparc-change-rabbitmq-names' of github.com:GitHK/os…
Sep 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion packages/aws-library/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@ aioboto3==13.1.1
aiobotocore==2.13.1
# via aioboto3
aiocache==0.12.2
# via -r requirements/_base.in
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/_base.in
aiodebug==2.3.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodocker==0.22.2
Expand Down Expand Up @@ -174,6 +176,8 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
pydantic==1.10.17
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
1 change: 1 addition & 0 deletions packages/service-library/requirements/_base.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@ opentelemetry-api
opentelemetry-exporter-otlp
opentelemetry-instrumentation-requests
opentelemetry-sdk
psutil
pydantic
pyinstrument
pyyaml
Expand Down
2 changes: 2 additions & 0 deletions packages/service-library/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,6 +138,8 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via -r requirements/_base.in
pydantic==1.10.17
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
4 changes: 2 additions & 2 deletions packages/service-library/requirements/_test.in
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,6 @@ docker
faker
flaky
openapi-spec-validator
psutil
pytest
pytest-aiohttp
pytest-asyncio
Expand All @@ -35,5 +34,6 @@ python-dotenv
respx
sqlalchemy[mypy]
types_aiofiles
types-psycopg2
types_tqdm
types-psutil
types-psycopg2
4 changes: 2 additions & 2 deletions packages/service-library/requirements/_test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,8 +141,6 @@ pluggy==1.5.0
# via pytest
pprintpp==0.4.0
# via pytest-icdiff
psutil==6.0.0
# via -r requirements/_test.in
py-cpuinfo==9.0.0
# via pytest-benchmark
pytest==8.3.2
Expand Down Expand Up @@ -244,6 +242,8 @@ termcolor==2.4.0
# via pytest-sugar
types-aiofiles==24.1.0.20240626
# via -r requirements/_test.in
types-psutil==6.0.0.20240901
# via -r requirements/_test.in
types-psycopg2==2.9.21.20240819
# via -r requirements/_test.in
types-tqdm==4.66.0.20240417
Expand Down
10 changes: 9 additions & 1 deletion packages/service-library/src/servicelib/rabbitmq/_utils.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
import logging
import os
import socket
from typing import Any, Final

import aio_pika
import psutil
from aiormq.exceptions import ChannelPreconditionFailed
from pydantic import NonNegativeInt
from tenacity import retry
Expand Down Expand Up @@ -51,7 +53,13 @@ async def wait_till_rabbitmq_responsive(url: str) -> bool:


def get_rabbitmq_client_unique_name(base_name: str) -> str:
return f"{base_name}_{socket.gethostname()}"
# NOTE: below prefix is guaranteed to change each time the preocess restarts
GitHK marked this conversation as resolved.
Show resolved Hide resolved
# Why is this desiarable?
# 1. the code base makes the above assumption, otherwise subcscribers and consumers do not work
# 2. enables restartability of webserver during [re]deploys
prefix_create_time = f"{psutil.Process(os.getpid()).create_time()}".strip(".")[-6:]
GitHK marked this conversation as resolved.
Show resolved Hide resolved

return f"{base_name}_{socket.gethostname()}_{prefix_create_time}"


async def declare_queue(
Expand Down
6 changes: 5 additions & 1 deletion packages/simcore-sdk/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
aio-pika==9.4.3
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiocache==0.12.2
# via -r requirements/_base.in
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/_base.in
aiodebug==2.3.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodocker==0.22.2
Expand Down Expand Up @@ -191,6 +193,8 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
psycopg2-binary==2.9.9
# via
# aiopg
Expand Down
9 changes: 8 additions & 1 deletion services/api-server/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,10 @@ aio-pika==9.4.1
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
aiocache==0.12.2
# via -r requirements/../../../packages/simcore-sdk/requirements/_base.in
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/_base.in
aiodebug==2.3.0
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
Expand Down Expand Up @@ -365,6 +368,10 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
psycopg2-binary==2.9.9
# via
# aiopg
Expand Down
7 changes: 6 additions & 1 deletion services/autoscaling/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,9 @@ aiobotocore==2.13.1
# via aioboto3
aiocache==0.12.2
# via
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/_base.in
aiodebug==2.3.0
# via
Expand Down Expand Up @@ -57,6 +59,7 @@ arrow==1.3.0
# -r requirements/../../../packages/aws-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/_base.in
# -r requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
Expand Down Expand Up @@ -344,9 +347,11 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==5.9.8
psutil==6.0.0
# via
# -c requirements/../../../services/dask-sidecar/requirements/_dask-distributed.txt
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
# distributed
pydantic==1.10.15
# via
Expand Down
2 changes: 1 addition & 1 deletion services/autoscaling/requirements/_test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,7 +173,7 @@ ply==3.11
# via jsonpath-ng
pprintpp==0.4.0
# via pytest-icdiff
psutil==5.9.8
psutil==6.0.0
# via
# -c requirements/_base.txt
# -r requirements/_test.in
Expand Down
6 changes: 5 additions & 1 deletion services/catalog/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
aio-pika==9.4.1
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiocache==0.12.2
# via -r requirements/_base.in
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/_base.in
aiodebug==2.3.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodocker==0.21.0
Expand Down Expand Up @@ -257,6 +259,8 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
psycopg2-binary==2.9.9
# via sqlalchemy
pydantic==1.10.14
Expand Down
10 changes: 8 additions & 2 deletions services/clusters-keeper/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@ aioboto3==13.1.0
aiobotocore==2.13.1
# via aioboto3
aiocache==0.12.2
# via -r requirements/../../../packages/aws-library/requirements/_base.in
# via
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
aiodebug==2.3.0
# via
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
Expand Down Expand Up @@ -54,6 +57,7 @@ arrow==1.3.0
# -r requirements/../../../packages/aws-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/_base.in
# -r requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
Expand Down Expand Up @@ -341,9 +345,11 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==5.9.8
psutil==6.0.0
# via
# -c requirements/../../../services/dask-sidecar/requirements/_dask-distributed.txt
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
# distributed
pydantic==1.10.15
# via
Expand Down
2 changes: 1 addition & 1 deletion services/clusters-keeper/requirements/_test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -193,7 +193,7 @@ pluggy==1.5.0
# via pytest
ply==3.11
# via jsonpath-ng
psutil==5.9.8
psutil==6.0.0
# via
# -c requirements/_base.txt
# -r requirements/_test.in
Expand Down
8 changes: 6 additions & 2 deletions services/dask-sidecar/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@ aio-pika==9.4.1
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiobotocore==2.13.0
# via s3fs
aiocache==0.12.2
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodebug==2.3.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodocker==0.21.0
Expand Down Expand Up @@ -251,8 +253,10 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==5.9.8
# via distributed
psutil==6.0.0
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# distributed
pydantic==1.10.15
# via
# -c requirements/../../../packages/dask-task-models-library/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
2 changes: 1 addition & 1 deletion services/dask-sidecar/requirements/_dask-distributed.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ partd==1.4.2
# via
# -c requirements/./_base.txt
# dask
psutil==5.9.8
psutil==6.0.0
# via
# -c requirements/./_base.txt
# distributed
Expand Down
6 changes: 5 additions & 1 deletion services/datcore-adapter/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
aio-pika==9.4.1
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiocache==0.12.2
# via -r requirements/_base.in
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/_base.in
aiodebug==2.3.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodocker==0.21.0
Expand Down Expand Up @@ -229,6 +231,8 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
pydantic==1.10.14
# via
# -c requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
6 changes: 5 additions & 1 deletion services/director-v2/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,8 @@ aio-pika==9.4.1
# -r requirements/_base.in
aiocache==0.12.2
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/_base.in
# -r requirements/_base.in
aiodebug==2.3.0
Expand Down Expand Up @@ -430,8 +432,10 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==5.9.8
psutil==6.0.0
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../services/dask-sidecar/requirements/_dask-distributed.txt
# distributed
psycopg2-binary==2.9.9
Expand Down
2 changes: 1 addition & 1 deletion services/director-v2/requirements/_test.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -215,7 +215,7 @@ pluggy==1.5.0
# via pytest
pprintpp==0.4.0
# via pytest-icdiff
psutil==5.9.8
psutil==6.0.0
# via
# -c requirements/_base.txt
# distributed
Expand Down
4 changes: 4 additions & 0 deletions services/dynamic-scheduler/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
aio-pika==9.4.1
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiocache==0.12.2
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodebug==2.3.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
aiodocker==0.21.0
Expand Down Expand Up @@ -233,6 +235,8 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via -r requirements/../../../packages/service-library/requirements/_base.in
psycopg2-binary==2.9.9
# via sqlalchemy
pydantic==1.10.15
Expand Down
12 changes: 9 additions & 3 deletions services/dynamic-sidecar/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,10 @@ aio-pika==9.4.1
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/_base.in
aiocache==0.12.2
# via -r requirements/../../../packages/simcore-sdk/requirements/_base.in
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/_base.in
aiodebug==2.3.0
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
Expand Down Expand Up @@ -316,8 +319,11 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==5.9.8
# via -r requirements/_base.in
psutil==6.0.0
# via
# -r requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/simcore-sdk/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/_base.in
psycopg2-binary==2.9.9
# via
# aiopg
Expand Down
10 changes: 9 additions & 1 deletion services/efs-guardian/requirements/_base.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@ aioboto3==13.1.0
aiobotocore==2.13.1
# via aioboto3
aiocache==0.12.2
# via -r requirements/../../../packages/aws-library/requirements/_base.in
# via
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
aiodebug==2.3.0
# via
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
Expand Down Expand Up @@ -54,6 +57,7 @@ arrow==1.3.0
# -r requirements/../../../packages/aws-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/aws-library/requirements/_base.in
# -r requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/../../../packages/models-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
Expand Down Expand Up @@ -280,6 +284,10 @@ protobuf==4.25.4
# via
# googleapis-common-protos
# opentelemetry-proto
psutil==6.0.0
# via
# -r requirements/../../../packages/aws-library/requirements/../../../packages/service-library/requirements/_base.in
# -r requirements/../../../packages/service-library/requirements/_base.in
pydantic==1.10.15
# via
# -c requirements/../../../packages/aws-library/requirements/../../../packages/models-library/requirements/../../../requirements/constraints.txt
Expand Down
Loading
Loading