Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Expose the API key/host/secret as variables for the users #29

Merged
merged 6 commits into from
Apr 22, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .bumpversion.cfg
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
[bumpversion]
current_version = 3.0.1
current_version = 3.0.2
commit = False
message = service version: {current_version} → {new_version}
tag = False
Expand Down
4 changes: 2 additions & 2 deletions .osparc/jupyter-math/metadata.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ description:

"
key: simcore/services/dynamic/jupyter-math
version: 3.0.1
version: 3.0.2
integration-version: 2.0.0
type: dynamic
authors:
Expand Down Expand Up @@ -199,4 +199,4 @@ boot-options:
description:
To start as Voila save a notebook as "voila.ipynb" in the root
folder
min-visible-inputs: 4
min-visible-inputs: 4
12 changes: 11 additions & 1 deletion .osparc/jupyter-math/runtime.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,14 @@ callbacks-mapping:
inactivity:
service: container
command: ["python", "/usr/local/bin/service-monitor/activity.py"]
timeout: 1
timeout: 1
compose-spec:
version: "3.7"
services:
jupyter-math:
image: $${SIMCORE_REGISTRY}/simcore/services/dynamic/jupyter-math:$${SERVICE_VERSION}
environment:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This does not exist in Osparc codebase ad it will not resolve on all deployments

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you mean this is because the API server is not available on some deployments? Or just because the variable is not set yet?

@pcrespov in your new PR on simcore, what will happen if there is no API server is available? Will this var be empty/None or so, or will it just not be set?

Copy link
Member

@pcrespov pcrespov Apr 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@GitHK Actually, waiting for you review ;-) ITISFoundation/osparc-simcore#5695

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@wvangeit i will then pull it and use it locally.

Copy link
Member

@pcrespov pcrespov Apr 19, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what will happen if there is no API server is available? Will this var be empty/None or so, or will it just not be set?

@wvangeit Good question. In this case, I can see in the code that it simply does not get replaced.

https://github.com/itisfoundation/osparc-simcore/blob/5035fe5cc7a7d7ee9fcab15fd7f8a116a4934cd6/services/director-v2/src/simcore_service_director_v2/modules/dynamic_sidecar/docker_compose_specs.py#L340-L349

Apparently the services will fail upon start if OSPARC_VARIABLE_API_HOST is not exposed in osparc. IMO it is a bit extreme since a normal user (i.e. somebody that is not the service owner) will not understand why this service failed to start. If this expression is an an .env file and OSPARC_VARIABLE_API_HOST is undefined it then OSPARC_API_HOST gets assigned an empty string! Nonetheless @GitHK, hwo is the code owner, has a different view. He will write it down later for you to be aware of it

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missed your PR. Got distracted. It's all good now.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would vote for empty string. Then the user and the python api client can catch this, and notice that the api host is undefined. (in a sense some 'null' instead of empty string would be better, but not sure if there is a stable null equivalent in the world of env vars)

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I prefer that OSPARC_VARIABLE_* have to be resolvable in order for the service to start. Unlike the OSPARC_VARIABLE_VENDOR_SECRET_*, which are define din the db, these are provided always by the platform.

If you are requesting an OSPARC_VARIABLE_*, which does not exist, this is a clear way to signal that you did something wrong.

Also I think it's the user's responsibility to make sure that the service stars once they make changes to it. He can always test it in a local deployment before publishing it elsewhere. Unfortunately we don't have a better workflow for this part.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • FYI vendor secrets are also provided by the service owner.
  • directorv2 also fails to start service when the vendor secrets is not available.

- OSPARC_API_HOST=$${OSPARC_VARIABLE_API_HOST}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very useful feature in the related PR, thanks @pcrespov. If I get this right, if the services specs are adapted correctly, users will not need anymore to create API keys and secrets when they run from inside oSPARC!

I am not sure what the conclusion about OSPARC_API_HOST. I think the obvious thing to avoid is that the service fails to start if this is not set. 😉

Small question: ${OSPARC_VARIABLE_API_HOST} and ${OSPARC_VARIABLE_API_KEY} will always be available, right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elisabettai yes, they are.
SEE also ITISFoundation/osparc-simcore-clients#149

@elisabettai can you please merge these changes and produce a new version of this service and inject in master registry so we can test it there?. thx

Copy link
Collaborator

@elisabettai elisabettai Apr 22, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To (auto-magically) publish a new version of this: @wvangeit, can you please run make version-patch? Then I'll merge and take care of publishing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@elisabettai could you also deploy it to osparc.io now.
Metamodelling is working there now, I need this variable accessible in the notebooks to test the axondeepseg metamodelling study. Thx.

- OSPARC_API_KEY=$${OSPARC_VARIABLE_API_KEY}
- OSPARC_API_SECRET=$${OSPARC_VARIABLE_API_SECRET}
container-http-entrypoint: jupyter-math
2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ SHELL = /bin/sh
.DEFAULT_GOAL := help

export DOCKER_IMAGE_NAME ?= jupyter-math
export DOCKER_IMAGE_TAG ?= 3.0.1
export DOCKER_IMAGE_TAG ?= 3.0.2


# PYTHON ENVIRON ---------------------------------------------------------------------------------------
Expand Down
4 changes: 2 additions & 2 deletions docker-compose-local.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
version: '3.7'
services:
jupyter-math:
image: simcore/services/dynamic/jupyter-math:3.0.1
image: simcore/services/dynamic/jupyter-math:3.0.2
ports:
- "8888:8888"
environment:
Expand All @@ -13,4 +13,4 @@ services:
- /tmp/.X11-unix:/tmp/.X11-unix
- ${PWD}/validation/workspace:/home/jovyan/work/workspace
- ${PWD}/validation/inputs:/tmp/inputs
- ${PWD}/validation/outputs:/tmp/outputs
- ${PWD}/validation/outputs:/tmp/outputs
Loading