R&D - component configuration semantics #675

jorgemoralespou · 2018-08-28T16:09:48Z

We need to provide configuration semantics in different scenarios:

Component creation time. When creating a component, we need to be able to provide configuration for this component in different manners.
- Just a value as an environment variable
- A list of values as environment variables
- A list of values stored in a Configuration object (ConfigMap/Secret)
- A list of values injected as a file in the component (ConfigMap/Secret)
After the fact. When a component exists, we need to be able to provide/update/remove configuration for this component in different manners.
- Just a value as an environment variable
- A list of values as environment variables
- A list of values stored in a Configuration object (ConfigMap/Secret)
- A list of values injected as a file in the component (ConfigMap/Secret)

We need to define what will be the primitives and possibilities supported. Let's use this issue to define possible options and then come up with a proposal before implementation.

Following is a list of examples that show what needs to be done now, and how this could look like.

UC-CF1: Create a component with an environment variable

Current state:

odo create wildfly frontend
oc set env dc/frontend DEBUG=true                       # This triggers a redeployment

Desired state at creation time: UC-CF1_1: Create a component with an environment variable

odo create wildfly frontend -e DEBUG=true

Desired state after creation time: UC-CF1_2: Update a component with an environment variable

odo create wildfly frontend
odo update -e DEBUG=true

UC-CF2: Create a component with properties from a configmap

Current state at creation time:

odo create wildfly frontend
oc create configmap my-config --from-file=path/to/ba
oc set env dc/frontend --from=configmap/my-config       # This triggers a redeployment

Desired state: UC-CF2_1: Create a component with properties from a configmap

odo create wildfly frontend -E path/to/ba

Desired state after creation time: UC-CF2_2: Update a component with properties from a configmap

odo create wildfly frontend
odo update -E path/to/ba

Alternate desired state after creation time: UC-CF2_2: Add configuration to a component with properties from a configmap

odo create wildfly frontend
odo config create path/to/ba

UC-CF3: Create a component with a configmap as a file

Current state:

odo create wildfly frontend
oc create configmap my-config --from-file=path/to/ba
oc set volume dc/frontend --add -m /myapp/config --configmap-name=my-config       # This triggers a redeployment

Desired state at creation time: UC-CF3_1: Create a component with a configmap as a file

odo create wildfly frontend --config-file path/to/ba:/myapp/config

Desired state after creation time: UC-CF3_2: Update a component with a configmap as a file

odo create wildfly frontend
odo update --config-file path/to/ba:/myapp/config

Desired state after creation time: UC-CF3_3: Update a component with a configmap as a file

odo create wildfly frontend
odo config create --config-file path/to/ba:/myapp/config

cc/ @kadel @marekjelen @gshipley

The text was updated successfully, but these errors were encountered:

cmoulliard · 2018-08-28T16:45:57Z

file in the component (ConfigMap/Secret)

Are you sure that a ConfigMap or Secret is the appropriate format @jorgemoralespou ? That will imply that the user knows the YAML/JSON syntax to be used to pass such additional information.

We could start with a very simple file as we prototype with Spring Boot to pass additional info such by example name of the application, env vars, service instance (parameters,...)

E.g

name: my-spring-boot
env:
  - name: SPRING_PROFILES_ACTIVE
    value: openshift-catalog
services:
  - class: dh-postgresql-apb
    name: my-postgresql-db
    plan: dev
    parameters:
      - name: postgresql_user
        value: luke
      - name: postgresql_password
        value: secret
      - name: postgresql_database
        value: my_data
      - name: postgresql_version
        value: 9.6

jorgemoralespou · 2018-08-28T17:25:14Z

I'm not quite sure about the reference you're making. This issue should provide for a mechanism to when you want to provide an application.properties file to your SpringBoot application. This properties file should be stored in a ConfigMap in OpenShift.
Or the use case when you want to provide an ssl certificate to your application to handle ssl communications. This ssl certificate should be stored in a Secret.

What we need is to abstract the user the fact of where and how that information will be stored and provide with semantics for him to use configuration files, ssl certificates, properties, environmental values,...

cmoulliard · 2018-08-29T05:23:47Z

he reference you're making

No as typicaly the services definition will be used to configure the service instance when you select a class from the catalog. Otherwise, it will be required that the odo catalog create command will process arguments passed to know how to configure the parameters (= what is defined within a plan) of the service

cmoulliard · 2018-08-29T05:26:30Z

ssl certificate

I agree with you that here we need a secret or a vault store to manage certificate/keys and mount them within the pod of the developer. Nevertheless, it could be possible that the developer will add within the METADATA yaml file some fields needed in order to know the algorithm to be used, ... to encrypt or decrypt the keys, ... So, again, this METADATA or APPLICATION.yml file will help us

jorgemoralespou · 2018-08-29T07:58:03Z

@cmoulliard then we need to define this METADATA file, and it's something that should be out of the scope of this issue. Maybe this issue should depend on us having defined that metadata file. There's an issue for that. #579

cmoulliard · 2018-08-29T08:59:24Z

having defined that metadata file

Ok to define the spec of the MANIFEST or application.yml file with the ticket #579

cmoulliard · 2018-08-29T09:05:13Z

that should be out of the scope of this issue

This is perhaps out of scope excepted if we will offer the possibility when we create a component to by example pass the env vars to be added to the DeploymentConfig - supervisord created using this file. Then, in this case, we need also to define this line too for the configuration semantics

A list of key/values stored in a Configuration file (MANIFEST or application.yml file)

syamgk · 2018-09-20T09:40:41Z

So I assume the goal here is to:
Implement a way for users to deal with environment variables and configuration files for the applications deployed on openshift cluster using odo.

User doesn’t need to be bothered about configmaps or secrets, so abstracting the complexity is a prime thing to consider here.

@jorgemoralespou @kadel

syamgk · 2018-09-20T09:55:50Z

[updated] Proposal : Abstracting config map and Secrets from the user

By default values will come from configmaps or secrets, the name of configmap/secret keys will be same as environment variable names.
configmap name should be unique for each component, so using a combination of appname + component name. eg: <app_name>-<component_name>

Problem with this way of implementing:
If the user goes into configmap using oc or webUI and modify key names then it becomes messy.

for setting environment variables

$odo config set variable <variable name>=<Value>
Environment variable with <variable name> created successfully

This will create a Config map with <variable name> as key in it and define env to consume from config map in deployment config.

We can also add --secret as a flag which can be used for creating secrets instead of configmap. So the command will be

$odo config set variable <variable name>=<Value> --secret
Environment variable with <variable name> created successfully

for viewing environment variables in a particular application Component

$odo config view variable <component name>
Environment Variable Name            Value
MM_DB_HOST                        postgres

OR for current component

$odo config view variable

This will show the name of environment variables and values which are fetched from configmap/Secrets.

for modifying already specified environment variable

$odo config set variable <variablename>=<value>
The variable with the name which you specified already exist.Do you want to override it?(Yes/No):

This pops up and ask:
The variable with the name which you specified already exist.Do you want to override it?(Yes/No)

In a similar way we can do for configuration files,

for pushing a local configuration file

$odo config set file <path to file>:<path to mount>
File `<path to file>` has been mounted at <path to mount> and can be consumed by the component <component name>

This will embed the file which is specified into a configmap and push to server, set the mount point as specified in the command.

for viewing configuration files

$odo config view file <component name>

This will show the file name and mount path, also by using --describe flag for displaying the entire file into the terminal screen.

syamgk · 2018-09-24T08:14:11Z

@kadel @jorgemoralespou ^ If this looks good for you guys, then I can look forward for Implementation part.

syamgk · 2018-09-25T10:34:34Z

@cmoulliard

kadel · 2018-09-25T12:27:42Z

This will create a Config map with <variable name> as key in it and define env to consume from config map in deployment config.

What will be the name of the ConfigMap?

What happens if I run
odo config set variable <variable name>=<Value>
multiple times for the same component? Will this add new data to the existing ConfigMap?

for viewing environment variables in a particular application

I think that it might be better to show variables just for particular compnent rather then for the whole app.

for pushing a local configuration file
$odo config set file <path to file> mountat <path to mount>

This syntax look weird. It should be --mountpath as it is argument, or even better we could use syntax like this:

odo config set file <path to file>:<path to mount>

syamgk · 2018-09-25T13:34:17Z

This will create a Config map with <variable name> as key in it and define env to consume from config map in deployment config.

What will be the name of the ConfigMap?

configmap name will be unique for each component, so <app_name>-<component_name>

What happens if I run
odo config set variable <variable name>=<Value>
multiple times for the same component? Will this add new data to the existing ConfigMap?

if new <variable_name>, then adds the new variable as key in the same configmap, append environment names in deployment config.

if specifying same <variable_name>, prompt for
The variable with the name which you specified already exist. do you want to update `variable_name`?(Yes/No)

for viewing environment variables in a particular application

I think that it might be better to show variables just for particular compnent rather then for the whole app.

aggree 👍
so by default current component,
and an option for specifying component and application.
same for odo config view variable

for pushing a local configuration file
$odo config set file <path to file> mountat <path to mount>
This syntax look weird. It should be --mountpath as it is argument, or even better we could use syntax like this:

odo config set file <path to file>:<path to mount>

odo config set file <path to file>:<path to mount>

aggree, ^ this one looks more appealing to me.

also, am updating the old comment #675 (comment) with the inputs from here

jorgemoralespou · 2018-10-02T12:02:20Z

Can we turn this into a proposal document type as a PR, as they do in K8S, so we can fix the proposal when comments are made, and when merge we have the full document of what we want and how we want?

Some examples:

https://github.com/kubernetes/community/pull/254
https://github.com/kubernetes/community/pull/2052

kadel · 2018-10-02T13:06:53Z

Can we turn this into a proposal document type as a PR, as they do in K8S, so we can fix the proposal when comments are made, and when merge we have the full document of what we want and how we want?

Sometimes I have a feeling that you are reading my mind :-D
Yesterday I was thinking about exactly this. It gets messy to discuss proposals like this using just comments.

proposing a way to a way for users to deal with environment variables and configuration files for the applications deployed on openshift cluster using odo. fix issue redhat-developer#675

syamgk · 2018-10-04T13:04:45Z

can continue discussions on this PR #806

kadel · 2019-02-21T08:23:40Z

Some commands related to this are specified in
https://docs.google.com/document/d/1RKva9lvIR6dLmDm2tlqmp8V4nFjlrXdulOQ4FSmLkr8/edit?ts=5c5ac66b

kadel · 2019-02-27T14:23:54Z

#1390

kadel · 2019-04-10T09:45:57Z

A lot of has changed since this issue started. We have reworked some of the commands, and partially implemented some of the ideas from this issue.
I'm going to close this. We will continue on improving the component configuration, but it needs a fresh look

surajnarwade added the kind/enhancement label Sep 11, 2018

syamgk self-assigned this Sep 12, 2018

cdrage added the priority/High Important issue; should be worked on before any other issues (except priority/Critical issue(s)). label Sep 12, 2018

kadel mentioned this issue Sep 26, 2018

Improve "odo create service" command to create a service (fix #721) #723

Merged

kadel added the kind/epic An issue categorized as a high-level Epic. Needs to be scoped and broken down in 1+ stories/tasks label Sep 27, 2018

syamgk mentioned this issue Oct 4, 2018

[PROPOSAL] add proposal for odo component configuration #806

Closed

kadel added this to the M milestone Dec 3, 2018

kadel changed the title ~~R&D - odo config semantics~~ R&D - component configuration semantics Dec 3, 2018

kadel modified the milestones: M, L Dec 3, 2018

kadel unassigned syamgk Dec 3, 2018

sspeiche modified the milestones: L, 1.0 Jan 28, 2019

kadel mentioned this issue Jan 29, 2019

UC 28: As a developer, I want to be able to manage configuration for a component in an easy way #173

Closed

kadel assigned jorgemoralespou Feb 4, 2019

kadel closed this as completed Apr 10, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

R&D - component configuration semantics #675

R&D - component configuration semantics #675

jorgemoralespou commented Aug 28, 2018 •

edited

Loading

cmoulliard commented Aug 28, 2018

jorgemoralespou commented Aug 28, 2018

cmoulliard commented Aug 29, 2018

cmoulliard commented Aug 29, 2018

jorgemoralespou commented Aug 29, 2018

cmoulliard commented Aug 29, 2018

cmoulliard commented Aug 29, 2018

syamgk commented Sep 20, 2018 •

edited

Loading

syamgk commented Sep 20, 2018 •

edited

Loading

syamgk commented Sep 24, 2018

syamgk commented Sep 25, 2018

kadel commented Sep 25, 2018

for viewing environment variables in a particular application

for pushing a local configuration file

syamgk commented Sep 25, 2018 •

edited

Loading

for viewing environment variables in a particular application

for pushing a local configuration file

jorgemoralespou commented Oct 2, 2018 •

edited

Loading

kadel commented Oct 2, 2018

syamgk commented Oct 4, 2018

kadel commented Feb 21, 2019

kadel commented Feb 27, 2019

kadel commented Apr 10, 2019

R&D - component configuration semantics #675

R&D - component configuration semantics #675

Comments

jorgemoralespou commented Aug 28, 2018 • edited Loading

UC-CF1: Create a component with an environment variable

UC-CF2: Create a component with properties from a configmap

UC-CF3: Create a component with a configmap as a file

cmoulliard commented Aug 28, 2018

jorgemoralespou commented Aug 28, 2018

cmoulliard commented Aug 29, 2018

cmoulliard commented Aug 29, 2018

jorgemoralespou commented Aug 29, 2018

cmoulliard commented Aug 29, 2018

cmoulliard commented Aug 29, 2018

syamgk commented Sep 20, 2018 • edited Loading

syamgk commented Sep 20, 2018 • edited Loading

[updated] Proposal : Abstracting config map and Secrets from the user

for setting environment variables

for viewing environment variables in a particular application Component

for modifying already specified environment variable

for pushing a local configuration file

for viewing configuration files

syamgk commented Sep 24, 2018

syamgk commented Sep 25, 2018

kadel commented Sep 25, 2018

for viewing environment variables in a particular application

for pushing a local configuration file

syamgk commented Sep 25, 2018 • edited Loading

for viewing environment variables in a particular application

for pushing a local configuration file

jorgemoralespou commented Oct 2, 2018 • edited Loading

kadel commented Oct 2, 2018

syamgk commented Oct 4, 2018

kadel commented Feb 21, 2019

kadel commented Feb 27, 2019

kadel commented Apr 10, 2019

jorgemoralespou commented Aug 28, 2018 •

edited

Loading

syamgk commented Sep 20, 2018 •

edited

Loading

syamgk commented Sep 20, 2018 •

edited

Loading

syamgk commented Sep 25, 2018 •

edited

Loading

jorgemoralespou commented Oct 2, 2018 •

edited

Loading