Add concept of configurability to language

[jclark]

Add configurability as a language-level concept. The source code would be declare something configurable and then there would be a standard way at runtime to supply a value for something configurable in place of the default.

This should apply to at least:

• initializers for module level variables
• values of annotations on module level variables

[jclark]

Some questions/thoughts on this:

• Should this be done via an annotation?
• It will often be necessary to do some validation on the supplied configuration value. It will often be the case that not all values of a type will be allowed. Sometimes validation will need to be done on a combination of configured values.
• It reminds me a bit of X Window System resources.

[sanjiva]

Here's what I see as the requirements:

• Many times module level variables need to be settable from outside the program as part of a startup configuration
• Same for annotation values for annotations on module level concepts
• Annotations which are constant (i.e. require values to be compile time consts) will still want to get those values from outside the program source during compile time
• Similarly for module constants
• Any default value assignment (e.g. to a record field) should be overrideable from outside as part of module initialization if the user wants
• Arbitrary places in code will want to be able to get a value from the runtime environment with some key to identify what they want to look up

Yes the X config model worked quite well ... and they had both a class and instance approach too.

In Ballerina, module level constructs (whether they're public or not) have globally unique names: orgName/moduleName:identifier. If such construct has structure (object, service, function) then can do identifier.x.y to get to the value. For record types a similar approach could work to address the property who's value is being set (even if the type decl is inside some other construct). Anonymous types can be ignored.

The programmer should indicate that the property should allow external configuration. I was thinking of a configurable modifier with an annotation to give a key if necessary/desired as there will be an implied key usually (which is similar to X).

Yes validation will be important and we'll need to decide what to do when validation fails - e.g. if the an integer variable is given a config value of type string then what? I guess startup failure would be ideal (in X it was silently ignored or logged to the console IIRC).

We have to decide where the config values are coming from too. For compile time values, the project's Ballerina.toml is a natural location. For runtime values we could follow something like X but I'm not thrilled with tying it to a file in a user home type thing. Maybe an environment var that gives the filename is better. That's more like the configMap stuff in K8s I think.

[sanjiva]

One more point: We also need some values to be kept encrypted. We have a model for that too in the current config approach .. see https://ballerina.io/learn/by-example/config-api.html for an example.

[jclark]

Most of this does not sound like it belongs in the language spec. I think we need to start a separate platform spec that can define this sort of thing, including TOML file, project layout etc.

From a language perspective, it seems to me that we only need a modifier (as you suggest). From that perspective, the semantics are just that the value specified in the program may be overridden by something external to the program. It's a bit like external except that in that case the value is always specified externally.

[jclark]

I wonder how this should relate to public. Is it orthogonal or related in some way?

[sanjiva]

I wondered about that too. It feels orthogonal as that's about other devs being able to see something from this module whereas config is about saying "I want to let someone give a value for this".

I agree its time to start a platform spec; there's a lot of stuff that needs to be covered there including namespace management of organization names.

Ref the comment about "It's a bit like external except that in that case the value is always specified externally": what I had in mind is configurable says a value can be given from outside. I think it might be best to couple it with defaults .. so you give a default value but mark it configurable. I can't think of cases where you have to force an outside config only.

[jclark]

Another application of this basic concept is in i18n, where a localization of the user interface (e.g. error messages) to a specific language needs to be able to substitute strings in the source with strings from the target language.

[jclark]

You might want to force an outside config for a password or other security related stuff.

[sanjiva]

For i18n we probably should look at some standard approach for resource bundles that are tied to the locale. Otherwise every program will have its own way of doing i18n of strings.

[jclark]

Each programming language has its own standard way of doing this: there is no language independent standard for this as far as I am aware. If we have the configurable concept, which can handle this, why would we not use that concept for this?

[sanjiva]

Yes same concept. However, because i18n is special and different (e.g. its not usually changed by the system admin deploying a solution), I feel we need a platform defined standard way to locate local specific resource bundles.

For example, a product built in Ballerina may ship with multiple language resource files but someone may need to add another for their language. That kind of thing is different from changing the default port.

Anyway lets wait for the platform level design part and see how it will play out. Maybe we don't need specialization for i18n.

[jclark]

Basic idea is to declare a module-level variable/const as configurable. This means that it can be initialized with a value supplied from outside the program.

Things to think through:

• when does the initialization happen relative to normal module initialization
• how does this interact with the module init function
• should make this work for default values of records/object fields
• should make make this work for annotations
• how to distinguish compile-time vs run-time configurability (maybe also link-time)
• how does this relate to public?
• what parts of this need to be in the platform vs in the language
  • should it be configurable or @configurable?
  • mechanism to supply values should be part of platform

[jclark]

There also ought eventually to be a way to specify constraints on the valid values (e.g. must be positive). This relates to #239 and #238. I think this can be handled within the framework of #238 by saying that the constraint is part of the type, and one of the language-defined points at which these constraints are checked is as part of configuration.

[afkham]

Once we have this feature, we need to make the code 2 cloud stuff also work seamlessly with this. For example, a configurable item should be configurable via a Kubernetes ConfigMap.

[jclark]

For Swan Lake, we are doing this in a very simplified form as follows.

There is a new configurable keyword, which can be applied to a module-level variable declaration that has an initializer. For example:

configurable int x = 1;

The semantics of configurable are to transform the semantics of the initializer expression. At the point where the expression E would normally be evaluated, the transformed semantics are:

• see if a value has been configured
• if so, don't evaluate E and use this value instead
• otherwise, evaluate E as usual

We allow a special value of ? in place of the expression in conjunction with configurable, meaning that a value must be configured. Program execution semantics is extended to check that the required configured values are provided for every module and terminate execution if they are not.

Configured values would always a subtype of readonly&anydata.