V1.1.0: RateString bounding

[perlboy]

Description

Following release to the Slate site of V1.1.0 yesterday we have now initiated analysis of changes using the diff previously provided in #56.

We note the change in bounding for RateString whereby the removal of the -100% to 100% bound has been removed, specifically this sentence: "A number constrained to the range [-1.0, 1.0]"

For clarity, this bound was originally in place prior to support for negative interest rates as 0 -> 100% and consequently was implemented (by me) in a way with the least impact on the existing Standard among some opposition to supporting negative rates in the first place....

This aside and with an understanding that some interest rates may be higher than 100% we recommend a "reasonable" bound be put in place. In our experience when processing existing payloads available in market we have found this bound has been a tell-tale sign of errors within bank payloads where invalid rates are published due to division errors.

Essentially, through this change there is likely to be conformant but highly unlikely payloads made available due to human error.

Area Affected

RateString definition

Change Proposed

Reinstate a "reasonable" bound for RateString. If -100% to 100% is unsuitable then maybe -200% to 200%. In our experience the most common incorrectly specified rate string we have identified has been something like 20.49% being described as a RateString of 20.49 rather than 0.2049.

Additionally, implement a similar rule to AmountString which requires at least 2 decimal places as percentages are, by definition, fractions of 100.

With the change made in V1.1.0 such an invalid payload would pass conformance.

Alternative

If RateString is going to represent nothing more than a number then there appears to be minimal value in forcing the payload to be a string. As such a conversion to a native number, potentially with a Min/Max value bound would be aligned to international specifications.

[perlboy]

Further on this it is noted that RateString does not specify a specific number of decimal places required. This appears to be an error as percentages by definition are fractions, have added additional change to original description.

[anzbankau]

@perlboy , regarding the number of decimal places, given the data is for machine consumption rather than human consumption, the standards should not enforce trailing zeros for RateString - even when it's (unusually) a string rather than a numeric type in the standards. Percentages are for presentation to users and should not influence storage or transmission data formats. An interest rate (really a percentage) of 3.25% is represented as '0.0325' but 10% should be represented as '0.1' rather than '0.10' or '0.1000'. There should only be a maximum number of decimals positions i.e. when more would be unnecessary for precision.

[perlboy]

@perlboy , regarding the number of decimal places, given the data is for machine consumption rather than human consumption, the standards should not enforce trailing zeros for RateString - even when it's (unusually) a string rather than a numeric type in the standards.

@anzbankau If it was originally designated as a number I would agree however RateString has been specifically created as A string representing an interest rate. and therefore contextual considerations are important.

Percentages are for presentation to users and should not influence storage or transmission data formats. An interest rate (really a percentage) of 3.25% is represented as '0.0325' but 10% should be represented as '0.1' rather than '0.10' or '0.1000'. There should only be a maximum number of decimals positions i.e. when more would be unnecessary for precision.

Once again, if rates were being described as raw number's I would agree but, it hasn't. The formatting of AmountString specifically states a minimum of two decimal places which adds value to supporting a custom data type, RateString appears to be no different. Without these specific constraints there is little value in designating these as custom data types in the first place. That is to say that if the logic above was to be followed AmountString and RateString would be abandoned because they would provide no additional precision then the use of a native number.

As I understood it at the time the *String types were specifically introduced to allow for these unique presentation requirements. The upside of specific formatting specification is that automated detection of invalid payloads is somewhat simplified.

This change request comes about to ensure that the custom data types have continued usefulness. The alternative would be a change request to abandon AmountString and RateString in favour of number, a proposal previously raised internally by the now abandoned DSB Engineering team. Such reversion to number I believe would also be aligned to the UK specifications and so I've added it as an Alternative in the original issue description.

[perlboy]

How can no change be supported when there are already clear issues with conformance in even existing payloads (see #162)?

As stated originally:

In our experience the most common incorrectly specified rate string we have identified has been something like 20.49% being described as a RateString of 20.49 rather than 0.2049.

Perhaps the DSB should consider those impacted more than those releasing invalid (but currently technically valid) payloads.

[perlboy]

Preamble: Biza currently has 42 open issues within Standards Maintenance. In an effort to optimise our own backlog we are closing those which have no actual response from the DSB. They may be reopened at a later time or referenced when the issues are highlighted by third parties.

No response has been received on this and the DSB does not appear to have shown interest in resolving it.