[Formatters] Add new currency and short number formatter, replacing default number and percent

[wylieconlon]

Release note

Adds new number formatters for currency and short numbers (thousands, millions), and allow # of decimals to be customized on most number formatters. The new formatters have a wider range of locales and currencies supported, and can target your browser's locale for display.

This change adds the following advanced settings:

• format:defaultLocale
• format:default_number:maxDecimals
• format:currency:defaultCurrency
• format:currency:maxDecimals

CSV export no longer formats numbers because node.js does not support localization in the version used by Kibana.

Deprecation note

Numeral.js patterns are no longer supported on the bytes and percent formatter. The following settings have been deprecated, and no longer have any affect:

• format:bytes:defaultPattern
• format:currency:defaultPattern
• format:percent:defaultPattern

While these settings are deprecated, there remain custom formatters in TSVB, Canvas, and SIEM that map to the deprecated settings.

Summary

Closes #39211 and closes #25993

The user-facing change here is that there are new number formatters that are more flexible than our previous ones which required knowledge of the numeral.js syntax. The higher level of flexibility is a pre-requisite for our usage of formatters in Lens. The new formatters are using the standard Intl.NumberFormat API, which is supported by all browsers we support.

Summary of changes:

• All number formats use a display locale. The new formatters support an option to detect the locale dynamically based on the viewer's browser, so for example a browser using de-DE for German in Germany will get the following languages: ['de-DE', 'de', 'en'] which will be used in order for formatting.

Numeral.js does not use the same technique because of its limited support, and it's one reason it's minimized in this change.

• There is a new Currency formatter. It allows selection of the top 35 currencies, but also allows typing another currency. It has a preference for default currency.

• There is a new Shorter Number formatter, which displays thousands/millions/billions:

• There is a new set of options for the Default Number formatter:

• The bytes formatter is mostly the same, but has no numeral.js pattern any more. Users who want to customize this will continue to use numeral.js

• All numeral.js format patterns will migrate to the "Custom numeral.js" formatter type for backwards compatibility. Numeral.js currently supports these locales:

Checklist

Use strikethroughs to remove checklist items you don't feel are applicable to this PR.

• This was checked for cross-browser compatibility, including a check against IE11
• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios

For maintainers

• This was checked for breaking API changes and was labeled appropriately
• This includes a feature addition or change that requires a release note and was labeled appropriately

[bmcconaghy]

@wylieconlon I love where this is going. I especially like the way the formatters explain where the defaults are coming from. I think we maybe need to think about this as part of the choose locale in browser work @Bamieh has some prototype work for. Would love to sync up on this some.

[Bamieh]

@wylieconlon I've tried pulling the code locally to play around with it but the bootstrap is failing:

Looking at the code in the PR the general approach looks good to me.

I am not sure what set of locales we want to provide for number formatting. For currencies, the top 35 currencies sound reasonable.

[wylieconlon]

@Bamieh Bootstrap fails due to some type checking issues, but the code still starts.

[wylieconlon]

This export is one way to access the same editors in Lens and Management. This is no longer the desired UX for Lens, so I will probably remove this.

[chrisdavies]

LGTM w/ tweaks.

[chrisdavies]

Should these be internationalized?

[wylieconlon]

They aren't currently for the regular number formatter, which this is based on.

[chrisdavies]

Nit: This and the if block would be tidier like this:

const locales = defaultLocale === 'detect'
  ? navigator.languages.concat(['en'])
  : [navigator.language].concat([defaultLocale, 'en']);

I had to stare at this a bit longer than necessary as it is.

[timroes]

Even more nit:

const locales = defaultLocale === 'detect'
  ? [...navigator.languages, 'en']
  : [navigator.language, defaultLocale, 'en'];

[chrisdavies]

I wonder if 3 is the right default for percentage? I almost think maybe 0 would be the right default, but maybe not.

[wylieconlon]

I think I will keep it at 3 because it's the minimum possible change- 3 was the previous default.

[timroes]

Looks good so far (want to wait for final tests/migration before approving). Left a couple of comments. Since https://github.com/elastic/kibana/pull/54240/files has been merged, I think we need to incorporate that flag also in the new field formatters in this PR.

[timroes]

Unfortunately the order of this array is currently also used for ordering the select in the UI. Since it was ordered alphabetically so far, I think we should keep this array alphabetically (by their English names) sorted (until we might in a later step properly sort them in some other place via their names).

[kibanamachine]

💚 Build Succeeded

• continuous-integration/kibana-ci/pull-request
• Commit: ff46904

History

• 💔 Build #21906 failed ecd2198
• 💔 Build #21788 failed f7bb12e
• 💔 Build #21773 failed 40f6c5c
• 💔 Build #21748 failed fbbfaf5
• 💔 Build #21716 failed b96444a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

[wylieconlon]

Updates:

• This is a breaking change as currently written. The only way to fix this is to write a migration that reads the config saved object when migrating index-patterns. This depends on Saved object migrations to allow pre-migration state collection #34996
• There is another breaking change here which is that CSV export no longer applied number formatting.

[tsullivan]

There is another breaking change here which is that CSV export no longer applied number formatting.

Reporting team has another top focus right now, which is our new platform migration. Our ongoing changes will likely cause conflicts in this PR.

We would like to better understand what kind of formatting would be left if number formatting goes away for CSV export, and if there is any, would the remainder be useful to keep? We have a discuss issue relevant to this: #55484 If the proposal is accepted, then your point about this wouldn't be a blocker. But usually we are very careful not to ship breaking changes in a minor release. If not done carefully it just makes it look like we have shipped a bug.

[rudolf]

This is a breaking change as currently written. The only way to fix this is to write a migration that reads the config saved object when migrating index-patterns. This depends on #34996

@wylieconlon Can you explain what the breaking change is and why #34996 would solve it?

Would it be possible to make these changes without a migration by "migrating" index-patterns inside the data plugin before exposing them to other plugins? I'm assuming plugins don't directly call the SavedObjectsClient to retrieve index patterns so if we have a share API for reading them that all plugins use, we could apply the migration there and at the same time have access to the config object.

[wylieconlon]

@rudolf Even if your assumption that there is only one way to read index patterns were accurate (it isn't, as far as I know), it would not solve the problem. The reason the migration needs the configuration is that the logic needs to be as follows:

1. The previous version had set a custom pattern for the percent formatter, such as 0,0%
2. The percent formatter is being used for something like system.cpu.total.pct
3. Because there was a custom format being used, we have to migrate system.cpu.total.pct to the number format with the 0,0% custom numeraljs format string.

[joshdover]

@wylieconlon Now that Github allows it, should we set this PR back to draft? Just been hanging out in review queues for a long time now.

[wylieconlon]

Despite support for the new formatting options in Node 14, I am going to close this PR as I don't expect to make the changes here.