[Lens] Filter columns not visible in the visualization for CSV download

[dej611]

Summary

Fix #121063

This PR makes the CSV download (within the Lens editor) aware of visible vs invisible columns in the visualization, filtering them for both (spreadsheet) formula check & csv download.

Because there's no easy way to directly test the fix, I've used the indirect formula check which produces a warning message to test it.

Some questions

• So far math functions as Cumulative Sum, Differences and others have been exported with their referenced columns. Is this "fix" considered as a breaking change? Or was the previous behaviour a bug?
• Would it make sense to have a "full" mode here to export the entire table?
• Should this feature be extended also to support the "hidden" feature in the datatable visualization?

Checklist

Delete any items that are not applicable to this PR.

• Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
• Documentation was added for features that require explanation or tutorials
• Unit or functional tests were updated or added to match the most common scenarios
• Any UI touched in this PR is usable by keyboard only (learn more about keyboard accessibility)
• Any UI touched in this PR does not create any new axe failures (run axe in browser: FF, Chrome)
• If a plugin configuration key changed, check if it needs to be allowlisted in the cloud and added to the docker list
• This renders correctly on smaller devices using a responsive layout. (You can test this in your browser)
• This was checked for cross-browser compatibility

[dej611]

@elasticmachine merge upstream

[dej611]

@elasticmachine merge upstream

[dej611]

@elasticmachine merge upstream

[elasticmachine]

Pinging @elastic/kibana-vis-editors @elastic/kibana-vis-editors-external (Team:VisEditors)

[flash1293]

On dashboard there’s also a csv export action which isn’t fixed by this pr. Can we find a solution to fix both places in the same way?

[dej611]

I think I've seen that already

On dashboard there’s also a csv export action which isn’t fixed by this pr. Can we find a solution to fix both places in the same way?

I had a look at that, but I see no particular way to get this type of meta information from outside Lens editor.
Maybe with #118516 some more informations can be extracted from the datatab le itself and perform this type of filtering at download level.

[flash1293]

Maybe we can introduce an expression function to filter all columns except the ones we need as a last step of the datasource expression

[dej611]

That would make it work also for the inspector download/table view.

[dej611]

@elasticmachine merge upstream

[flash1293]

That would make it work also for the inspector download/table view.

Right, this seems like a good thing - everything will be consistent

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: dac9de7
• Storybooks Preview

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
lens	1020.7KB	1021.0KB	+350.0B

History

• 💚 Build #15488 succeeded 0b2ec90
• 💔 Build #15456 failed dfdcc32
• 💔 Build #15253 failed 4d2987c
• 💔 Build #15218 failed 3a12158
• 💔 Build #14842 failed 8b1af61
• 💔 Build #14572 failed a53b32c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream