Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Kibana 7.12, 7.13, 7.14 and 7.15 support #197

Closed
tonykoval opened this issue Mar 29, 2021 · 57 comments
Closed

Add Kibana 7.12, 7.13, 7.14 and 7.15 support #197

tonykoval opened this issue Mar 29, 2021 · 57 comments

Comments

@fbaligand
Copy link
Owner

I have done a first test on Kibana 7.12, and unfortunately, there are breaking changes in Kibana 7.12.
So I have to change the plugin code to be compatible with Kibana 7.12.

@mvenukadasula
Copy link

Do you think it's going to take time for this version? Waiting to try this soon and buy you a cup of coffee (or drink). Saved Search / Data Table (aggregation) have been our pain point for long time.

@fbaligand
Copy link
Owner

Difficult to estimate time, as I have to find what’s wrong, find a solution and then pass to the next problem.
And all of that stuff time is quite impossible to predict.

@mvenukadasula
Copy link

Thanks Fabien.

@bitonp
Copy link

bitonp commented Apr 21, 2021

Hi Fabien.. such a great product, then elastico throw 7.12 Kibana at us. Doh!.
However, I echo @mvenukadasula .. hope you can find time to fix it soon...and keep up the great work. Will also buy you 6 months of coffee

@ppreyas
Copy link

ppreyas commented Apr 22, 2021

Hello Fabien, we greatly use enhanced table and would absolutely love having it for 7.12. Hope you can fix it soon.

@iamjosh007
Copy link

Hello Fabien, Appreciate your efforts as we would like to see this one for 7.12.

@fbaligand
Copy link
Owner

Hi,
I currently work on kibana 7.12 support.
I have to say that it’s not easy because there is a major breaking change that requires a lot of stuff.

But be sure that I work on it.

@tonykoval tonykoval changed the title Add Kibana 7.12 support Add Kibana 7.12 and 7.13 support Jun 1, 2021
@tonykoval
Copy link
Author

@fbaligand Do you plan also support 7.13?

@fbaligand
Copy link
Owner

Yes

@isac71
Copy link

isac71 commented Jun 12, 2021

@fbaligand Any ETA on support for kibana 7.12 ?

@pawelniedzwiecki83
Copy link

@fbaligand may I attach to the above question? Are any chances for supporting 7.12 next month or its more long term possibility ?

@carnicod
Copy link

Hi Fabien, I' interesting too :-)

@diliu2
Copy link

diliu2 commented Aug 5, 2021

Hi Fabien, just like the above posts, our team heavily uses enhanced table for our kibana visualizations, and recently we have moved to 7.13 and would really appreciate you for supporting 7.13 as well.

@fbaligand
Copy link
Owner

fbaligand commented Aug 5, 2021

I fully understand your interest for a new version compatible with kibana 7.12, 7.13 and even 7.14.

On my side, I’m interested by anyone ready to contribute to bring support for new kibana versions (first 7.12). I’m ready to collaborate with any volunteer.

@tonykoval tonykoval changed the title Add Kibana 7.12 and 7.13 support Add Kibana 7.12, 7.13 and 7.14 support Aug 9, 2021
@zurtix
Copy link

zurtix commented Aug 12, 2021

With version 7.14 it appears that import { Schemas } from '../../../src/plugins/vis_default_editor/public'; no longer contains the schema.js file among other components that are found in previous version of Kibana.

@fbaligand
Copy link
Owner

Thanks for the information @Surphix
One of the several breaking changes at each new minor version :(

@hoberhol
Copy link

hoberhol commented Sep 9, 2021

Hi Fabien,
Any update on this one?
I'm aware that you are working on this project in your spare time and I'm really thankful for the great work.
My problem is now that we are heavily using enhanced table in almost any of our reports.
In the elastic cloud it is now no longer possible to create instanced below 7.13.4, therefore I'm stuck and cannot proceed from here.
Thanks a lot for your support and again we really appreciate your work and love enhanced table!
Kind regards,
Hans

@dminovski0
Copy link

Is it possible for the enhanced table plugin to be integrated into the official Kibana release аnd its functions to be added to Lens or the data table visualization?

@tonykoval tonykoval changed the title Add Kibana 7.12, 7.13 and 7.14 support Add Kibana 7.12, 7.13, 7.14 and 7.15 support Sep 30, 2021
@cpiment
Copy link
Contributor

cpiment commented Oct 23, 2021

On my side, I’m interested by anyone ready to contribute to bring support for new kibana versions (first 7.12). I’m ready to collaborate with any volunteer.

Hi @fbaligand. Since this plugin has been proven very important for our users, and it can be a stopper for our future 7.x updates, I'm would like to help to get it compatible with 7.12.1

I have only worked with React and Angular at personal projects and in a "hacky" way, not professional at all, but I'm eager to learn.

I have managed to start the plugin in Kibana 7.12.1 with the changes of this commit, but when I try to create a new visualization, before the "Choose source" screen appears, the plugin crashes with this error:

Error: state.data.aggs is undefined
DefaultEditorAggGroup/group<@http://localhost:5601/zmk/9007199254740991/bundles/plugin/visDefaultEditor/visDefaultEditor.chunk.0.js:1844:77
mountMemo@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:381299:19
useMemo@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:381618:16
useMemo@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:404029:21
DefaultEditorAggGroup@http://localhost:5601/zmk/9007199254740991/bundles/plugin/visDefaultEditor/visDefaultEditor.chunk.0.js:1844:70
renderWithHooks@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:380726:27
mountIndeterminateComponent@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:383260:13
beginWork$1@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:384628:16
callCallback@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:364802:14
invokeGuardedCallbackDev@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:364851:16
invokeGuardedCallback@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:364906:31
beginWork$$1@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:390246:28
performUnitOfWork@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:389164:12
workLoopSync@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:389137:22
performSyncWorkOnRoot@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:388736:11
flushSyncCallbackQueueImpl/<@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:376665:24
unstable_runWithPriority@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:428111:12
runWithPriority$2@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:376615:10
flushSyncCallbackQueueImpl@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:376660:24
flushSyncCallbackQueue@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:376648:3
batchedUpdates$1@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/kbn-ui-shared-deps.js:388858:7
processQueue@http://localhost:5601/zmk/9007199254740991/bundles/kbn-ui-shared-deps/[email protected]:139620:76

I thought that I could advance to see what happens trying to skip the "undefined" error with this line, but that doesn't solve the problem either. As I said, the plugin crashes before showing the "Choose source" screen, and I think that the "undefined" error happens because the user haven't selected any index pattern or saved search. Any ideas how can I force the plugin to show that screen?

Thanks for the great job


Edit: What I'm trying now is to force Kibana to show the "Choose Sources" screen. I'm comparing the enhanced-table plugin with the default Kibana table plugin and copying/pasting code trying to adapt it. I'm (maybe wrongly) assuming that the responsible of showing the screen is the missing toExpressionAst in the enhancedTableVisTypeDefinition. It seems that other plugins have an to_ast.tsfile where that function is defined and that function references the esaggs and index patterns.

My unsuccessful code right now is in this commit. Despite all the changes, and that I have modified the setup plugin function, the error continues being the same, so maybe I'm wasting my time with this approach. What do you think?

@cpiment
Copy link
Contributor

cpiment commented Oct 25, 2021

I think I have found the change that is impacting the enhanced-table plugin: elastic/kibana#88317

It was merged in Kibana 7.12 and is related with the toExpressionAst I mentioned and some more things:

This mainly cleans up the visualizations plugin, other dependant plugins changes updated to follow API changes.

  • VisOptionsProps type renamed into VisEditorOptionsProps and moved into visualizations plugin to avoid cross references;
  • removed the common visualization renderer - each visualization should have its own renderer registered via expressions.registerRenderer; dependant components Visualization & VisualizationChart removed, ExprVis class removed;
  • removed the common visualization expression function - each visualization should have its own expression function registred via expressions.registerFunction;
  • removed the buildPipeline function; replaced with ast expression -> see src/plugins/visualizations/public/embeddable/to_ast.ts;
  • removed createReactVisualization from visualizations setup contract; dependant ReactVisController & ReactVisType were removed too;
  • VisTypeDefinition type were enhanced to have stricter types for registering a visualization;
  • the custom visualization plugin (test/plugin_functional/plugins/kbn_tp_custom_visualizations) was updated with a custom expression function and renderer;

Dev docs

There is no more possibility to use a common visualization expression function and a common visualization renderer to retrieve data and render your custom visualization.

To register a custom visualization you need:

  1. register a visualization type using visualizations.createBaseVisualization( config ) function, where the config should be a type of VisTypeDefinition. VisTypeDefinition is documented (see src/plugins/visualizations/public/vis_types/types.ts) for usability.
  2. register an expression function definition to handle your custom expression using expressions.registerFunction( functionDefinition ), where the functionDefinition describes your expression parameters.
  3. register an explicit renderer for your visualization using expressions.registerRenderer( rendererDefinition ), where the rendererDefinition is type of ExpressionRenderDefinition.

I will try to review all this information and will get back to you with my findings.

@jingjunchao
Copy link

Master, how long is it expected to support 7.13

@fbaligand
Copy link
Owner

Hi @cpiment,

First, thanks for having started the work to make enhanced-table plugin compatible with Kibana 7.12.
I can confirm that "toExpressionAst" function is the key point in this migration.
This new method implies a truly structuring change.
And that's the reason why I still did not provide yet the release for Kibana 7.12.
Concerning the way to do, personally, I always compare the changes in "classic" data table between previous and next minor version. It helps a lot to know what to change in enhanced table plugin.
So concretely, I advice you to see the changes between 7.11 and 7.12 branches on elastic/kibana repository.

@fbaligand
Copy link
Owner

@dminovski0
I'm sorry but enhanced table will probably never be integrated as is in Kibana core, especially since Lens is a very different plugin.
That said, in last Kibana versions, there are some new big features in Lens that are very similar to enhanced table:

@dminovski0
Copy link

@fbaligand Thanks for the reply. The formulas offer flexibility and can do some of the Enhanced Table's operations.

Related to this, I opened a feature request on Kibana's GitHub about them adding new functionality to the Lens formulas. It will be equivalent to calculating the total hits count. A member of Elastic wrote that this would represent "overall count" in the Formulas, but they haven't added it yet. They linked a previously opened topic for another enhancement. If they implement that enhancement, it will also cover the total hits feature.
elastic/kibana#115770 (comment)

@fbaligand
Copy link
Owner

Thanks for the info @dminovski0

@jappereuling
Copy link

Really nice if you guys would get this to work!! Ill buy you a couple of drinks

@fbaligand
Copy link
Owner

Thanks @cpiment for your amazing work!
As I understand it, it seems that you did all the changes required so that enhanced table plugin is fully compatible with kibana 7.12!

I will review the code and say you if there are some changes to do.

That said, a remaining point is packaging and installation on a standalone kibana (not in dev mode).
So could you:

  • call yarn compile
  • call yarn build
  • Install generated package (in build directory) on a standalone kibana instance (using kibana-plugin tool)
  • test the plugin on this kibana instance

@jappereuling
Copy link

jappereuling commented Nov 11, 2021

If needed / useful : we can test in an acceptance env with quite some data in it. Happy to help

@cpiment
Copy link
Contributor

cpiment commented Nov 11, 2021

That said, a remaining point is packaging and installation on a standalone kibana (not in dev mode). So could you:

* call yarn compile

* call yarn build

* Install generated package (in build directory) on a standalone kibana instance (using kibana-plugin tool)

* test the plugin on this kibana instance

I have had to modify the package.json because "plugin_helpers.js" was not in the same location, but after that I was able to build a working version of the plugin ready to deploy in Kibana 7.12.1

enhancedTable-7.12.1.zip (Edit 2021-11-21) Replaced by the version in this comment

Feel free to download and test it but it is not a final version and it has not been thoroughly tested. For example, I have noticed that if the table has too many columns there is no horizontal scrollbar displayed to navigate to the columns at the far right.

@cpiment
Copy link
Contributor

cpiment commented Nov 21, 2021

Hi everyone,

After @fbaligand and I made some changes, the errors that we found in the previous version of the plugin are fixed. I don't know if there is any error left, but if any of you want to try the plugin and tell us if you discover more errors that would be great.

enhancedTable-7.12.1.zip

The same disclaimer applies: you can download this file and test it but it is not a final version and it has not been thoroughly tested. Use at you own discretion.

Thanks everyone!

@rful011
Copy link

rful011 commented Nov 26, 2021

Happy to test this but I can't figure out how to install a plugin from a file. Whatever I do it tries to down load

@cpiment
Copy link
Contributor

cpiment commented Nov 26, 2021

Hi @rful011

You can use this command:

bin/kibana-plugin install https://github.com/fbaligand/kibana-enhanced-table/files/7576138/enhancedTable-7.12.1.zip

Or if you prefer to use your downloaded file, if the location of the file is /var/tmp/enhancedTable-7.12.1.zip you can use:

bin/kibana-plugin install file:///var/tmp/enhancedTable-7.12.1.zip

Link to Kibana docs

@rful011
Copy link

rful011 commented Nov 26, 2021 via email

@fbaligand
Copy link
Owner

Well, if you download the zip package on your computer into /var/tmp directory and launch this command:
bin/kibana-plugin install file:///var/tmp/enhancedTable-7.12.1.zip

Then it should work

@rful011
Copy link

rful011 commented Nov 27, 2021 via email

@rful011
Copy link

rful011 commented Nov 27, 2021

sigh...

rful011@secmgrtst01:~$ sudo  /usr/share/kibana/bin/kibana-plugin install file:///home/rful011/enhancedTable-7.12.1.zip 
Found previous install attempt. Deleting...
Attempting to transfer from file:///home/rful011/enhancedTable-7.12.1.zip
Transferring 3265366 bytes....................
Transfer complete
Retrieving metadata from plugin archive
Extracting plugin archive
Extraction complete
Plugin installation was unsuccessful due to error "Plugin enhancedTable [7.12.1] is incompatible with Kibana [7.14.0]"

You clearly need to be running 7.12.1...

@fbaligand
Copy link
Owner

Well, the name of the package is quite clear :)
And You have to know that each minor kibana version brings several breaking changes for kibana plugins. That’s the reason why enhanced table for kibana 7.12 is still not out.

@rful011
Copy link

rful011 commented Nov 27, 2021 via email

@tonykoval tonykoval changed the title Add Kibana 7.12, 7.13, 7.14 and 7.15 support Add Kibana 7.13, 7.14 and 7.15 support Dec 8, 2021
@fbaligand
Copy link
Owner

Hi,

I'm glad to announce that enhanced-table plugin v1.12.0 is out, with Kibana 7.12 support!
https://github.com/fbaligand/kibana-enhanced-table/releases/tag/v1.12.0

Thanks a lot @cpiment for his awesome work on this enhancement!

@fbaligand
Copy link
Owner

New great news: After tests, Kibana 7.13 doesn't need code changes for the plugin!
So I'm glad to announce that enhanced-table plugin v1.12.0 brings also support for Kibana 7.13!

You can get packages for Kibana 7.13 here:
https://github.com/fbaligand/kibana-enhanced-table/releases/tag/v1.12.0

@fbaligand fbaligand changed the title Add Kibana 7.13, 7.14 and 7.15 support Add Kibana 7.12, 7.13, 7.14 and 7.15 support Dec 10, 2021
@tonykoval tonykoval changed the title Add Kibana 7.12, 7.13, 7.14 and 7.15 support Add Kibana 7.12, 7.13, 7.14, 7.15 and 7.16 support Dec 10, 2021
@cpiment
Copy link
Contributor

cpiment commented Dec 11, 2021

Today I have tried to run the plugin in Kibana 7.14.2 but it failed... Let's see if I can find what needs to be changed!

@fbaligand
Copy link
Owner

Hi all,

Great news: enhanced-table v1.12.1 is out and brings support for Kibana 7.14!
Thanks again for @cpiment for his work on this enhancement!

All packages for Kibana 7.14 are available here:
https://github.com/fbaligand/kibana-enhanced-table/releases/tag/v1.12.1

@jappereuling
Copy link

Very nice, thanks and respect! We will try to test asap but kind of low staffed due to holidays ;-)

@fbaligand
Copy link
Owner

enhanced-table v1.12.2 is out and brings support for Kibana 7.15!
Once again, thanks @cpiment for his work!

All packages for Kibana 7.15 are available here:
https://github.com/fbaligand/kibana-enhanced-table/releases/tag/v1.12.2

@manuelbs88
Copy link

It'd great to see a new version supporting Kibana 7.16.2.
Congratulations on your work in this plugin that is quite helpful @fbaligand , keep rocking!!

@fbaligand
Copy link
Owner

It is on the roadmap, but unfortunately not so easy.
There is a separate issue for Kibana 7.16 support : #237
I close this issue in favor of #237
Please follow this one.

@fbaligand fbaligand changed the title Add Kibana 7.12, 7.13, 7.14, 7.15 and 7.16 support Add Kibana 7.12, 7.13, 7.14 and 7.15 support Jan 28, 2022
@fbaligand
Copy link
Owner

Enhanced Table plugin version 1.13.2 is out, with support for all these Kibana versions: 7.16, 7.17, 8.0, 8.1 and 8.2!

You can get the packages here:
https://github.com/fbaligand/kibana-enhanced-table/releases/tag/v1.13.2

@rful011
Copy link

rful011 commented Jun 8, 2022 via email

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests