Zabbix 5.4: method not found. Incorrect API "application"

[nomad-geek]

Describe the bug
I began rebuilding my Zabbix/Grafana monitoring again from scratch (I wanted to start over) using Docker containers.
I had everything functional and suddenly Grafana won't display some values on a Dashboard with error Incorrect API "application". I haven't been able to narrow down exactly why some values will display and some wont.
I went so far as to start over with Grafana in case I I'd caused the issue but a fresh installation of Grafana and the plugin result in the same issue.

Expected behavior
Grafana should show all widgets.

Screenshots
Non-functional default Zabbix Dashboard

Error from non-functional default Zabbix Dashboard

Other default dashboard still functional: (Github won't let me upload it for some reason)
https://imgur.com/a/llyY3Kp

Network data
{"datasourceId":1,"method":"application.get","params":{"output":"extend","hostids":[]}}

{"error":"Internal Server Error","message":"Method not found. Incorrect API \"application\"."}

Software versions

Grafana	Zabbix	Grafana-Zabbix Plugin
7.5.4	5.4.0rc1	4.1.4

[nomad-geek]

To be clear, I'd spent all day yesterday building a dashboard with various data on it and everything functioned fine. Near the end of that work while I was editing a widget I started receiving this error and I haven't been able to figure out what is causing it.

[alexanderzobnin]

I checked the latest Zabbix API docs and application.get seems to be there. So I guess it's Zabbix bug (rc1 is not a stable version, so this is more than probable).

[i-ky]

@alexanderzobnin, it's not a bug, it's a feature: ZBXNEXT-2976. Likely changes have not been documented yet. But it's clear that applications are gone.

[catmater]

Confirm, critical implementing from Zabbix, breaking functionality this plugin completely. And still exist in 5.4.0 beta 3

[Codixer]

Anyone got any pointers how to downgrade zabbix? Through apt or something

[i-ky]

@thecoolone123, it is not possible if you don't have DB backup. During major upgrades Zabbix server and proxies apply database upgrade patches which are irreversible. You can downgrade Zabbix server binary through package manager like you usually do, but it will likely complain about DB version mismatch.

[Codixer]

Any pointers on that, i've not been sucessfull going back to 5.3-beta2

[alexanderzobnin]

It looks like application API was deleted in 5.4 https://support.zabbix.com/browse/ZBXNEXT-2976

[catmater]

Yes, and long time ago. Now Zabbix 5.4.0 Beta 3 are so-so stable build, but this "improvement" completely break you great plugin functionality. And no way to downgrade Zabbix (only revert to backup dump of mysql base).

[i-ky]

Yes, and long time ago.

..and in 100% secrecy. Zabbix does not share design documents and still has not updated documentation (proof). Please understand that @alexanderzobnin can't start working on the fix before new API is documented. I encourage you to contact Zabbix and share your opinion on their policies.

[nomad-geek]

Thanks for the replies, everyone - I'm sort of glad it isn't just me - I thought I was going crazy.
I appreciate your work, @alexanderzobnin - I eagerly but patiently await your fix [for Zabbix's trash fire.]
I've fallen back to the adequate but far less good Zabbix Dashboards for the time being.

cheers!

[catmater]

Zabbix 5.4.0-0.7rc1 - the same error.

[catmater]

Grafana 8.0.0 beta1 and Zabbix 5.4.0 0.8rc2 - Error still exist.

[GregoryOrciuch]

Hi, now the stuff it's gonna explode (because 5.4.0 went GA today in docker hubs). I stepped into this bug because my automatic docker script upgraded today to zabbix to 5.4 .
Shame on zabbix to release undocumented changes which brakes backward compatibility.

[nomad-geek]

Welcome to the party, everyone. Have some popcorn while we wait for Zabbix.
Edit: 👇 Progress made!

[i-ky]

Well, at least changes are documented now:

• Upgrade Notes
• API Changes
• Item Tag

[roysbike]

Zabbix 5.4.0 Error still exist.(

[GregoryOrciuch]

Not sure if known, but I discovered a hack in Grafana to resurrect broken graphs. You edit your graph and then remove the content of "Application" field, leaving the item and the rest of them untouched, then it started to present graph. Discovered accidentialy because one of graphs had the application field empty and that one worked well after zbx 5.4.0 kicked in yesterday.
Link to evidence.

[mfitzgerald2]

I can confirm this workaround works, I am running zabbix 5.4.0 and Grafana v7.4.2. You still get the "Method not found. Incorrect API "application"." error but at least you can see the data!

EDIT: It appears in that case, I was missing a data block, the error is completely gone!

[StefanSa]

privjet @alexanderzobnin
thanks for the quick fix, but the error has not yet been completely resolved (please reopen this Ticket).
Now i get all values again, but i get this error message when I refresh.

[alexanderzobnin]

@StefanSa you obviously cannot use Applications since it were removed in Zabbix. This is only compatibility fix, Applications will be replaced by tags in next plugin release.

[StefanSa]

Spasibo @alexanderzobnin
thank you for the clarification

[brunoscota]

Thank you @alexanderzobnin. I'm also waiting for this.

[tomukasteris]

I did update plugin to the latest version but still doesn't work.

[tomukasteris]

I have managed to fix by deleting data source and adding again with the same name.

[muhammadanees83]

Not sure if known, but I discovered a hack in Grafana to resurrect broken graphs. You edit your graph and then remove the content of "Application" field, leaving the item and the rest of them untouched, then it started to present graph. Discovered accidentialy because one of graphs had the application field empty and that one worked well after zbx 5.4.0 kicked in yesterday.
Link to evidence.

thanks bro..... this trick is working

[joachim-vertommen]

@StefanSa you obviously cannot use Applications since it were removed in Zabbix. This is only compatibility fix, Applications will be replaced by tags in next plugin release.

Hello, do you have an estimate when the next plugin release will arrive?
For us this is a blocking factor at the moment to upgrade Zabbix because it will break our Grafana graphs completely.

Thanks.

[akibukis]

Zabbix 5.4.2 on Ubuntu 20.10, Grafana v8.0.5 (cbb2aa5001).

Same problem:
Method not found. Incorrect API "application".

Are there any plans to finally fix this? It breaks many Grafana dashboards. :/

[akibukis]

UPDATE:

Upgraded alexanderzobnin-zabbix-app to v4.1.5 in Grafana and everything works without any problems!
Thank you, guys!

[akibukis]

UPDATE #2

Although all of my Grafana panels/graphs work, I am unable to query 'Application' items:

[i-ky]

@akibukis, see #1188 (comment)

[hexicans]

Hello

Same issue