Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

AEMET OpenData #107755

Closed
kilem86 opened this issue Jan 10, 2024 · 12 comments · Fixed by #111092
Closed

AEMET OpenData #107755

kilem86 opened this issue Jan 10, 2024 · 12 comments · Fixed by #111092

Comments

@kilem86
Copy link

kilem86 commented Jan 10, 2024

The problem

Since I updated to the 2024.1 version of HA, the AEMET integration give me an error, It cannot be configured.

And I lost all weather details.

What version of Home Assistant Core has the issue?

2024.1.x

What was the last working version of Home Assistant Core?

2023.12.x

What type of installation are you running?

Home Assistant OS

Integration causing the issue

AEMET

Link to integration documentation on our website

No response

Diagnostics information

Logger: homeassistant.config_entries
Source: config_entries.py:406
First occurred: 22:27:37 (1 occurrences)
Last logged: 22:27:37

Error setting up entry AEMET for aemet
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/config_entries.py", line 406, in async_setup
result = await component.async_setup_entry(hass, self)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/src/homeassistant/homeassistant/components/aemet/init.py", line 37, in async_setup_entry
await aemet.select_coordinates(latitude, longitude)
File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 410, in select_coordinates
town_data = await self.get_town_by_coordinates(latitude, longitude)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 370, in get_town_by_coordinates
towns = await self.get_towns()
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 390, in get_towns
return await self.api_call("maestro/municipios")
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aemet_opendata/interface.py", line 135, in api_call
resp: ClientResponse = await self.aiohttp_session.request(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/aiohttp/client.py", line 500, in _request
with timer:
File "/usr/local/lib/python3.11/site-packages/aiohttp/helpers.py", line 735, in exit
raise asyncio.TimeoutError from None
TimeoutError

Example YAML snippet

No response

Anything in the logs that might be useful for us?

No response

Additional information

No response

@home-assistant
Copy link

Hey there @Noltari, mind taking a look at this issue as it has been labeled with an integration (aemet) you are listed as a code owner for? Thanks!

Code owner commands

Code owners of aemet can trigger bot actions by commenting:

  • @home-assistant close Closes the issue.
  • @home-assistant rename Awesome new title Renames the issue.
  • @home-assistant reopen Reopen the issue.
  • @home-assistant unassign aemet Removes the current integration label and assignees on the issue, add the integration domain after the command.
  • @home-assistant add-label needs-more-information Add a label (needs-more-information, problem in dependency, problem in custom component) to the issue.
  • @home-assistant remove-label needs-more-information Remove a label (needs-more-information, problem in dependency, problem in custom component) on the issue.

(message by CodeOwnersMention)


aemet documentation
aemet source
(message by IssueLinks)

@Noltari
Copy link
Contributor

Noltari commented Jan 11, 2024

Maybe the AEMET OpenData API was down at that time.
@kilem86 can you check again?

@kilem86
Copy link
Author

kilem86 commented Jan 15, 2024

Yes, I just checked and it is the same, sometimes the data are available, and sometimes not.

The problem is that I have some sensors related to the data, and when is not available, I lost the sensor´s information.

@kilem86
Copy link
Author

kilem86 commented Jan 15, 2024

image

@Noltari
Copy link
Contributor

Noltari commented Jan 22, 2024

Yes, I just checked and it is the same, sometimes the data are available, and sometimes not.

The problem is that I have some sensors related to the data, and when is not available, I lost the sensor´s information.

I know, but this is the expected behaviour.
If the AEMET OpenData API is unstable then the problem isn't related to the integration.

@afharo
Copy link

afharo commented Feb 12, 2024

I was on my way to report a similar issue, but I thought I'd post here because I think it's related. Please, let me know if you think I should open it separately.

My logs:

Logger: homeassistant.config_entries
Source: config_entries.py:444
First occurred: 11 February 2024 at 23:45:46 (1 occurrences)
Last logged: 11 February 2024 at 23:45:46

Error setting up entry AEMET for aemet
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 444, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/aemet/__init__.py", line 37, in async_setup_entry
    await aemet.select_coordinates(latitude, longitude)
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 399, in select_coordinates
    await self.get_conventional_observation_station_by_coordinates(
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 305, in get_conventional_observation_station_by_coordinates
    stations = await self.get_conventional_observation_stations()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 298, in get_conventional_observation_stations
    return await self.api_call("observacion/convencional/todas", fetch_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 135, in api_call
    resp: ClientResponse = await self.aiohttp_session.request(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 504, in _request
    with timer:
  File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

It's been happening to me for a few releases: every time I restart HASS the integration fails to setup due to timeouts. If I wait for HASS to settle and then reload this integration, it successfully loads.

FWIW, I can see many other timeout errors coming from other integrations (prob my Raspberry Pi is not able to handle all the startup requests and they just timeout).

I wonder if this is a matter of retries: I have the feeling (I haven't checked the code) that, if the integration fails to setup, it stays as failed forever and never retries.

I worked around this with an automation that runs 5 minutes after HASS started, checks if AEMET's state is unavailable and reloads the integration.
Then, it waits 1 minute, and checks again. It repeats this up to 60 times before giving up. Usually, on the first attempt, it goes through.

TL;DR, is it worth adding a retry logic on timeouts to the integration's setup process?

Thank you @Noltari for your integration 😇

@figorr
Copy link

figorr commented Feb 13, 2024

I was on my way to report a similar issue, but I thought I'd post here because I think it's related. Please, let me know if you think I should open it separately.

My logs:

Logger: homeassistant.config_entries
Source: config_entries.py:444
First occurred: 11 February 2024 at 23:45:46 (1 occurrences)
Last logged: 11 February 2024 at 23:45:46

Error setting up entry AEMET for aemet
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 444, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/aemet/__init__.py", line 37, in async_setup_entry
    await aemet.select_coordinates(latitude, longitude)
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 399, in select_coordinates
    await self.get_conventional_observation_station_by_coordinates(
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 305, in get_conventional_observation_station_by_coordinates
    stations = await self.get_conventional_observation_stations()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 298, in get_conventional_observation_stations
    return await self.api_call("observacion/convencional/todas", fetch_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 135, in api_call
    resp: ClientResponse = await self.aiohttp_session.request(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 504, in _request
    with timer:
  File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

It's been happening to me for a few releases: every time I restart HASS the integration fails to setup due to timeouts. If I wait for HASS to settle and then reload this integration, it successfully loads.

FWIW, I can see many other timeout errors coming from other integrations (prob my Raspberry Pi is not able to handle all the startup requests and they just timeout).

I wonder if this is a matter of retries: I have the feeling (I haven't checked the code) that, if the integration fails to setup, it stays as failed forever and never retries.

I worked around this with an automation that runs 5 minutes after HASS started, checks if AEMET's state is unavailable and reloads the integration. Then, it waits 1 minute, and checks again. It repeats this up to 60 times before giving up. Usually, on the first attempt, it goes through.

TL;DR, is it worth adding a retry logic on timeouts to the integration's setup process?

Thank you @Noltari for your integration 😇

Similar Issue since I updated Home Assistant to 2024.2.

Every time I restarted HA ... the integration fails to load ... so I need to reload. Usually one reload is enough ... but sometimes it fails several times and then stays as failed. So it needs a new reload after a while.

Another strange behaviour is ... every day at 19:00h seems to be an error recovering some data (prepitation probability, storm probability and snow probability) ... and these sensors become "unavailable" or "unknown". And they became unavailable until midnight even if I reload the integration. I though this could be a problem with the station that provides the data ... but the weird thing is they stay as "unavailable" or "unknown" forever unless I reload the integration after the midnight.

Before the midnight ... the reload didn't recover these sensors. After midnight if I reload the integration these sensors are recovered and they started to show the correct data again.

@Noltari
Copy link
Contributor

Noltari commented Feb 13, 2024

Another strange behaviour is ... every day at 19:00h seems to be an error recovering some data (prepitation probability, storm probability and snow probability) ... and these sensors become "unavailable" or "unknown". And they became unavailable until midnight even if I reload the integration. I though this could be a problem with the station that provides the data ... but the weird thing is they stay as "unavailable" or "unknown" forever unless I reload the integration after the midnight.

Before the midnight ... the reload didn't recover these sensors. After midnight if I reload the integration these sensors are recovered and they started to show the correct data again.

@figorr this is a different issue, so please create a new one instead of reporting it on an unrelated issue.

@figorr
Copy link

figorr commented Feb 13, 2024

Ok, thanks @Noltari. I will open the new issue this afternoon to see if I can see anything in the logs.

@Noltari
Copy link
Contributor

Noltari commented Feb 19, 2024

Same problem

@rdlvm no info at all provided, so please refrain from doing this in the future.
Doing this just bothers the person that's going to fix it, so the only thing you're getting with this kind of comments is a delay.

@afharo
Copy link

afharo commented Feb 20, 2024

I worked around this with an automation that runs 5 minutes after HASS started, checks if AEMET's state is unavailable and reloads the integration.
Then, it waits 1 minute, and checks again. It repeats this up to 60 times before giving up. Usually, on the first attempt, it goes through.

In case anyone is interested in the workaround, this is my definition in yaml:

alias: "[HASS] Ensure AEMET works"
description: ""
trigger:
  - platform: homeassistant
    event: start
condition: []
action:
  - delay:
      hours: 0
      minutes: 2
      seconds: 0
      milliseconds: 0
  - alias: Ensure AEMET works
    repeat:
      sequence:
        - service: homeassistant.reload_config_entry
          target:
            entity_id: weather.aemet
          data: {}
          continue_on_error: true
        - delay:
            hours: 0
            minutes: 1
            seconds: 0
            milliseconds: 0
      while:
        - condition: and
          conditions:
            - condition: state
              entity_id: weather.aemet
              state: unavailable
            - condition: template
              value_template: "{{ repeat.index < 60 }}"
              alias: Less than 60 iterations
    enabled: true
mode: single

@diamant-x
Copy link

I'm having the same issue as Original post.
This is the logs when enabling debugging in the integration:

2024-02-21 10:52:48.922 ERROR (MainThread) [homeassistant.config_entries] Error setting up entry AEMET for aemet
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/config_entries.py", line 444, in async_setup
    result = await component.async_setup_entry(hass, self)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/aemet/__init__.py", line 37, in async_setup_entry
    await aemet.select_coordinates(latitude, longitude)
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 399, in select_coordinates
    await self.get_conventional_observation_station_by_coordinates(
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 305, in get_conventional_observation_station_by_coordinates
    stations = await self.get_conventional_observation_stations()
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 298, in get_conventional_observation_stations
    return await self.api_call("observacion/convencional/todas", fetch_data)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aemet_opendata/interface.py", line 135, in api_call
    resp: ClientResponse = await self.aiohttp_session.request(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.12/site-packages/aiohttp/client.py", line 504, in _request
    with timer:
  File "/usr/local/lib/python3.12/site-packages/aiohttp/helpers.py", line 735, in __exit__
    raise asyncio.TimeoutError from None
TimeoutError

If i reload the integration it connects and retrieves it, but not permanently; after some time it fails again.
I even tried deleting it, restarting, requesting a new API key in aemet and reconfiguring it from scratch, but that won't help either. At the same time I started seeing another issue which might point towards Python 3.12 not being supported maybe? #110538 (comment)

Below my system data. This issue happens even if I restart in Safe Mode
image

Thanks.

System Information

version core-2024.2.2
installation_type Home Assistant OS
dev false
hassio true
docker true
user root
virtualenv false
python_version 3.12.1
os_name Linux
os_version 6.1.63-haos-raspi
arch aarch64
timezone Europe/Madrid
config_dir /config
Home Assistant Cloud
logged_in false
can_reach_cert_server ok
can_reach_cloud_auth ok
can_reach_cloud ok
Home Assistant Supervisor
host_os Home Assistant OS 11.5
update_channel stable
supervisor_version supervisor-2024.02.0
agent_version 1.6.0
docker_version 24.0.7
disk_total 116.5 GB
disk_used 72.5 GB
healthy true
supported true
board rpi4-64
supervisor_api ok
version_api ok
installed_addons AdGuard Home (5.0.3), ESPHome (2024.2.0), FTP (5.0.1), Firefly iii (6.1.9), Firefly iii Data Importer (1.4.3), Home Assistant Google Drive Backup (0.112.1), InfluxDB2 (0.1.0.0), WireGuard (0.10.2), Nginx Proxy Manager (1.0.1), Samba share (12.3.0), Prowlarr NAS (nightly-1.14.0.4286-ls116), Vaultwarden (Bitwarden) (0.21.1), chrony (3.0.1), motionEye (0.20.0), qBittorrent (4.6.3-4), Sonarr (4.0.1.1131), Radarr (5.2.6.8376), Lidarr NAS (2.1.7.4030), Plex Media Server (3.5.0), MariaDB (2.6.1), Bazarr NAS (1.4.1), DOODS2 (1), Castsponsorskip (0.7.2), phpMyAdmin (0.9.1), File editor (5.8.0), Traccar (0.25.0)
Dashboards
dashboards 7
resources 17
views 17
mode storage
Recorder
oldest_recorder_run 23 de diciembre de 2023, 09:42
current_recorder_run 21 de febrero de 2024, 10:52
estimated_db_size 4023.14 MiB
database_engine mysql
database_version 10.6.12

@github-actions github-actions bot locked and limited conversation to collaborators Mar 22, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

6 participants