Storage Analytics never completes

[ObadaS]

Hello,
It seems like the storage analytics never completes once started, and might not even trigger the soft timeout ?

This is the result of docker compose logs --no-log-prefix site_worker | grep "analytics"

  . analytics.tasks.create_storage_analytics_snapshot
  . analytics.tasks.delete_orphan_files
  . analytics.tasks.reset_computed_storage_analytics
  . apps.analytics.tasks.create_storage_analytics_snapshot
  . apps.analytics.tasks.delete_orphan_files
  . apps.analytics.tasks.reset_computed_storage_analytics
  . analytics.tasks.create_storage_analytics_snapshot
  . analytics.tasks.delete_orphan_files
  . analytics.tasks.reset_computed_storage_analytics
  . apps.analytics.tasks.create_storage_analytics_snapshot
  . apps.analytics.tasks.delete_orphan_files
  . apps.analytics.tasks.reset_computed_storage_analytics
[2024-11-08 11:26:51,656: INFO/MainProcess] Received task: analytics.tasks.create_storage_analytics_snapshot[88f27753-478f-433b-8e5a-0b8e7cb7f854]  
[2024-11-08 11:26:51,661: INFO/ForkPoolWorker-2] Task create_storage_analytics_snapshot started
[2024-11-10 02:00:00,251: INFO/Beat] Scheduler: Sending due task create_storage_analytics_snapshot (analytics.tasks.create_storage_analytics_snapshot)
[2024-11-10 02:00:00,461: INFO/MainProcess] Received task: analytics.tasks.create_storage_analytics_snapshot[ad36d01a-efcd-4506-bda7-b6ae57362a17]  
[2024-11-10 02:00:00,464: INFO/ForkPoolWorker-11] Task create_storage_analytics_snapshot started
  . analytics.tasks.create_storage_analytics_snapshot
  . analytics.tasks.delete_orphan_files
  . analytics.tasks.reset_computed_storage_analytics
  . apps.analytics.tasks.create_storage_analytics_snapshot
  . apps.analytics.tasks.delete_orphan_files
  . apps.analytics.tasks.reset_computed_storage_analytics

The last successful analytics on the production server dates from September 2, 2024.

I launched the analytics on the test server, I will post the results here when it's done, but it does show that the last successful run was November 10, 2024.

[OhMaley]

That is very strange!

Here is an unsorted non exhaustive list or random ideas where the problem could be:

• The logger does not display anything
• The task is queued but never picked up by the site worker (may be too many tasks in the queue)
• The server memory is not big enough to handle the analytics
• The soft limit is never triggered (could be a known bug on the celery version we use)

We can discuss that during our meeting