🐛 Bug/autoscaling datetime failure

[pcrespov]

What do these changes do?

Fixes unhandled exception in the sort predicate when autoscaling services is sorting tasks. This issue was found in production AWS logs

  File "/home/scu/.venv/lib/python3.9/site-packages/servicelib/background_task.py", line 33, in _periodic_scheduled_task
    await task(**task_kwargs)
  File "/home/scu/.venv/lib/python3.9/site-packages/servicelib/redis_utils.py", line 30, in wrapper
    return await func(*args, **kwargs)
  File "/home/scu/.venv/lib/python3.9/site-packages/simcore_service_autoscaling/dynamic_scaling_core.py", line 552, in cluster_scaling_from_labelled_services
    cluster = await _scale_cluster(app, cluster)
  File "/home/scu/.venv/lib/python3.9/site-packages/simcore_service_autoscaling/dynamic_scaling_core.py", line 515, in _scale_cluster
    pending_tasks = await utils_docker.pending_service_tasks_with_insufficient_resources(
  File "/home/scu/.venv/lib/python3.9/site-packages/simcore_service_autoscaling/utils/utils_docker.py", line 160, in pending_service_tasks_with_insufficient_resources
    sorted_tasks = sorted(
  File "/home/scu/.venv/lib/python3.9/site-packages/simcore_service_autoscaling/utils/utils_docker.py", line 165, in <lambda>
    to_datetime(
  File "/home/scu/.venv/lib/python3.9/site-packages/servicelib/docker_utils.py", line 12, in to_datetime
    return datetime.strptime(
  File "/usr/local/lib/python3.9/_strptime.py", line 568, in _strptime_datetime
    tt, fraction, gmtoff_fraction = _strptime(data_string, format)
  File "/usr/local/lib/python3.9/_strptime.py", line 352, in _strptime
    raise ValueError("unconverted data remains: %s" %
ValueError: unconverted data remains: Z
ERROR:servicelib.background_task:Unhandled exception: unconverted data remains: Z

Once merged, i will prepare a hotfix to production ASAP

Related issue/s

• Addresses incident https://s4llite.statuspage.io/incidents/r810b51d1l88
  • Issue detected in AWS production (not in staging) in the autoscaling service and reported by @mrnicegyu11

How to test

• Includes tests that reproduce the error in the log

[mrnicegyu11]

thanks a lot!

[codecov]

Codecov Report

Merging #3976 (6eee91e) into master (5878e8f) will decrease coverage by 4.9%.
The diff coverage is 100.0%.

@@            Coverage Diff            @@
##           master   #3976      +/-   ##
=========================================
- Coverage    84.5%   79.7%    -4.9%     
=========================================
  Files         943     625     -318     
  Lines       40665   26408   -14257     
  Branches      848     447     -401     
=========================================
- Hits        34383   21053   -13330     
+ Misses       6072    5223     -849     
+ Partials      210     132      -78     

Flag	Coverage Δ
integrationtests	77.1% <ø> (+11.0%)	⬆️
unittests	78.9% <100.0%> (-3.0%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
.../simcore_service_autoscaling/utils/utils_docker.py	100.0% <100.0%> (ø)

... and 375 files with indirect coverage changes

[codeclimate]

Code Climate has analyzed commit 6eee91e and detected 0 issues on this pull request.

View more on Code Climate.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[sanderegg]

thanks for fixing this one. pretty nasty. still I wonder why this happened now and not before

[sanderegg]

thanks for fixing this one. pretty nasty. still I wonder why this happened now and not before