✨🐛Webserver: enable socketio horizontal scaling + ensure only 1 update goes through (⚠️ devops)

[sanderegg]

What do these changes do?

• enable python-socket-io horizontal scaling by leveraging RabbitMQ
• new wb-db-listener service that ensure only 1 service gets the DB notifications
  • this is a temporary fix until legacy dynamic services are hopefully one day gone
  • this service cannot be scaled up, since it will re-create the same issues as described in Isolate comp_tasks_listening_task runtime (inconsistent port data) #3849
  • this service cannot be restarted without breaking stuff as it is meant to run alone (@mrnicegyu11) it was malfunctioning in the webserver and needs to run alone. That is the best I can do for now.
  • it basically runs a LISTEN vs the comp_tasks table in the dB, when a comp_tasks outputs/run_hash or state changes it triggers, update the projects table in the dB and notify the connected users
  • the notifications works as is because of the horizontal scaling feature of python-socketio
• ⚠️ devops: there is a new service called wb-db-event-listener
  Bonuses
• probably fixed some flakyness in webserver [int] tests
• cleanup tests in integration
• re-enabled test_computation_start_stop

Related issue/s

• fixes Isolate comp_tasks_listening_task runtime (inconsistent port data) #3849
• relates to M1-12 Maintenance and DevOps osparc-issues#675
• fixes Flaky tests/integration/01/test_garbage_collection.py::test_t1_while_guest_is_connected_no_resources_are_removed #3672

How to test

DevOps Checklist

[codecov]

Codecov Report

Merging #4286 (8f3ea94) into master (a25c044) will increase coverage by 11.0%.
The diff coverage is 100.0%.

@@            Coverage Diff            @@
##           master   #4286      +/-   ##
=========================================
+ Coverage    72.6%   83.7%   +11.0%     
=========================================
  Files         781     468     -313     
  Lines       33173   24364    -8809     
  Branches      600     173     -427     
=========================================
- Hits        24087   20393    -3694     
+ Misses       8948    3922    -5026     
+ Partials      138      49      -89     

Flag	Coverage Δ
integrationtests	68.0% <88.3%> (+0.1%)	⬆️
unittests	80.7% <98.3%> (+3.5%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files	Coverage Δ
...rc/simcore_service_webserver/db_listener/_utils.py	100.0% <ø> (ø)
.../simcore_service_webserver/notifications/plugin.py	100.0% <ø> (ø)
.../web/server/src/simcore_service_webserver/_meta.py	100.0% <100.0%> (ø)
...erver/src/simcore_service_webserver/application.py	97.9% <100.0%> (+49.0%)	⬆️
.../simcore_service_webserver/application_settings.py	98.7% <100.0%> (+18.4%)	⬆️
...re_service_webserver/application_settings_utils.py	97.9% <100.0%> (+1.2%)	⬆️
...erver/db_listener/_db_comp_tasks_listening_task.py	87.0% <100.0%> (ø)
...rc/simcore_service_webserver/db_listener/plugin.py	100.0% <100.0%> (ø)
...src/simcore_service_webserver/garbage_collector.py	100.0% <100.0%> (ø)
...imcore_service_webserver/garbage_collector_task.py	84.7% <100.0%> (+0.3%)	⬆️
... and 3 more

... and 644 files with indirect coverage changes

[GitHK]

👍 Nice to see this one

[pcrespov]

Great job!! Some minor comments
Thanks for catching up on this topic since so far I was not able to tackle it!

[codeclimate]

Code Climate has analyzed commit 8f3ea94 and detected 0 issues on this pull request.

View more on Code Climate.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication