Fix Controller: pending messages are stuck in the scheduled map

[co42]

Motivation

When using the Controller. If some messages are put in the pending queue using pop_queue_message_into_pending they are not removed from the scheduled map and will never be.

The message in pending will be reconciled later, but all matching messages that follow will not.

Solution

When adding a message to the pending queue remove it from the scheduled map (like it's done in poll_pop_queue_message).

May fix #1322

[codecov]

Codecov Report

Merging #1324 (b325981) into main (9c81f1f) will increase coverage by 0.1%.
The diff coverage is 100.0%.

Additional details and impacted files

@@           Coverage Diff           @@
##            main   #1324     +/-   ##
=======================================
+ Coverage   72.4%   72.4%   +0.1%     
=======================================
  Files         75      75             
  Lines       6343    6351      +8     
=======================================
+ Hits        4586    4595      +9     
+ Misses      1757    1756      -1     

Files	Coverage Δ
kube-runtime/src/controller/runner.rs	95.2% <100.0%> (+0.2%)	⬆️
kube-runtime/src/scheduler.rs	97.2% <100.0%> (+0.1%)	⬆️

... and 1 file with indirect coverage changes

[einarmo]

Yep. This is what I also found, and it does fix #1322.

[clux]

This looks sensible to me, and matches the other way we do it. Thanks a lot for investigating and extending the tests for this.

(The deny issue should be unrelated and fixed in main.)