Split Sync statuses to smaller objects in etcd

[samutamm]

Close #541 .

Change periodically (every 5s) called StatusSyncController.handleStatus.

• save each status to cluster in individual etcd transaction
• For TrafficController and RawconfigSyncController, split the large status data structure to smaller items so that each HTTPServer object and each HTTPPipeline object has unique key "${kind}-${specName}" in etcd.

Adapt pkg/api/cluster.go to these changes:

• egctl object status list does not require changes
• egctl object status get ${httpPipelineName|httpServerName} uses _getStatusObjectFromTrafficController --> adapt this function to read new status data from etcd

[codecov-commenter]

Codecov Report

Merging #542 (f5415a4) into main (8649d00) will decrease coverage by 1.82%.
The diff coverage is 0.00%.

@@            Coverage Diff             @@
##             main     #542      +/-   ##
==========================================
- Coverage   80.56%   78.74%   -1.83%     
==========================================
  Files          88       94       +6     
  Lines       10123    10814     +691     
==========================================
+ Hits         8156     8515     +359     
- Misses       1514     1814     +300     
- Partials      453      485      +32     

Impacted Files	Coverage Δ
pkg/cluster/layout.go	85.00% <0.00%> (-4.48%)	⬇️
pkg/object/httpserver/spec.go	7.14% <0.00%> (ø)
pkg/object/httpserver/context.go	100.00% <0.00%> (ø)
pkg/object/httpserver/cache.go	76.47% <0.00%> (ø)
pkg/object/httpserver/mux.go	63.24% <0.00%> (ø)
pkg/object/httpserver/httpserver.go	66.66% <0.00%> (ø)
pkg/object/httpserver/runtime.go	42.59% <0.00%> (ø)
pkg/filter/proxy/proxy.go	80.10% <0.00%> (+1.45%)	⬆️

---

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 36480f3...f5415a4. Read the comment docs.

[samutamm]

Here's a memory usage of the same scenario as described in issues #413 and #541 :

The memory usage is captured by running free -s 2 that outputs memory usage every 2 seconds. The two tests have exactly same code, but the difference is that for the orange one has an extra process that runs egctl object update -f 'path/to/pipelines' every 5 minutes. The two tests were run in different VMs that might also differ slightly. However there was no more the issue of growing memory usage during first 1h 40 minutes of execution.

Here's another view of the same data, for 6 hours of execution and with additional change that pipelines are saved to etcd in groups of 20:

Summary

This graph shows that the PR helps to fix the memory leak in the main branch (graphs in the left, blue line)