Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Skupper podman -- FE router & service-controller pods CrashLoopBackOff #1296

Closed
unreal-asr opened this issue Nov 28, 2023 · 1 comment
Closed

Skupper podman -- FE router & service-controller pods CrashLoopBackOff #1296

unreal-asr opened this issue Nov 28, 2023 · 1 comment

Comments

@unreal-asr
Copy link

Hi,

following the use case described in this guide (sub-section Example 02: Exposing a service from a Podman site ) it seems that after a few seconds of good operation, the FE site router & service-controller pod CrashLoopBackOff (in k8s enviroment). The BE services router & controller on podman site seams to work well without restart.
The VAN link and base communication works well for a few moments before the pods on FE site crash.
The skupper console also becomes unreachable.

Following are the skupper-router logs on FE site.


2023-11-28 10:58:10.924713 +0000 FLOW_LOG (info) Event collection for flow events detected.  Begin sending flow events.
2023-11-28 10:58:10.924803 +0000 SERVER (info) [C277] Accepted connection to :5671 from 10.130.0.146:39862
2023-11-28 10:58:10.924854 +0000 SERVER (info) [C276] Accepted connection to :5671 from 10.130.0.146:39848
2023-11-28 10:58:12.420530 +0000 SERVER (info) [C278] Accepted connection to localhost:5672 from ::1:43002
2023-11-28 10:58:12.420676 +0000 SERVER (info) [C279] Accepted connection to localhost:5672 from ::1:43004
2023-11-28 10:58:12.421342 +0000 SERVER (info) [C280] Accepted connection to localhost:5672 from ::1:43016
2023-11-28 10:58:13.001394 +0000 SERVER (info) [C281] Accepted connection to localhost:5672 from ::1:43022
2023-11-28 10:58:13.001531 +0000 SERVER (info) [C282] Accepted connection to localhost:5672 from ::1:43026
2023-11-28 10:58:13.001704 +0000 SERVER (info) [C283] Accepted connection to localhost:5672 from ::1:43038
2023-11-28 10:58:21.028979 +0000 SERVER (error) [C263] Connection from ::1:42982 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029081 +0000 SERVER (error) [C283] Connection from ::1:43038 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029160 +0000 SERVER (error) [C282] Connection from ::1:43026 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029303 +0000 SERVER (error) [C280] Connection from ::1:43016 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029370 +0000 SERVER (error) [C279] Connection from ::1:43004 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029495 +0000 SERVER (error) [C278] Connection from ::1:43002 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029559 +0000 SERVER (error) [C265] Connection from ::1:42992 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029613 +0000 SERVER (error) [C264] Connection from ::1:42986 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029727 +0000 SERVER (error) [C262] Connection from ::1:39936 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029767 +0000 SERVER (error) [C260] Connection from ::1:39918 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029859 +0000 SERVER (error) [C261] Connection from ::1:39930 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.029913 +0000 SERVER (error) [C259] Connection from ::1:39910 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.030063 +0000 SERVER (error) [C257] Connection from ::1:39890 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.030175 +0000 SERVER (error) [C256] Connection from ::1:39874 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.030349 +0000 SERVER (error) [C255] Connection from ::1:39868 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.031479 +0000 SERVER (error) [C258] Connection from ::1:39904 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.031681 +0000 SERVER (error) [C281] Connection from ::1:43022 (to localhost:5672) failed: amqp:connection:framing-error connection aborted
2023-11-28 10:58:21.031937 +0000 FLOW_LOG (info) Event collection for log events ended.  Stop sending log events.
2023-11-28 10:58:21.125335 +0000 FLOW_LOG (info) LOG [xxkjg:224] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C263] Connection from ::1:42982 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125439 +0000 FLOW_LOG (info) LOG [xxkjg:225] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C283] Connection from ::1:43038 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125473 +0000 FLOW_LOG (info) LOG [xxkjg:226] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C282] Connection from ::1:43026 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125502 +0000 FLOW_LOG (info) LOG [xxkjg:227] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C280] Connection from ::1:43016 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125529 +0000 FLOW_LOG (info) LOG [xxkjg:228] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C279] Connection from ::1:43004 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125556 +0000 FLOW_LOG (info) LOG [xxkjg:229] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C278] Connection from ::1:43002 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125582 +0000 FLOW_LOG (info) LOG [xxkjg:230] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C265] Connection from ::1:42992 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125606 +0000 FLOW_LOG (info) LOG [xxkjg:231] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C264] Connection from ::1:42986 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125642 +0000 FLOW_LOG (info) LOG [xxkjg:232] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C262] Connection from ::1:39936 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125668 +0000 FLOW_LOG (info) LOG [xxkjg:233] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C260] Connection from ::1:39918 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125694 +0000 FLOW_LOG (info) LOG [xxkjg:234] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C261] Connection from ::1:39930 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125721 +0000 FLOW_LOG (info) LOG [xxkjg:235] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C259] Connection from ::1:39910 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125746 +0000 FLOW_LOG (info) LOG [xxkjg:236] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C257] Connection from ::1:39890 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125770 +0000 FLOW_LOG (info) LOG [xxkjg:237] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C256] Connection from ::1:39874 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125796 +0000 FLOW_LOG (info) LOG [xxkjg:238] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C255] Connection from ::1:39868 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125819 +0000 FLOW_LOG (info) LOG [xxkjg:239] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C258] Connection from ::1:39904 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:21.125845 +0000 FLOW_LOG (info) LOG [xxkjg:240] BEGIN END parent=xxkjg:0 logSeverity=3 logText=LOG_SERVER: [C281] Connection from ::1:43022 (to localhost:5672) failed: amqp:connection:framing-error connection aborted sourceFile=/build/src/server.c sourceLine=1088
2023-11-28 10:58:25.817392 +0000 ROUTER (info) ram:23.46GiB vm:547.61MiB rss:76.00MiB pool:15.08KiB qdtm_router_t:64:0 qd_timer_t:64:0 qd_connection_t:80:16 qd_listener_t:64:0 qdr_core_timer_t:64:0 qdr_terminus_t:320:0 qd_management_context_t:256:0 qdr_action_t:384:128 qdr_field_t:320:128 qdr_connection_info_t:256:64 qdr_connection_ref_t:64:0 qdr_link_work_t:384:128 qdr_general_work_t:448:64 qdr_delivery_cleanup_t:384:128 qdr_link_ref_t:512:128 qdr_link_t:256:128 qdr_delivery_ref_t:384:128 qdr_node_t:64:0 qdr_address_config_t:64:0 qdr_address_t:128:0 qdr_forward_deliver_info_t:64:0 qdr_error_t:64:0 qdr_delivery_t:256:64 qdr_connection_work_t:448:0 qdr_connection_t:256:64 qdr_query_t:320:0 qdr_address_watch_t:64:0 vflow_work_t:320:128 vflow_record_t:320:64 vflow_rate_t:64:0 vflow_attribute_data_t:128:64 qd_parse_node_t:128:0 qd_parse_tree_t:64:0 qd_parsed_field_t:320:192 qd_message_stream_data_t:192:0 qd_message_content_t:384:128 qd_message_t:448:128 qd_log_entry_t:1072:0 qd_iterator_t:384:192 qd_hash_handle_t:192:0 qd_hash_item_t:192:0 qd_delivery_state_t:256:0 qd_session_t:256:0 qd_link_ref_t:256:128 qd_link_t:256:0 qd_node_t:64:0 qd_composed_field_t:448:0 qd_composite_t:448:0 qd_buffer_t:80:48 qd_bitmask_t:320:128 qd_adaptor_listener_t:64:0 qd_tcp_adaptor_config_t:64:0 qdr_tcp_connection_t:48:0 qd_tcp_listener_t:64:0 qdr_tcp_stats_t:64:0 qd_adaptor_config_t:64:0 qd_adaptor_buffer_t:80:0 

BE (skupper switch --> podman)

client version                 1.5.0
transport version              quay.io/skupper/skupper-router:2.5.0 (sha256:55f014d0fcf4)

FE (skupper switch --> kubernetes)

client version                 1.5.0
transport version              quay.io/skupper/skupper-router:2.5.0 (sha256:55f014d0fcf4)
controller version             quay.io/skupper/service-controller:1.5.0 (sha256:11a88e11ee66)
config-sync version            quay.io/skupper/config-sync:1.5.0 (sha256:6a8f5846a805)
flow-collector version         quay.io/skupper/flow-collector:1.5.0 (sha256:f07ea035141f)

I understand that the guide states "This is a preview feature and may change before becoming fully supported by [skupper.io]"
I hope the issue can be resolved anyway.
@unreal-asr unreal-asr changed the title Skupper podman falls into the FE router & service-controller pod CrashLoopBackOff Skupper podman -- FE router & service-controller pods CrashLoopBackOff Nov 28, 2023
@ajssmith
Copy link
Member

This is fixed by #1299

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@ajssmith @unreal-asr