Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

__socket_cached_read opvector的 len 非法,导致req_len 0,导致返回0 认为对端关闭socket 反复重启链接 #4448

Open
feizai131 opened this issue Jan 1, 2025 · 0 comments
Assignees

Comments

@feizai131
Copy link

Description of problem:
以下栈 增加的校验core req_len ==0 core
#1 0x00007fa08df34e65 in abort () from /lib64/libc.so.6
#2 0x00007fa08df34d39 in __assert_fail_base.cold.0 () from /lib64/libc.so.6
#3 0x00007fa08df59e86 in __assert_fail () from /lib64/libc.so.6
#4 0x00007f9fa7cded85 in __socket_cached_read (this=0x7f9f68005990, opvector=0x7f9f68005f40, opcount=1) at socket.c:407
#5 0x00007f9fa7cdf3d3 in __socket_rwv (this=0x7f9f68005990, vector=0x7f9f68005f40, count=1, pending_vector=0x7f9f68005fc0, pending_count=0x7f9f68005fc8,
bytes=0x7f9fa723a038, write=0) at socket.c:554
#6 0x00007f9fa7cdf79a in __socket_readv (this=0x7f9f68005990, vector=0x7f9f68005f40, count=1, pending_vector=0x7f9f68005fc0, pending_count=0x7f9f68005fc8,
bytes=0x7f9fa723a038) at socket.c:641
#7 0x00007f9fa7ce2680 in __socket_read_vectored_request (this=0x7f9fa7ce2680 <__socket_read_vectored_request+3522>, vector_sizer=0x7f9fa723a190)
at socket.c:1333
#8 0x00007f9fa7ce3959 in __socket_read_request (this=0x7f9f68005990) at socket.c:1574
#9 0x00007f9fa7ce6719 in __socket_read_frag (this=0x7f9f68005990) at socket.c:2135
#10 0x00007f9fa7ce7b97 in __socket_proto_state_machine (this=0x7f9f68005990, pollin=0x7f9fa723a410) at socket.c:2523
#11 0x00007f9fa7ce8041 in socket_proto_state_machine (this=0x7f9f68005990, pollin=0x7f9fa723a410) at socket.c:2630
#12 0x00007f9fa7ce82eb in socket_event_poll_in_worker (data=0x193f01) at socket.c:2679
#13 0x00007f9fa7ce8b59 in socket_event_handler (fd=38, idx=6, gen=1, data=0x7f9f68005990, poll_in=1, poll_out=0, poll_err=0) at socket.c:2847
#14 0x00007fa08f93934d in event_dispatch_epoll_handler (event_pool=0x8595b0, event=0x7f9fa723a544) at event-epoll.c:523
#15 0x00007fa08f939624 in event_dispatch_epoll_worker (data=0x8a62a0) at event-epoll.c:592
#16 0x00007fa08e7e31ca in start_thread () from /lib64/libpthread.so.0
#17 0x00007fa08df4c8d3 in clone () from /lib64/libc.so.6
(gdb) p opvector
No symbol "opvector" in current context.
(gdb) f 4
#4 0x00007f9fa7cded85 in __socket_cached_read (this=0x7f9f68005990, opvector=0x7f9f68005f40, opcount=1) at socket.c:407
Download failed: Function not implemented. Continuing without source file /data/cpf/2.0.0/hifs-sdk/rpc/rpc-transport/socket/src/socket.c.
407 socket.c: No such file or directory.
(gdb) p opvector
$1 = (struct iovec *) 0x7f9f68005f40
(gdb) p opvector[0]
$2 = {iov_base = 0x7f9f68005f32, iov_len = 21474836480}

The exact command to reproduce the issue:
if (in->ra_served < in->ra_read) {
ret = iov_load(opvector, opcount, &in->ra_buf[in->ra_served],
min(req_len, (in->ra_read - in->ra_served)));

    in->ra_served += ret;
    /* Do not read uncached and cached in the same call */
    goto out;
}由于eq_len 0,导致返回ret 0  此时并不是关闭socket了

The full output of the command that failed:

Expected results:

Mandatory info:
- The output of the gluster volume info command:

- The output of the gluster volume status command:

- The output of the gluster volume heal command:

**- Provide logs present on following locations of client and server nodes -
/var/log/glusterfs/

**- Is there any crash ? Provide the backtrace and coredump

#define __socket_proto_update_pending(priv)
do {
uint32_t remaining;
struct gf_sock_incoming_frag* frag;
frag = &priv->incoming.frag;
if (frag->pending_vector->iov_len == 0) {
remaining = (RPC_FRAGSIZE(priv->incoming.fraghdr) - frag->bytes_read);

frag->pending_vector->iov_len = (remaining > frag->remaining_size) ? frag->remaining_size : remaining;

frag->remaining_size -= frag->pending_vector->iov_len;
、*********priv->incoming.pending_vector = frag->pending_vector; ********
}
} while (0)

priv->incoming.pending_vector != frag->pending_vector;

Additional info:

- The operating system / glusterfs version:

Note: Please hide any confidential data which you don't want to share in public like IP address, file name, hostname or any other configuration

@pranithk pranithk self-assigned this Jan 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants