Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

sendmmsg panics with assertion failed #2031

Open
kalradivyanshu opened this issue May 3, 2023 · 0 comments
Open

sendmmsg panics with assertion failed #2031

kalradivyanshu opened this issue May 3, 2023 · 0 comments

Comments

@kalradivyanshu
Copy link

I cant get sendmmsg to work with GSO and txtime

This function uses sendmsg and works:

pub fn send_to_gso_pacing(
    socket: &tokio::net::UdpSocket,
    buf: &[u8],
    send_to: SocketAddr,
    send_at: Instant,
    segment_size: usize,
) -> io::Result<usize> {
    let iov = [IoSlice::new(buf)];
    let segment_size = segment_size as u16;
    let dst = SockaddrStorage::from(send_to);
    let sockfd = socket.as_raw_fd();

    // GSO option.
    let cmsg_gso = ControlMessage::UdpGsoSegments(&segment_size);
    let nanos_per_sec: u64 = 1_000_000_000;

    // Pacing option.
    let mut time_spec = libc::timespec {
        tv_sec: 0,
        tv_nsec: 0,
    };

    unsafe {
        std::ptr::copy_nonoverlapping(
            &send_at as *const _ as *const libc::timespec,
            &mut time_spec,
            1,
        )
    };

    let send_time = time_spec.tv_sec as u64 * nanos_per_sec + time_spec.tv_nsec as u64;

    let cmsg_txtime = ControlMessage::TxTime(&send_time);

    match sendmsg::<SockaddrStorage>(
        sockfd,
        &iov,
        &[cmsg_gso, cmsg_txtime],
        MsgFlags::empty(),
        Some(&dst),
    ) {
        Ok(v) => Ok(v),
        Err(e) => Err(e.into()),
    }
}

but the same message using sendmmsg fails:

pub fn sendmmsg_to_gso_pacing(
    socket: &tokio::net::UdpSocket,
    bufs: &mut [Vec<u8>],
    send_to: SocketAddr,
    send_at: Instant,
    batch_size: usize,
    segment_size: usize,
) -> io::Result<Vec<usize>> {
    let segment_size = segment_size as u16;
    let mut iovs = Vec::with_capacity(batch_size);
    let mut addrs = Vec::with_capacity(batch_size);
    let dst = SockaddrStorage::from(send_to);
    let sockfd = socket.as_raw_fd();

    for buf in bufs {
        iovs.push([IoSlice::new(buf)]);
        addrs.push(Some(dst));
    }

    // GSO option.
    let cmsg_gso = ControlMessage::UdpGsoSegments(&segment_size);
    let nanos_per_sec: u64 = 1_000_000_000;

    // Pacing option.
    let mut time_spec = libc::timespec {
        tv_sec: 0,
        tv_nsec: 0,
    };

    unsafe {
        std::ptr::copy_nonoverlapping(
            &send_at as *const _ as *const libc::timespec,
            &mut time_spec,
            1,
        )
    };

    let send_time = time_spec.tv_sec as u64 * nanos_per_sec + time_spec.tv_nsec as u64;

    let cmsg_txtime = ControlMessage::TxTime(&send_time);
    let cmsg_buffer = cmsg_space!(UdpGsoSegment, TxTime);
    println!("cmsg_buffer size: {}", cmsg_buffer.capacity());
    let mut data = MultiHeaders::preallocate(batch_size, Some(cmsg_buffer));

    match sendmmsg(
        sockfd,
        &mut data,
        &iovs,
        &addrs,
        [cmsg_gso, cmsg_txtime],
        MsgFlags::empty(),
    ) {
        Ok(v) => {
            let mut sizes = Vec::with_capacity(batch_size + 1);
            for item in v {
                sizes.push(item.bytes);
            }
            Ok(sizes)
        }
        Err(e) => Err(e.into()),
    }
}

This panics with the error:

thread 'tokio-runtime-worker' panicked at 'assertion failed: `(left != right)`
  left: `0x0`,
 right: `0x0`', path/nix-0.26.2/src/sys/socket/mod.rs:1550:13

Why does this panic? Can anyone point me to an example of how to use sendmmsg with txtime and GSO?

Thanks!

@Jan561 Jan561 mentioned this issue Nov 23, 2023
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant