Tracking issue for io_slice_advance

[Thomasdezeeuw]

This is a tracking issue for IoSlice::{advance, advance_slices} and IoSliceMut::{advance, advance_slices}.

Feature gate: #![feature(io_slice_advance)].

Steps:

• Implement the RFC.
  • Add {IoSlice, IoSliceMut}::advance #62987
  • Rename IoSlice(Mut)::advance to advance_slice and add IoSlice(Mut)::advance #85802
• FCP
• Stabilization PR.

Current API additions:

impl<'a> IoSlice<'a> { // And `IoSliceMut`
    pub fn advance(&mut self, n: usize);
    pub fn advance_slices(bufs: &mut &mut [IoSlice<'a>], n: usize);
}

---

Old issue:

Writing rust-lang/futures-rs#1741 I needed to resort to unsafe code to change the underlying slice in IoSlice (and IoSliceMut). I'm missing a method that can change the underlying slice. @Nemo157 said that I should open an issue.

Current idea would be something like Buf::advance from the bytes crate.

impl IoSlice {
    // Advance the internal cursors of the slice by `n` bytes.
    fn advance(&mut self, n: usize) {
        // ..
    }
}

[Thomasdezeeuw]

Looking for some feedback as to whether this would be accepted.

[Thomasdezeeuw]

Can someone reopen this as the tracking issue for io_slice_advance? I've updated the the first comment, but I'm not sure it's 100% correct.

[seanmonstar]

I was trying to impl Buf for IoSlice in the bytes crate, and noticed there's currently an issue with lifetimes:

impl Buf for IoSlice<'_> {
    fn advance(&mut self, cnt: usize) {
        *self = IoSlice::new(&self[cnt..]); // conflicting lifetime requirements
    }
}

To allow such a thing externally, we could add a method to IoSlice that gives the slice with the same lifetime as IoSlice (whereas Deref uses an anonymous lifetime).

impl<'a> IoSlice<'a> {
    pub fn inner_ref(&self) -> &'a [u8] {
        // ...
    }
}

[Thomasdezeeuw]

@seanmonstar can get around the lifetime issues by replacing self with an empty slice and then replacing it again with the advanced slice? Something like can be found here: https://github.com/rust-lang/rust/pull/62987/files#diff-668f8f358d4a93474b396dcb3727399eR975.

[seanmonstar]

@Thomasdezeeuw I don't think so. It's not possible to get the inner slice with the same lifetime due to Deref:

let tmp = std::mem::replace(self, IoSlice::new(&[]));
let tail = &tmp[cnt..]; // <- lifetime isn't same as `IoSlice<'a>`
*self = IoSlice::new(tail);

[Nemo157]

@seanmonstar it's not an amazing API to build that on top of, but it is possible (playground):

let mut slices = [std::mem::replace(self, IoSlice::new(&[]))];
let slices = IoSlice::advance(&mut slices[..], cnt);
match slices {
    [slice] => {
        *self = std::mem::replace(slice, IoSlice::new(&[]));
    }
    [] => {
        *self = IoSlice::new(&[]);
    }
    _ => unreachable!(),
}

[seanmonstar]

Oh right, that is using the unstable function. I was referring to be able to implement this now.

[Nemo157]

Ah, I just assumed since that's what the tracking issue is about. It seems like having IoSlice<'a>::into_inner(self) -> &'a [u8] along with Clone and Copy would make sense since it's just a newtyped slice (hopefully that's possible with the actual os-specific implementations).

[mpdn]

I've been trying to write a write_all_vectored function that continuously calls write_vectored, but it seems like this is not really possible to do without some way of advancing the IoSlices (or resorting to unsafe, of course). It does not seem to me like vectored IO is very practical without this feature or something equivalent.

[Ekleog]

Are there any news on landing this? I'm trying to do vectored IO too, and it would be very nice if it were possible to just use this function — I think this is the only unstable feature I currently depend on, and it would be very nice to be able to compile on stable without having to add unsafe code.

into_inner as discussed above is another primitive that would make it possible to implement advance as a user, and it would most likely be great to have it, but seeing the way vectored IO is done I can't see a world where a function like advance isn't required.

[PureWhiteWu]

Hi, is there any progress of this feature?

[rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[rfcbot]

The final comment period, with a disposition to merge, as per the review above, is now complete.

As the automated representative of the governance process, I would like to thank the author for their work and everyone else who contributed.

This will be merged soon.

[zh-jq-b]

The following assert won't work after this if the reader call IoSliceMut::advance() in it's read_vectored().

fn what_to_read<R>(reader: R)
where
    R: Read,
{
    let mut b1 = [0u8; 16];
    let mut b2 = [0u8; 32];

    let mut iov = [IoSliceMut::new(&mut b1), IoSliceMut::new(&mut b2)];
    let nr = reader.read_vectored(&mut iov).unwrap();

    if nr > 0 {
        assert_eq!(b1[0], iov[0][0]);
    }
    if nr >= b1.len() {
        assert_eq!(iov[0].len(), b1.len());
    }
}

[Thomasdezeeuw]

@zh-jq-b in your example you don't call IoSliceMut::advance, so I don't see how it's relevant to the method?

[zh-jq-b]

@zh-jq-b in your example you don't call IoSliceMut::advance, so I don't see how it's relevant to the method?

Yes I didn't call IoSliceMut::advance but the reader.read_vectored() may call that indirectly?
What I want to say is that, there is no guranteen that IoSliceMut::advance won't get called inside reader.read_vectored() or any other existed functions that take a IoSliceMut param as a way to receive data.

[Thomasdezeeuw]

I don't think we can guarantee that the method will never be incorrectly used. Same for process::exit for example... I don't know what you expect the standard library to change about this.

[zh-jq-b]

I don't think we can guarantee that the method will never be incorrectly used. Same for process::exit for example... I don't know what you expect the standard library to change about this.

Before the stabilization of io_slice_advance, users can use both the original buffer and also the iov to get the read data.
After the stabilization, I'm confused with which one should be used?
For all old methods that take a IoSliceMut, should I still read the data by using the old way? Will it broken without noticeable API changes?

[dead-claudia]

@zh-jq-b Consider filing a bug report against anyone who sets them incorrectly.

It's not Rust's fault if someone implements read_vectored wrong. It's the fault of whoever wrote the bad implementation for that io::Read trait instance method.

[zh-jq-b]

@zh-jq-b Consider filing a bug report against anyone who sets them incorrectly.

It's not Rust's fault if someone implements read_vectored wrong. It's the fault of whoever wrote the bad implementation for that io::Read trait instance method.

The read_vectored should be documented to specify what's the wrong implementation. So I opened #128669 for further confirmation.