Use magic to choose an enum variant without consuming the matched data?

[briandorsey]

I'm parsing RIFF/WAV chunks and each chunk starts with a four byte chunk id. I'm using magic to choose variants with embedded structs (simplified version below, original source here. Each chunk also stores this chunk id. Unfortunately, magic has already consumed the data.

Sometimes I'd like to parse specific chunk types without using the enum. I couldn't figure out a way to also store the chunk id in the structs without using seek_before, or other work-arounds that make the structs awkward to use on their own.

My guess is that I'm missing something which binrw already does, but I haven't found it yet. What would you recommend in this situation? If there was a non-consuming version of magic, that would work without needing any seeking in the enclosed structs.

Example below:

use std::io::SeekFrom;

use binrw::binrw;

// this is a subset of RIFF/WAV chunk parsing
// using magic to choose the right variant works great!
#[binrw]
#[brw(little)]
#[derive(Debug, PartialEq, Eq)]
pub enum Chunk {
    // ... snip
    #[brw(magic = b"data")]
    Data(DataChunk),
    #[brw(magic = b"MD5 ")]
    Md5(Md5Chunk),
    Unknown {
        id: FourCC,
        size: u32,
        #[br(count = size )]
        #[bw()]
        raw: Vec<u8>,
    },
}

// but each of the variants contains a struct, and I'd love to be able to
// use them directly and naturally. Currently I'm using "seek_before"
// to move the reader position back to where it was before parsing the
// magic in the enclosing enum .... but this makes it awkward to 
// parse structs on their own
#[binrw]
#[brw(little)]
#[derive(Debug, PartialEq, Eq)]
pub struct Md5Chunk {
    #[brw(seek_before = SeekFrom::Current(-4))]
    id: FourCC,
    size: u32,
    md5: u128,
}

#[binrw]
#[brw(big)]
#[derive(Debug, PartialEq, Eq, Clone, Copy)]
pub struct FourCC(pub [u8; 4]);

#[binrw]
#[brw(little)]
#[derive(Debug, PartialEq, Eq)]
pub struct DataChunk {
    #[brw(seek_before = SeekFrom::Current(-4))]
    id: FourCC,
    size: u32,
    #[brw(seek_before = SeekFrom::Current(size.to_owned().into()), ignore)]
    end_of_chunk: [u8; 0],
}

Options I considered to remove seek_before:

use a combination of a fixed value and ignore

#[br(map = |_| FourCC(*b"MD5 "))]
#[bw(ignore)]
id: FourCC,

This does mean that (de)serialization must always happen from the enum,
not underlying struct (or it wouldn't write out the id via magic)

no id field in struct at all, use an id() function to access id?

As above, can only (de)serialize via enum, not base struct to make sure the chunk id was written out.

restore_position gives an error

The "restore_position" attribute next to "magic" seems like it would
be perfect, but I get an error when I try to use it on an enum

variant: error: expected one of: `stream`, `big`, `little`, `is_big`, 
`is_little`, `map`, `try_map`, `repr`, `map_stream`, `magic`, `import`,
`import_raw`, `assert`, `pre_assert`

[kitlith]

As a general rule of thumb, enum variants attempt to be parsed in order, and any parsing error means that the next one is tried. This means that moving the magic directives from the enum variants to your structs will likely do the trick for you, or asserting on your 'id' field if you want it to be a field on the struct.

(I didn't remember this first though, so was going to suggest things like user defined types as magic (I don't even know if that actually works), a generic wrapper whose only job is to provide the FourCC when you're not using the enum, or passing an argument indicating whether or not to parse the magic >_>)

[briandorsey]

Thank you, this worked great. I ended up removing the magic from the enum and adding it to the id:

#[binrw]
#[brw(little)]
#[derive(Debug, PartialEq, Eq)]
pub struct Md5Chunk {
    #[brw(magic = b"MD5 ", seek_before = SeekFrom::Current(-4))]
    id: FourCC,
    size: u32,
    md5: u128,
}

It seems to consistently do those in order, forcing the magic match and then seeking backwards right after. I decided to use magic rather than assert because I think it slightly better encodes the intent of the format.

Thank you! I somehow hadn't even considered just moving it.

[kitlith]

I mean, I'd probably remove the id field from the struct and move that magic to the top level like so:

#[binrw]
#[brw(little, magic = b"MD5 ")]
#[derive(Debug, PartialEq, Eq)]
pub struct Md5Chunk {
    size: u32,
    md5: u128,
}

or, if you really need the id field in the chunk, you could use calc in order to avoid the seek

#[binrw]
#[brw(little)]
#[derive(Debug, PartialEq, Eq)]
pub struct Md5Chunk {
    #[brw(magic = b"MD5 ", calc = FourCC(*b"MD5 "))]
    id: FourCC,
    size: u32,
    md5: u128,
}