feat: VID take iterator instead of slice

[ggutoski]

Description

closes: #375
closes: #300

• conversion.rs: add new bytes_to_field and field_to_bytes that act on iterators instead of slices. The code is more complex than I like. But if you trust the tests then you don't need to get too deep into it. The cause of this complexity is that we need to append an extra field element at the end to encode some length information in order to enable inversion of this process.
  • These methods would be much simpler if we did not need to encode this metadata. But that would make these functions useless because the caller must know in advance how many bytes are in the iterator so he can truncate the decoded bytes accordingly. Since we're now using iterators, not slices, it seems unreasonable for the caller to know this.
  • These methods work only on PrimeField, not Field. We could extend it to Field but I don't want to take extra time for that now.
• I did not delete the existing functions bytes_to_field_elements, bytes_from_field_elements that act on slices. But we could delete them if needed. (Currently they're still used in plonk.)
• Update VID API to take iterators instead of slices, use the new conversion methods.
• Add a new elems_len field to Common struct in VID. This data is needed because recovery adds extra zero elements via RS decoding. Previously we did not need this because the existing bytes_to_field_elements encodes the total byte length in the output. Thus, bytes_from_field_elements automatically drops any extra zero elements added by RS decoding. By contrast, the new bytes_to_field encodes only the byte length of the final element (not the total byte length). So now VID needs to keep track of how many zero elems were added by RS decoding.

Other stuff snuck into this PR

• clean itertools dependency
• add new derivations for VidDisperse struct that are useful downstream

---

Before we can merge this PR, please make sure that all the following items have been
checked off. If any of the checklist items are not applicable, please leave them but
write a little note why.

• Targeted PR against correct branch (main)
• Linked to GitHub issue with discussion and accepted design OR have an explanation in the PR that describes this work.
• Wrote unit tests
• Updated relevant documentation in the code
• Added a relevant changelog entry to the Pending section in CHANGELOG.md
• Re-reviewed Files changed in the GitHub PR explorer

[mrain]

Any specific reason for making it PrimeField?

[ggutoski]

I was in a rush. 😊 I'm happy to extend it to Field if you request it. That gives me an excuse to allocate the time to do it.

[ggutoski]

The answer to your question is here:

jellyfish/primitives/src/vid/advz.rs

Lines 152 to 153 in 474fcdf

	// `PrimeField` needed only because `bytes_to_field` needs it.
	// Otherwise we could relax to `FftField`.

This #381 (comment) is an explanation for why I restricted bytes_to_field to PrimeField.

[mrain]

Got it. We can settle with PrimeField for now, and add a follow-up issue for the future improvement.

[ggutoski]

#382

[mrain]

LGTM. We can merge it after some follow-up issues and CI fixes.