chore(utils): remove host endianness check #161
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Signed-off-by: hainenber <[email protected]>
rbtr
added
lang/go
|
@microsoft-github-policy-service agree |
|
Thanks for the fix. I am double checking something around this area, please hold off on merging this PR. Thank you! |
anubhabMajumdar
requested changes
Apr 1, 2024
There was a problem hiding this comment.
@hainenber Updated the issue with some more details - #139 (comment) .
We want to handle the conversion at the bpf code and not duplicate the check/conversion at the userspace level. llvm has util functions to do this in the C code. I have added references. Can you give them a try? You do have to change for Packetparser and Dropreason plugin. Let me know if you have more questions. Thanks!
|
This PR will be closed in 7 days due to inactivity. |
github-actions
bot
added
the
meta/waiting-for-author
|
Pull request closed due to inactivity. |
7 tasks
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jun 7, 2024
# Description
Function `determineEndian()` always returns the same value since the
host byte order is always little-endian, so it can be skipped.
What's more, the byte order of IP address values passed by eBPF programs
doesn't need to be changed, because Golang's `net.IP` type expects
big-endian byte arrays.
**Details:**
We always enter this if-branch `binary.LittleEndian.PutUint32(ip, nn)`
which looks like it performs conversion TO little-endian, while it
actually expects to receive input already IN little-endian.
Note, the input number's bytes are kept in the same order - this
function merely converts a uint32 to a byte array. The result is we
convert a big-endian number to a big-endian byte array.
```
func (littleEndian) PutUint32(b []byte, v uint32) {
_ = b[3] // early bounds check to guarantee safety of writes below
b[0] = byte(v)
b[1] = byte(v >> 8)
b[2] = byte(v >> 16)
b[3] = byte(v >> 24)
}
```
Crucially, `net.IP` expects big-endian byte arrays. For example, IP
`15.14.0.0` (`0x0F0E0000` in host order, and `0x00000E0F` in network
order) should be passed in as `[0F, 0E, 00, 00]` (most significant byte
at the lowest memory address) and so doesn't need inverting. Example
here: https://play.golang.com/p/ykiP6mLB_gG
In summary, we currently do not perform byte order conversion and
shouldn't start to, either in GO or eBPF code.
## Related Issue
Fixes #139
This PR looks almost identical to
#161 which was rejected because
it didn't include byte order conversion in eBPF, but turns out that's
not needed. It also uses the correct uint-to-array conversion functions.
## Checklist
- [x] I have read the [contributing
documentation](https://retina.sh/docs/contributing).
- [x] I signed and signed-off the commits (`git commit -S -s ...`). See
[this
documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification)
on signing commits.
- [x] I have correctly attributed the author(s) of the code.
- [x] I have tested the changes locally.
- [x] I have followed the project's style guidelines.
- [x] I have updated the documentation, if necessary.
- [x] I have added tests, if applicable.
Signed-off-by: Igor Klemenski <[email protected]>
Carpe-Wang
pushed a commit
to Carpe-Wang/retina
that referenced
this pull request
Jun 11, 2024
…microsoft#443) Function `determineEndian()` always returns the same value since the host byte order is always little-endian, so it can be skipped. What's more, the byte order of IP address values passed by eBPF programs doesn't need to be changed, because Golang's `net.IP` type expects big-endian byte arrays. **Details:** We always enter this if-branch `binary.LittleEndian.PutUint32(ip, nn)` which looks like it performs conversion TO little-endian, while it actually expects to receive input already IN little-endian. Note, the input number's bytes are kept in the same order - this function merely converts a uint32 to a byte array. The result is we convert a big-endian number to a big-endian byte array. ``` func (littleEndian) PutUint32(b []byte, v uint32) { _ = b[3] // early bounds check to guarantee safety of writes below b[0] = byte(v) b[1] = byte(v >> 8) b[2] = byte(v >> 16) b[3] = byte(v >> 24) } ``` Crucially, `net.IP` expects big-endian byte arrays. For example, IP `15.14.0.0` (`0x0F0E0000` in host order, and `0x00000E0F` in network order) should be passed in as `[0F, 0E, 00, 00]` (most significant byte at the lowest memory address) and so doesn't need inverting. Example here: https://play.golang.com/p/ykiP6mLB_gG In summary, we currently do not perform byte order conversion and shouldn't start to, either in GO or eBPF code. Fixes microsoft#139 This PR looks almost identical to microsoft#161 which was rejected because it didn't include byte order conversion in eBPF, but turns out that's not needed. It also uses the correct uint-to-array conversion functions. - [x] I have read the [contributing documentation](https://retina.sh/docs/contributing). - [x] I signed and signed-off the commits (`git commit -S -s ...`). See [this documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) on signing commits. - [x] I have correctly attributed the author(s) of the code. - [x] I have tested the changes locally. - [x] I have followed the project's style guidelines. - [x] I have updated the documentation, if necessary. - [x] I have added tests, if applicable. Signed-off-by: Igor Klemenski <[email protected]> Signed-off-by: Carpe-Wang <[email protected]>
matmerr
pushed a commit
to matmerr/retina
that referenced
this pull request
Jul 3, 2024
…microsoft#443) # Description Function `determineEndian()` always returns the same value since the host byte order is always little-endian, so it can be skipped. What's more, the byte order of IP address values passed by eBPF programs doesn't need to be changed, because Golang's `net.IP` type expects big-endian byte arrays. **Details:** We always enter this if-branch `binary.LittleEndian.PutUint32(ip, nn)` which looks like it performs conversion TO little-endian, while it actually expects to receive input already IN little-endian. Note, the input number's bytes are kept in the same order - this function merely converts a uint32 to a byte array. The result is we convert a big-endian number to a big-endian byte array. ``` func (littleEndian) PutUint32(b []byte, v uint32) { _ = b[3] // early bounds check to guarantee safety of writes below b[0] = byte(v) b[1] = byte(v >> 8) b[2] = byte(v >> 16) b[3] = byte(v >> 24) } ``` Crucially, `net.IP` expects big-endian byte arrays. For example, IP `15.14.0.0` (`0x0F0E0000` in host order, and `0x00000E0F` in network order) should be passed in as `[0F, 0E, 00, 00]` (most significant byte at the lowest memory address) and so doesn't need inverting. Example here: https://play.golang.com/p/ykiP6mLB_gG In summary, we currently do not perform byte order conversion and shouldn't start to, either in GO or eBPF code. ## Related Issue Fixes microsoft#139 This PR looks almost identical to microsoft#161 which was rejected because it didn't include byte order conversion in eBPF, but turns out that's not needed. It also uses the correct uint-to-array conversion functions. ## Checklist - [x] I have read the [contributing documentation](https://retina.sh/docs/contributing). - [x] I signed and signed-off the commits (`git commit -S -s ...`). See [this documentation](https://docs.github.com/en/authentication/managing-commit-signature-verification/about-commit-signature-verification) on signing commits. - [x] I have correctly attributed the author(s) of the code. - [x] I have tested the changes locally. - [x] I have followed the project's style guidelines. - [x] I have updated the documentation, if necessary. - [x] I have added tests, if applicable. Signed-off-by: Igor Klemenski <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Remove incorrect host endianness check
Closes #139