Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add a way to disable arf-string behavior in fs_utf8 #186

Closed
sunfishcode opened this issue Sep 14, 2021 · 3 comments · Fixed by #188
Closed

Add a way to disable arf-string behavior in fs_utf8 #186

sunfishcode opened this issue Sep 14, 2021 · 3 comments · Fixed by #188

Comments

@sunfishcode
Copy link
Member

cap-std's fs_utf8 module was originally built as part of an experiment with UTF-8 paths in WASI, and one of the questions was whether we could make it possible to access files with non-Unicode encodings while still giving applications valid UTF-8 strings. This led to an experiment known as arf-strings, which is a scheme for losslessly encoding non-UTF-8 bytes within a UTF-8 filename. arf-strings are currently built into fs_utf8, so specially crafted UTF-8 strings can be used to create and manipulate non-UTF-8 filenames.

However, at least some users of fs_utf8 will not likely want this behavior, so cap-std should have a way to disable it, and possibly should disable it by default to be less surprising.

@repi
@repi
Copy link

repi commented Sep 15, 2021

ah was wondering what the arf-strings really was about :)

indeed sounds like a good fit for a feature flag that is not on by default. we error everywhere on non-UTF8 paths and don't need to support them.

sunfishcode added a commit that referenced this issue Sep 15, 2021
@sunfishcode
Make usage of arf-strings in fs_utf8 optional, and off by default.
77c2a4a 
The main tricky part here is that `fs_utf8::DirEntry::file_name()` now
has to return `io::Result<String>` instead of just `String` so that it
can fail if the name is unencodable.

Fixes #186.
@sunfishcode sunfishcode mentioned this issue Sep 15, 2021
Merged
@sunfishcode
Copy link
Member Author

#188 is a PR which fixes this using a cargo feature.

sunfishcode added a commit that referenced this issue Oct 7, 2021
@sunfishcode
Make usage of arf-strings in fs_utf8 optional, and off by default. (#188
2cbe484 
)

The main tricky part here is that `fs_utf8::DirEntry::file_name()` now
has to return `io::Result<String>` instead of just `String` so that it
can fail if the name is unencodable.

Fixes #186.

* Make the `test_invalid_utf8` tolerant of OS's which disallow such paths.

It appears MacOS rejects invalid filenames; on such platforms, don't
try to test that we can open invalid filenames.

* Temporarily disable fuzz test compilation.

This works around rust-fuzz/cargo-fuzz#276.

* Remove the ubuntu-16.04 builds, as ubuntu-16.04 is no longer supported.
@sunfishcode
Copy link
Member Author

#188 is now released in 0.20.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Make usage of arf-strings in fs_utf8 optional, and off by default. bytecodealliance/cap-std
2 participants
@repi @sunfishcode