-
Notifications
You must be signed in to change notification settings - Fork 17.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
path/filepath: add Localize to safely convert a slash-separated path into an operating system path #57151
Comments
A concrete example of where this could be useful: |
I'm starting to get confused about which of these new filepath functions should be used when. |
This proposal has been added to the active column of the proposals project |
I am very sympathetic to that confusion. Counting
You should use You should use You should use |
It is unclear to me why the function is called The proposal says that it supports paths that Then there is fs.ValidPath that rejects absolute paths. If FromFS is supposed to support absolute paths and since FromFS is intended to replace |
I don't think it's a given that everyone who uses FromSlash today should use FromFS. Today, FromSlash converts slashes to the canonical form for the host OS but preserves the meaning of the existing path. So for example on Windows today, FromSlash("c:/foo") gives you c:\foo, which is the canonical form of its input. Similarly on a Mac you get FromSlash("c:/foo") is c:/foo, and in both cases the result of os.Open(p) and os.Open(FromSlash(p)) are the same. It sounds like FromFS would not do that. You should use FromFS when the input is meant to be a "portable" slash-separated path as opposed to a slash-separated path interpreted according to the local OS. Programs that accept a file name on the command line but want to convert to native conventions should probably keep using FromSlash. The compiler does this sometimes for arranging canonical outputs and then inverting them. It should keep using FromSlash and ToSlash. Maybe something reading from a zip file should use FromFS, but why not just have it use IsLocal+FromSlash instead of FromFS? Or is it just programs implementing an fs.FS that need to use FromFS? I don't think the exact scope is clearly defined yet. |
It seems like we are stuck on the name here. I noticed that internal/safepath.FromFS is called with paths beginning with / (like /foo) but those are not actually io/fs paths. I'm not sure if the proposed FromFS rejects those or not, but safepath.FromFS does not. Also it's probably too indirect a meaning to use "FS" here. There are fundamentally two kinds of FromSlash: ones that are canonicalizing the OS interpretation and ones that are converting from "portable" to "local OS". The current FromSlash does the former. We need a name for the latter. Maybe the From prefix is tripping us up and we should name this operation with some verb that can be the function name. |
|
I can't come up with a verb, because filepath.Verb implies an operation on host OS paths. This is an operation on a non-host path.
Probably impractical: Keep |
Localize means "restrict (something) to a particular place" and can also be interpreted as converting to local syntax, which it also does. That seems like a good name. Any better names? |
+1 for this proposal. I always fear using the path/filepath packages because I never truly understand what to expect. As for |
How does s, err := Localize("/a/b/c") // s = "/a/b/c", err = nil
IsLocal(s) // false In other words, the result of Localize could be non-local, because local paths cannot be absolute. Since @rsc mentioned
maybe we should define what a portable path is and use something like It seems having a definition of all the supported path variants in the package docs would be helpful in any case. |
|
Given that io/fs does not accept /a/b/c as a path and we started at FromFS meaning "from io/fs paths", my understanding was that Localize would reject /a/b/c and ../a/b/c, at which point Localize would guarantee to return a path satisfying IsLocal. If we do that, is Localize no longer useful enough? |
Suppose we have func Localize(string) (string, error), which converts the input to be a local file system path, satisfying IsLocal, or else returns an error. Localize("/a/b/c") and Localize("../a/b/c") are both errors, as are things like Localize("com1") on Windows. Does that work for the use cases we want it for? |
This is blending two different concerns: converting a slash-separated path into an operating system path, and enforcing constraints on the path (e.g., not absolute). It feels to me like we're arriving at a behavior because of the ambiguous meaning of "localize", not because we want that particular behavior. But along those lines, how about the original name |
I wrote:
Then Damien wrote:
Are these two descriptions of the same behavior with different names, or are the behaviors different? It appears they are the same. But if the behaviors are the same, then Localize seems like a clearer name than FromFS, because lots of people using filepath are unfamiliar with io/fs and won't think of FS as indicating io/fs. They might think it means something about the local operating system's file system. filepath.Localize in contrast sounds very much like a conversion to local conventions (and it is), in addition to being a containment of what the path can refer to (and it is). |
What should If it's an error, because the input isn't a valid path according to I think this behavior works for the cases we care about. I'm okay with either the name |
Yes, I was assuming any mention of .. in Localize's argument is an error. |
Based on the discussion above, this proposal seems like a likely accept. |
No change in consensus, so accepted. 🎉 |
Fixes golang#57151. Change-Id: I98c6a211d77fa5c8733306b5ef39950bae07fb5c
Change https://go.dev/cl/520268 mentions this issue: |
Implemented filepath.Localize using internal/safefilepath.FromFS. Fixes golang#57151 Change-Id: I98c6a211d77fa5c8733306b5ef39950bae07fb5c
Change https://go.dev/cl/531677 mentions this issue: |
The
filepath.FromSlash
function sounds like it converts a /-separated path into an operating system path. What it actually does is replace every / in its input withos.PathSeparator
.FromSlash
can map an input path into semantically different paths on different operating systems. For example,FromSlash("a\b")
returns the filenamea\b
on Unix and the fileb
in the directorya
on Windows.FromSlash("C:/foo")
returns a relative path on Unix and an absolute path on Windows.#56694 involves failures in the standard library to safely convert a non-platform-specific /-separated path into a semantically equivalent operating system path. The fix (https://go.dev/cl/455716) introduces an internal function to perform this operation. We should have a public API for this.
The proposal:
FromFS
rejects empty paths ("") and, on Windows, reserved device names such as NUL.FromFS
andIsLocal
(#56219) are similar in that both involve performing safety checks on a potentially-untrusted input path. They serve different roles, however:FromFS
takes a /-separated path in the form operated on by thepath
package and safely converts it to a semantically-equivalent operating system path.IsLocal
takes an operating system path and reports whether it refers to something surprising.The text was updated successfully, but these errors were encountered: