Skip to content
This repository has been archived by the owner on Sep 9, 2020. It is now read-only.

check Gopkg filenames case on case insensitive systems #1114

Merged
merged 3 commits into from
Sep 22, 2017

Conversation

sudo-suhas
Copy link
Contributor

What does this do / why do we need it?

This checks the manifest and lock file name on case insensitive systems to ensure that the case exactly matches Gopkg.toml and Gopkg.lock.

Do you need help or clarification on anything?

Looking for feedback on the tests and added functions.

Which issue(s) does this PR fix?

fixes #1035

Changelog

  • fs
    • Export IsCaseSensitiveFilesystem. Add and run test on windows, linux.
    • Add function ReadActualFilenames to read actual file names of given
      string slice. Add tests to be run on windows.
  • project
    • Add function checkCfgFilenames to check the filenames for manifest
      and lock have the expected case. Use fs#IsCaseSensitiveFilesystem
      for an early return as the check is costly. Add test to be run on windows.
  • context
    • Call project#checkCfgFilenames after resolving project root. Add test
      for invalid manifest file name to be run on windows.

@sudo-suhas sudo-suhas requested a review from sdboyer as a code owner September 2, 2017 08:33
@googlebot
Copy link
Collaborator

Thanks for your pull request. It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

📝 Please visit https://cla.developers.google.com/ to sign.

Once you've signed, please reply here (e.g. I signed it!) and we'll verify. Thanks.


  • If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address. Check your existing CLA data and verify that your email is set on your git commits.
  • If your company signed a CLA, they designated a Point of Contact who decides which employees are authorized to participate. You may need to contact the Point of Contact for your company and ask to be added to the group of authorized contributors. If you don't know who your Point of Contact is, direct the project maintainer to go/cla#troubleshoot.
  • In order to pass this check, please resolve this problem and have the pull request author add another comment and the bot will run again.

@sudo-suhas
Copy link
Contributor Author

I signed it!

@googlebot
Copy link
Collaborator

CLAs look good, thanks!

@googlebot googlebot added cla: yes and removed cla: no labels Sep 2, 2017
Copy link
Collaborator

@darkowlzz darkowlzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.
But since most of the tests are set to run on windows only and appveyor doesn't seem to be running for some reason, I'm not sure of the test results. Both Windows & macOS are case-insensitive (by default), so all those windows-only tests can be enabled to run on macOS, for which we have machine setup in travis.

Also, added some inline suggestions for changes. Looks pretty good. 👍

// ReadActualFilenames is used to determine the actual file names in given directory.
//
// On case sensitive file systems like ext4, it will check if those files exist using
// `os#Stat` and return a map with key and value as filenames which exist in the folder.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we prefer writing os.Stat.

return nil, errPathNotDir
}

// Ideally, we would use `os#Stat` for getting the actual file names
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

os.Stat

project.go Outdated
}

actualLfName := actualFilenames[LockName]
if len(actualLfName) > 0 && actualLfName != LockName {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason for this length check? I think just inequality check would be enough. Just curious 😊

Copy link
Contributor Author

@sudo-suhas sudo-suhas Sep 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If a file is not found, the string map returned by ReadActualFilenames will not have an entry for the given filename. Since lock file is optional, we should check for equality only if it was found.

Edit: Will add a code comment explaining this.

project_test.go Outdated
invalidLfName := strings.ToLower(LockName)

cases := []struct {
errExpected bool
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's follow our "want" naming style and change this to wantErr.

project_test.go Outdated
cases := []struct {
errExpected bool
createFiles []string
errMsg string
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

errMsg -> wantErrMsg

project_test.go Outdated
if err == nil {
t.Fatalf("did not get expected error - '%s'", c.errMsg)
} else if err.Error() != c.errMsg {
t.Fatalf("expected message was '%s' but got '%s'", c.errMsg, err.Error())
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's make these GOT/WNT style failure messages.
Example:
("unexpected error message: \n\t(GOT) %s\n\t(WNT) %s", err.Error(), c.errMsg)

func TestIsCaseSensitiveFilesystem(t *testing.T) {
if runtime.GOOS != "windows" && runtime.GOOS != "linux" {
t.Skip("skip this test on non-Windows, non-Linux")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

darwin? We run macOS specific tests on travis and by default macOS is also case insensitive, like windows.

Copy link
Contributor Author

@sudo-suhas sudo-suhas Sep 4, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are many OS recognised by Go runtime - https://github.com/golang/go/blob/master/src/go/build/syslist.go#L7. I thought that since I wasn't sure about the rest of them, it would be sufficient to test for 1 case sensitive and 1 case insensitive system. But I agree that it makes sense to check for macOS as it is among the most popular ones.

func TestReadActualFilenames(t *testing.T) {
if runtime.GOOS != "windows" {
t.Skip("skip this test on non-Windows")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And darwin?

if err != nil {
t.Fatalf("unexpected error: %+v", err)
}
reflect.DeepEqual(c.want, got)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a conditional check and GOT/WNT style failure message here.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apologies. Missed this one.

@darkowlzz
Copy link
Collaborator

oh! appveyor isn't reporting here but it's working fine https://ci.appveyor.com/project/golang/dep/build/1963 and tests are passing 😊
But since those tests are applicable to macOS too, let's add darwin support too.

@ibrasho
Copy link
Collaborator

ibrasho commented Sep 2, 2017

You should be able to simplify this a bit by using fs.EquivalentPaths (from the just merged #940).

@ibrasho
Copy link
Collaborator

ibrasho commented Sep 2, 2017

@sudo-suhas @sdboyer : Just to clarify, we are trying to prevent users from changing the letter-casing of Gopkg.toml and Gopkg.lock?

I would suggest using a different approach and erroring only when using a case-senstive filesystem.
This would make the performance a lesser issue, since this happens on a failure path.

If dep fails to find Gopkg.lock or Gopkg.lock on a case-senstive filesystem, let's do the expensive check to find out if the file is available in a different casing, and show a clear error message saying:

could not find Gopkg.toml. Found gopkg.toml instead. Please make sure the manifest file is named Gopkg.toml.

instead of:

could not find project Gopkg.toml, use dep init to initiate a manifest

@sudo-suhas sudo-suhas force-pushed the check-cfg-filename-case branch from e24eef4 to 7d0e19a Compare September 4, 2017 06:07
@sudo-suhas
Copy link
Contributor Author

@darkowlzz I have implemented the requested changes while rebasing for now.

You should be able to simplify this a bit by using fs.EquivalentPaths

@ibrasho I am not sure I follow. On case insensitive system, when findProjectRoot succeeds, the only information I have is that we were able to find a manifest file which is a case-insensitive match to Gopkg.toml. I do not actually have a second path to compare against.

I would suggest using a different approach and erroring only when using a case-senstive filesystem.

This would mean that dep behaves differently on case sensitive and case insensitive systems. Specifically, a package with gopkg.toml(or gopkg.TOML etc) would work fine on Windows/macOS but throw an error on linux. I am not sure if this would be the right behavior. Additionally, since the lock file is optional, that introduces more complexity for our check. If Gopkg.lock is not found, should we search all files for a case-insensitive match? And if we do find gopkg.lock, should we throw an error?

@sdboyer Could you please weigh in on this?

@sudo-suhas sudo-suhas force-pushed the check-cfg-filename-case branch 2 times, most recently from 2d40f42 to 723f1bf Compare September 4, 2017 06:36
return nil, errPathNotDir
}

// Ideally, we would use `os.Stat` for getting the actual file names
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: i'm not sure what line length rule you're following here, and we're not really stringent about following any particular one in this project. but, at least within a particular comment block, let's be consistent 😄

Copy link
Contributor Author

@sudo-suhas sudo-suhas Sep 5, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay. Wrapping at 90 chars.

project.go Outdated
// But error message will be a tad inaccurate.
actualMfName := actualFilenames[ManifestName]
if actualMfName != ManifestName {
return fmt.Errorf("manifest filename '%s' does not match '%s'", actualMfName, ManifestName)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: %q instead of '%s'

// Otherwise, it reads the contents of the directory
func ReadActualFilenames(dirPath string, names []string) (map[string]string, error) {
actualFilenames := make(map[string]string, len(names))
if len(names) <= 0 {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

len can't actually be less then zero, afaik - let's make it the clearer check len(names) == 0

defer dir.Close()

// Pass -1 to read all files in directory
files, err := dir.Readdir(-1)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it should be quite a bit cheaper to use Readdirnames() instead of Readdir().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think it will be cheaper. I skimmed through the source - https://golang.org/src/os/dir_windows.go. It actually uses dir.Readdir under the hood and only returns the names. By using dir.Readdir, we can avoid file name comparisons for non-regular(file.Mode().IsRegular()) files. Also, 1 less loop 😛

Copy link
Member

@sdboyer sdboyer Sep 5, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

that's the windows implementation, but the unix implementation is the reverse - Readdir relies on Readdirnames https://golang.org/src/os/dir_unix.go#L24. when we do it this way, both platforms are slow; when we do it with Readdirnames, only windows is slow.

we could benchmark it, but i imagine that constant factors of retrieving full file stat info for each node will drastically outweigh the reduction in N from skipping a some subset of the names.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh. Okay. I saw https://golang.org/src/os/dir_plan9.go which was same as windows. Didn't know unix was different. I am guessing the unix one is used on macOS as well. Changing this to use Readdirnames.

Copy link
Member

@sdboyer sdboyer Sep 5, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

still, i do appreciate the thoughtfulness!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ha ha.. That was because all the code I saw was super through. Very very good code comments etc.

@sdboyer
Copy link
Member

sdboyer commented Sep 5, 2017

Just to clarify, we are trying to prevent users from changing the letter-casing of Gopkg.toml and Gopkg.lock?

the goal is to make sure that we only consider files with the literal name Gopkg.toml and Gopkg.lock to be valid files. we're not trying to make things more convenient (which seems to be what @ibrasho is aiming at); rather, we're trying to strictly enforce the casing, so that the tool doesn't allow ambiguity in this case.

so, i think @sudo-suhas is correct here in his approach - we don't actually need to do work on case-sensitive filesystems, because we're already guaranteed that addressing the file with the exact name we expect will succeed only if the file exists with that exact name.

context_test.go Outdated
func TestLoadProjectCfgFileCase(t *testing.T) {
if runtime.GOOS != "windows" && runtime.GOOS != "darwin" {
t.Skip("skip this test on non-Windows, non-macOS")
}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to use fs.IsCaseSensitiveFilesystem instead? Since that's what we are testing.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am slightly unsure about this. The function checkCfgFilenames uses fs.IsCaseSensitiveFilesystem as well.

Copy link
Collaborator

@ibrasho ibrasho Sep 5, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say that's fine since checkCfgFilenames really depends on fs.IsCaseSensitiveFilesystem being correct to function properly.
(i.e. a bug in fs.IsCaseSensitiveFilesystem will most likely result in checkCfgFilenames not behaving properly.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But a bug in fs.IsCaseSensitiveFilesystem could prevent the test from being run.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ibrasho @sdboyer If you are sure about this, I will do the requested changes. Could you please confirm?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would vote to change this to fs.IsCaseSensitiveFilesystem, but I'll leave the final call to @sdboyer . 😄

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think, for this specific case (and the other test doing this check in this PR), we should probably prefer to have the test results for fs.IsCaseSensitiveFilesystem() and this test vary independently, so it's preferable to use the OS heuristic.

however, we need a comment explaining that this is the specific justification, as this is (or should be) the only place that we prefer the heuristic over doing the actual work of validating fs case sensitivity via fs.IsCaseSensitiveFilesystem().

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sdboyer, Could you please help me phrase the comment? Is this good enough - This is the only place that we prefer the heuristic over doing the actual work of validating fs case sensitivity via fs.IsCaseSensitiveFilesystem() ? While I get the gist, I am wondering if it can be phrased in simpler words.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i struggle with such phrasing things, as well :)

that seems good enough, but it also needs a bit that explains why we're avoiding it here - the reasons you gave earlier, about ensuring that our testing outcomes aren't correlated.

@@ -229,6 +231,112 @@ func TestRenameWithFallback(t *testing.T) {
}
}

func TestIsCaseSensitiveFilesystem(t *testing.T) {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a bit inaccurate. You can mount a case-sensitive filesystem on Linux, a case-insensitive one on Linux/macOS...

Can we think of a better way to check this rather than relying on the OS?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. But I cannot think of a better way. To some extent, I think we can make some assumptions about the test environment.

Copy link

@colek42 colek42 Sep 6, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will not work on Windows Subsystem for Linux. dF -Th will give you the actual file system type if the OS is linux. I'm not sure how it works with samba shares, etc.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colek42 Is Windows Subsystem really something that the tests should handle? Isn't it an edge case we can safely ignore?

Copy link

@colek42 colek42 Sep 7, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sudo-suhas it is easy enough to test for. See microsoft/WSL#423, can you just skip the test if it is WSL? Some of us are forced to use Windows.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some of us are forced to use Windows.

Ha ha.. Don't worry I am on windows too. I will look into this. Thank you.

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sudo-suhas let me know if you need me to test any changes.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure what's the best way to test this tbh. It's not easy to find a cross-platform solution for this as far as I know...

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sdboyer Do I need to check for BashOnWindows? How would I check this in golang? Is it as simple as a file read from /proc/version?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@sudo-suhas not here - this is just a test. unless we have an actual reason to believe the real logic (not the test logic) fails on WSL, then it's fine to omit here.

}

func TestReadActualFilenames(t *testing.T) {
if runtime.GOOS != "windows" && runtime.GOOS != "darwin" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to use fs.IsCaseSensitiveFilesystem instead?

project.go Outdated
// found. If it is found but the case does not match, an error is returned. If a lock
// file is not found, no error is returned as lock file is optional. If it is found but
// the case does not match, an error is returned.
func checkCfgFilenames(projectRoot string) error {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: I would prefer to name this checkGopkgFilenames. 😁

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, let's make that change.

project_test.go Outdated
@@ -61,6 +63,75 @@ func TestFindRoot(t *testing.T) {
}
}

func TestCheckCfgFilenames(t *testing.T) {
if runtime.GOOS != "windows" && runtime.GOOS != "darwin" {
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we change this to use fs.IsCaseSensitiveFilesystem instead?

{true, []string{invalidMfName}, manifestErrMsg(invalidMfName)},
// Error should be returned if the project has a valid manifest file and an
// invalid lock file.
{true, []string{ManifestName, invalidLfName}, lockErrMsg(invalidLfName)},
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens when the project has an invalidMfName and a valid LockName?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The check is sequential. So if the manifest file name is invalid an error is returned and we do not check the lock file.

@sudo-suhas sudo-suhas changed the title check cfg filename case on case insensitive systems check Gopkg filenames case on case insensitive systems Sep 7, 2017
@sdboyer
Copy link
Member

sdboyer commented Sep 11, 2017

@carolynvs i'm not entirely sure where the logic paths are for root vs. injected analyzer now, so could you have a quick look at this just to make sure that we're also rejecting improperly cased metadata file names in the injected analyzer?

(might take a minute for that, as she's at a conference this week)

@carolynvs
Copy link
Collaborator

There are lots of places in dep where we stat dep.ManifestName or dep.LockName. The injected analyzer:

dep/analyzer.go

Lines 19 to 23 in 28fb6b0

func (a Analyzer) HasDepMetadata(path string) bool {
mf := filepath.Join(path, ManifestName)
fileOK, err := fs.IsRegular(mf)
return err == nil && fileOK
}

Also the importers (for external config files):

func (g *glideImporter) HasDepMetadata(dir string) bool {
// Only require glide.yaml, the lock is optional
y := filepath.Join(dir, glideYamlName)
if _, err := os.Stat(y); err != nil {
return false
}
return true
}

Init also stats that filename to check if dep has already been run before:

dep/cmd/dep/init.go

Lines 102 to 112 in beb6bb1

mf := filepath.Join(root, dep.ManifestName)
lf := filepath.Join(root, dep.LockName)
vpath := filepath.Join(root, "vendor")
mok, err := fs.IsRegular(mf)
if err != nil {
return err
}
if mok {
return errors.Errorf("manifest already exists: %s", mf)
}

If all these need the same logic in checkCfgFilenames it would be helpful to have a parameterized function to safely handle checking if a file exists, without the caller needing to look at if the system is case-sensitive itself, or having to call ReadActualFilenames directly.

@sudo-suhas
Copy link
Contributor Author

@carolynvs Are all of these mentioned files in independent execution paths? For example, init is. What I am trying to say is, if the different file paths are part of the same command(ex: dep ensure), it shouldn't be necessary to check it in multiple places.

Also the importers (for external config files):

Do we really need to enforce case sensitive file name checks for external config files?

it would be helpful to have a parameterized function to safely handle checking if a file exists, without the caller needing to look at if the system is case-sensitive itself

Correct me if I am wrong here, but ReadActualFilenames fits this description.

@carolynvs
Copy link
Collaborator

Are all of these mentioned files in independent execution paths?

Each of the code snippets above could be the first check for a particular file in an execution path:

  • dep init -> init's check for existing dep config in the root.
  • dep init and dep ensure -> an importer's check for external config in a root or dependency.
  • dep ensure -> analyzer's check for dep config in a dependency.

Do we really need to enforce case sensitive file name checks for external config files?

I'm not sure about enforcing rather ensuring consistent behavior on both filesystems. However consider the following scenario: When I am on a mac and have a dependency on a package that uses glide, but the package has "GLIDE.LOCK", dep checks for "glide.lock", finds that and uses it during solve. But on linux, it wouldn't because the case is not what dep expected. Getting different solve results due to case-sensitive/insensitive filesystems is undesired.

ReadActualFilenames fits this description

I was suggesting something that wraps ReadActualFilenames because it's not a straightforward call that importers and others can swap out with os.Stat to check if a file exists. As I understand it, to use that function properly, one should first check if running on a case-sensitive filesystem, then know how to handle the resulting map. A function that could do the performance optimization, and return if the file exists with the exact name, or an error message explaining the desired filename and what was found, would be incredibly helpful. checkCfgFilenames does all of that but is hard-coded for the manifest and lock files.

I'm not saying any of this feedback must be addressed this PR (I'll leave that up to the original reviewers). I am just sharing context on where else in init/ensure dep does file checks that may be impacted by case-sensitive file systems.

@sudo-suhas
Copy link
Contributor Author

As I understand it, to use that function properly, one should first check if running on a case-sensitive filesystem, then know how to handle the resulting map.

There is no need to check if the filesystem is case-sensitive before calling ReadActualFilenames. The function behavior is consistent on different filesystems. It takes a directory and an array of names to read actual filenames for. On a case-sensitive system, it uses os.Stat to detect if the files exist. But on case-insensitive systems, it uses a more costly method to read the filenames(dir.Readdirnames). And this is the reason it accepts an array of filenames rather than a single filename. If a file is found, there will be an entry in the map returned with the actual filename as the value.

@carolynvs
Copy link
Collaborator

There is no need to check if the filesystem is case-sensitive before calling ReadActualFilenames.

This is a comment from the source code of this PR 😀

// ReadActualFilenames is actually costly. Since this check is not relevant to
// case-sensitive filesystems like ext4(linux), try for an early return.

None of my comments are changes requested for this PR, just context. I've provided it and will leave it up to the main reviewers from here.

@sudo-suhas
Copy link
Contributor Author

@carolynvs My apologies 😅. I will update the comment. Btw, just a thought. All the checks you mentioned, they are for the same folder(project root) right? So how about I define a helper struct which can be reused in all places? The struct would hold the actual filename for all files in the project root and would be created only once.

@sdboyer @ibrasho @darkowlzz I really don't mind doing any changes including the changes suggested by @carolynvs. But I would really appreciate it if you could finalise the requested changes so that I can tackle all of them.

@carolynvs
Copy link
Collaborator

carolynvs commented Sep 14, 2017

Btw, just a thought. All the checks you mentioned, they are for the same folder(project root) right?

The checks will be different directories once we start checking for external config in depndencies during solve.

project.go Outdated
@@ -43,10 +43,16 @@ func findProjectRoot(from string) (string, error) {
}

// checkCfgFilenames validates filename case for the manifest and lock files.
// This is relevant on case-insensitive systems like Windows.
//
// This is relevant on case-insensitive file systems like Windows and macOS.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: s/like Windows/like the defaults in Windows/

project.go Outdated
// found. If it is found but the case does not match, an error is returned. If a lock
// file is not found, no error is returned as lock file is optional. If it is found but
// the case does not match, an error is returned.
func checkCfgFilenames(projectRoot string) error {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, let's make that change.

context_test.go Outdated
func TestLoadProjectCfgFileCase(t *testing.T) {
if runtime.GOOS != "windows" && runtime.GOOS != "darwin" {
t.Skip("skip this test on non-Windows, non-macOS")
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i think, for this specific case (and the other test doing this check in this PR), we should probably prefer to have the test results for fs.IsCaseSensitiveFilesystem() and this test vary independently, so it's preferable to use the OS heuristic.

however, we need a comment explaining that this is the specific justification, as this is (or should be) the only place that we prefer the heuristic over doing the actual work of validating fs case sensitivity via fs.IsCaseSensitiveFilesystem().

@sudo-suhas sudo-suhas force-pushed the check-cfg-filename-case branch from d2eef8e to 277ba75 Compare September 20, 2017 06:02
- `fs`
  - Export `IsCaseSensitiveFilesystem`. Add and run test on windows, linux and
    macOS.
  - Add function `ReadActualFilenames` to read actual file names of given
    string slice. Add tests to be run on windows and macOS.
- `project`
  - Add function `checkCfgFilenames` to check the filenames for manifest
    and lock have the expected case. Use `fs#IsCaseSensitiveFilesystem`
    for an early return as the check is costly. Add test to be run on windows
    and macOS.
- `context`
  - Call `project.go#checkCfgFilenames` after resolving project root. Add test
    for invalid manifest file name to be run on windows and macOS.
- `project`
  - `checkCfgFilenames`
    - Improve function and code comments
    - Use boolean value indicating whether value was found in actual filenames.
      If manifest file was not found, return `errProjectNotFound`. Use boolean
      to determine if lock file was not found instead of length check.
  - `TestCheckCfgFilenames` - Add code comments for test cases explaining the
    expected behavior
- `fs`
  - `ReadActualFilenames`
    - Use cleaner check(`<=` ➡ `==`) to see if `names` parameter for
      `ReadActualFilenames` actually has any values.
    - Use `Readdirnames` instead of `Readdir`. This is expected to perform
      better in most cases.
  - `TestReadActualFilenames` - Add code comments for test cases explaining the
    expected behavior
- general
  - Make line length for code comments consistent(90), add periods where
    appropriate.
  - String formatting, use %q instead of '%s'
- `project.go`
  - Rename method {checkCfgFilenames => checkGopkgFilenames}
  - Improve funciton comment as suggested by @sdboyer
  - Fix ambigious comment explaining rationale behind early return.
- Add comment explaining why we do not use `fs.IsCaseSensitiveFilesystem` for
  skipping following tests:
  - context_test.go#TestLoadProjectGopkgFilenames
  - project_test.go#TestCheckGopkgFilenames
  - fs_test.go#TestReadActualFilenames
@sudo-suhas sudo-suhas force-pushed the check-cfg-filename-case branch from 277ba75 to d451fa0 Compare September 20, 2017 06:06
@sudo-suhas
Copy link
Contributor Author

@sdboyer I have done the requested changes. AFAIK the CI failure is not related to any changes I made. Please take a look.

@sdboyer
Copy link
Member

sdboyer commented Sep 22, 2017

yeah, the issue there was unrelated - #1200. kicking the job just for cleanliness' sake

@sdboyer
Copy link
Member

sdboyer commented Sep 22, 2017

idk why we have a fail from travis, everything passed. well, anyway - in we go, thanks! 🎉

@sdboyer sdboyer merged commit c564e78 into golang:master Sep 22, 2017
@sudo-suhas sudo-suhas deleted the check-cfg-filename-case branch September 22, 2017 13:22
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Suggestion: Check case of configuration filenames to avoid cross-platform incompatibilities
7 participants