Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix: Implementing a limitation for the open() #2314

Merged
merged 5 commits into from
Jan 12, 2022
Merged

Conversation

olegbespalov
Copy link
Contributor

Implementing a limitation for the open() that limits opening files
by the list of the files that were opened during the initialization
step (__VU == 0).

For example, a code like:

if (__VU >0) {
   JSON.parse(open("./arr.json"));
}

Should return an error.

Closes #1771

@olegbespalov olegbespalov self-assigned this Jan 4, 2022
@na-- na-- added this to the v0.36.0 milestone Jan 5, 2022
Copy link
Contributor

@mstoykov mstoykov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nicely done 👏 !

I have left some nitpick comments, but mostly I want the message fixed ... possibly when the file is missing as well :).
I have tested it and it seems to work great with archives as well ;)

lib/fsext/cacheonread.go Outdated Show resolved Hide resolved
lib/fsext/cacheonread.go Outdated Show resolved Hide resolved
js/runner_test.go Outdated Show resolved Hide resolved
js/initcontext.go Outdated Show resolved Hide resolved
js/initcontext.go Show resolved Hide resolved
@olegbespalov
Copy link
Contributor Author

Thanks, @mstoykov 🙇

Addressed all comments 👍

@olegbespalov olegbespalov requested a review from mstoykov January 5, 2022 15:29
lib/fsext/cacheonread.go Outdated Show resolved Hide resolved
mstoykov
mstoykov previously approved these changes Jan 6, 2022
Copy link
Contributor

@mstoykov mstoykov left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM 👏 I have 2 more comments with notes, but I wouldn't argue you should work on them now especially if the plan is to move to io/fs in the next release, which will in practice redo all of this either way.

Comment on lines 317 to 326
defer func() {
if errors.Is(err, fsext.ErrPathNeverRequestedBefore) {
// loading different files per VU is not supported, so all files should are going
// to be used inside the scenario should be opened during the init step (without any conditions)
err = fmt.Errorf(
"open() can't be used under with files that weren't opened during initialization (__VU==0), path: %q",
filename,
)
}
}()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note(not actionable): Technically, this error will either happen on afero.IsDir or not at all. But it seems kind of strange to check it only there and also when/if we ever stop checking that as we move to io/fs it will be a bit confusing why it wasn't previously checked.

I am noting this for future reference only, don't do anything about it just based on that.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤔 not sure if I get it. The error happens afero.IsDir and afero.ReadFile. Even if we move to the io/fs still should happen on the equivalent of the afero.ReadFile, no?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we move – yes, but when we move to io/fs, afero.IsDir will hopefully be just dropped ;).

What I am trying to say is that if afero.IsDir did not return fsext.ErrPathNeverRequestedBefore than afero.Readfile will not return it either ... or we have bugs I guess or some strange race condition.

I wondered for a bit if we can't just add afero.IsDir check in the already reimplemented Open , but I am not certain we always use the CacheOnReadFs and IIRC we don't so 🤷

Comment on lines 340 to 348
fs := i.filesystems["file"]

alreadyOpenedFS, ok := fs.(fsext.OnlyOpenedEnabler)
if !ok {
return
}

alreadyOpenedFS.AllowOnlyOpened()
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note(not actionable): This seems a bit fragile, as we can at any point change what the top fs for file is and then this suddenly won't work. This is unlikely as this code is very rarely changed and will probably be spotted.

The test below also doesn't test with loader.CreateFilesystems which is what is used in reality. An integration test with it might be better, but I see no test actually has ever used it so 🤷 maybe there are some problems I can't remember at the moment. Either way this is fine for now, hopefully we will be moving to io/fs soon and that will lead to some bigger changes and hopefully more tests.

Copy link
Contributor

@yorugac yorugac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall 👍 but I have a renaming request and some smaller cleanup remarks.

One more thing. To improve test coverage it might also make sense to add tests for successful cases (i.e. with if (__VU==0) and with absent if condition)?

js/runner_test.go Outdated Show resolved Hide resolved
js/initcontext.go Outdated Show resolved Hide resolved
js/runner_test.go Outdated Show resolved Hide resolved
lib/fsext/cacheonread.go Outdated Show resolved Hide resolved
js/runner_test.go Outdated Show resolved Hide resolved
@olegbespalov olegbespalov added breaking change for PRs that need to be mentioned in the breaking changes section of the release notes documentation-needed A PR which will need a separate PR for documentation labels Jan 10, 2022
mstoykov
mstoykov previously approved these changes Jan 11, 2022
yorugac
yorugac previously approved these changes Jan 11, 2022
Copy link
Contributor

@yorugac yorugac left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the renaming! LGTM 👍

olegbespalov and others added 5 commits January 11, 2022 18:03
Implementing a limitation for the open() that limits opening files
by the list of the files that were opened during the initialization
step (__VU == 0).

For example, a code like:

```js
if (__VU >0) {
   JSON.parse(open("./arr.json"));
}
```

Should return an error.

Closes #1771
Co-authored-by: Mihail Stoykov <[email protected]>
@mstoykov mstoykov requested a review from yorugac January 12, 2022 10:24
@olegbespalov olegbespalov merged commit 57ec741 into master Jan 12, 2022
@olegbespalov olegbespalov deleted the fix/1771-fixes branch January 12, 2022 10:27
js/bundle.go Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
breaking change for PRs that need to be mentioned in the breaking changes section of the release notes documentation-needed A PR which will need a separate PR for documentation
Projects
None yet
Development

Successfully merging this pull request may close these issues.

open() seems to not always return the full data if that wasn't loaded in __VU =0
4 participants