Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Resolve inconsistent processing logic in mux.ServeHTTP #536

Merged
merged 8 commits into from
Mar 14, 2022

Conversation

samutamm
Copy link
Contributor

@samutamm samutamm commented Mar 9, 2022

Fix #535.

At high level this PR aims to fix the issue, that when there are two httpserver.mux rules, with same path but different Method, only first one is considered.

This involves following changes:

@codecov-commenter
Copy link

codecov-commenter commented Mar 9, 2022

Codecov Report

Merging #536 (aff281a) into main (dce1b84) will decrease coverage by 3.99%.
The diff coverage is 58.00%.

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #536      +/-   ##
==========================================
- Coverage   80.63%   76.64%   -4.00%     
==========================================
  Files          88       94       +6     
  Lines       10122    10807     +685     
==========================================
+ Hits         8162     8283     +121     
- Misses       1510     2064     +554     
- Partials      450      460      +10     
Impacted Files Coverage Δ
pkg/object/httpserver/cache.go 0.00% <ø> (ø)
pkg/object/httpserver/mux.go 32.08% <58.00%> (ø)
pkg/cluster/cluster.go 51.87% <0.00%> (-1.03%) ⬇️
pkg/object/httpserver/spec.go 2.85% <0.00%> (ø)
pkg/object/httpserver/httpserver.go 36.36% <0.00%> (ø)
pkg/object/httpserver/runtime.go 0.00% <0.00%> (ø)
pkg/object/httpserver/context.go 0.00% <0.00%> (ø)
pkg/object/mqttproxy/client.go 80.54% <0.00%> (+0.90%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update dce1b84...aff281a. Read the comment docs.

@localvar
Copy link
Collaborator

IMHO, packages in the utils folder should only depend on the standard packages or 3rd party packages (although there are already exceptions).

The pathsearch package of this PR depends on the context package, and its interface/implementation is also tightly bound to HTTP, so I propose not to put it in the utils folder, and maybe we don't need to define the MuxPathInterface.

@samutamm
Copy link
Contributor Author

IMHO, packages in the utils folder should only depend on the standard packages or 3rd party packages (although there are already exceptions).

The pathsearch package of this PR depends on the context package, and its interface/implementation is also tightly bound to HTTP, so I propose not to put it in the utils folder, and maybe we don't need to define the MuxPathInterface.

Good point, I updated the PR and the description. It now involves less refactoring

@haoel
Copy link
Contributor

haoel commented Mar 10, 2022

@aniaan please help to review this PR.

Copy link
Collaborator

@aniaan aniaan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, one small suggestion, we should use path + method + header as the unique key of path, the ip check should be checked after matching the path, in addition this piece will be better if there is a unit test coverage


if !path.pass(ctx) {
m.handleIPNotAllow(ctx)
return
return IPNotAllowed, path
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A little suggestion, our path matching rule should be matchPath + matchMethod + matchHeader, after the above three conditions are matched, only then request is considered to match the path, so only when the above conditions are met, should we check whether the IP is passed.

if !path.matchMethod(ctx) {
	continue
}
// at least one path has correct method
methodAllowed = true

var searchResult SearchResult

if path.hasHeaders() {
	if !path.matchHeaders(ctx) {
		continue
	}
	searchResult = FoundSkipCache
} else {
	searchResult = Found
}

if !path.pass(ctx) {
	return IPNotAllowed, nil
}

return searchResult, path

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, we can implement the following rule configuration

rules:
  - paths:
    - pathPrefix: /pipeline
      method: POST
      headers:
       X-version: v1
      ip:
        AllowIPs:
         - "8.8.8.8"
      backend: post-v1-demo
    
    - pathPrefix: /pipeline
      method: POST
      headers:
       X-version: v2
      ip:
        AllowIPs:
         - "9.9.9.9"
      backend: post-v2-demo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for review @aniaan ! I agree that it's better to have unit tests here so that it would be easier to make these modifications, however currently it would require adding unit test for the whole httpserver module.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like your suggestion about routing based on IP and headers.. I think that when the path does not define them, route should match all headers and IPs. If header or IP is defined, it will be used to match and request causes MethodNotAllowed if no correct header or IP is present in the request.

So this will match requests with correct path and method and any IP or any or none header

pathPrefix: /pipeline
method: POST
backend: post-v2-demo

and in your example it's MethodNotAllowed if request path is correct but header or IP is not present in the request. @aniaan Is this what you mean?

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe my description is not very accurate, I want to say is that a path matching rule is path + method + headers, ip is not part of the path matching, we have to put the headers check to the front, if the path + method + headers are matched to complete, that the user is matching the rule, and then check whether the ip is passed, ip if passed, go backend, not passed, directly return 403 response.

image

@xxx7xxxx
Copy link
Contributor

Good refactoring, but please fix the errors

@samutamm
Copy link
Contributor Author

Good refactoring, but please fix the errors

Fixed!

@samutamm samutamm merged commit 0c02071 into easegress-io:main Mar 14, 2022
@samutamm samutamm deleted the duplicate-path-fix-535 branch March 14, 2022 10:03
samutamm pushed a commit to samutamm/easegress that referenced this pull request Mar 17, 2022
)

* separate path search to own function

* move to pathsearch to utils module and unittest

* remove useless file

* move SearchPath to original httpserver module

* make fmt

* update SearchPath and add unittest for it

* fix path NotFound case and add tests

* fix code analysis issue
localvar pushed a commit that referenced this pull request Mar 29, 2022
* separate path search to own function

* move to pathsearch to utils module and unittest

* remove useless file

* move SearchPath to original httpserver module

* make fmt

* update SearchPath and add unittest for it

* fix path NotFound case and add tests

* fix code analysis issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inconsistent processing logic for matchMethod and matchHeaders in mux.ServeHTTP.
7 participants