Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

πŸš€ FEATURE: add queries function #2475

Merged
merged 20 commits into from
Jun 12, 2023
Merged

Conversation

dozheiny
Copy link
Contributor

@dozheiny dozheiny commented May 23, 2023

This PR adds a new function called Queries() to the Ctx struct. This function returns a map of query parameters and their values.

@dozheiny dozheiny changed the title Queries πŸš€ FEATURE: add queries function May 25, 2023
ctx_test.go Outdated Show resolved Hide resolved
@dozheiny dozheiny requested a review from gaby May 27, 2023 15:11
Copy link
Member

@gaby gaby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thoughts

ctx.go Outdated Show resolved Hide resolved
ctx.go Outdated Show resolved Hide resolved
ctx.go Outdated Show resolved Hide resolved
@dozheiny dozheiny requested a review from gaby May 27, 2023 16:34
Copy link
Member

@gaby gaby left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Member

@efectn efectn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add benchmark

@dozheiny
Copy link
Contributor Author

Sure, I'll add it

@dozheiny dozheiny requested a review from efectn May 27, 2023 21:01
Copy link
Member

@ReneWerner87 ReneWerner87 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the body of the method still needs to be corrected
encoded strings should not be taken into account, because the control characters are therefore invalid

@dozheiny
Copy link
Contributor Author

the body of the method still needs to be corrected encoded strings should not be taken into account, because the control characters are therefore invalid

Sure.

Copy link

@ghost ghost left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is more of a question to @ReneWerner87 I suppose: but why not use c.app.getString here? Is there a specific reason? It is way slower than using c.app.getString.

I ran a benchmark with string(...) and c.app.getString both 100 times, and the results are these:

with string(...) it takes an average of 1,006 ns/op, uses 418 B/op, and 11 allocs/op See Queries.xlsx

with c.app.getString it takes an average of 725 ns/op, uses 336 B/op, and 2 allocs/op
See QueriesPreAllocOpt.xlsx

This (string(...)) is also in GetReqHeaders, and GetRespHeaders, no other occurance of string(...) in ctx.go, and tests seem to pass just fine with c.app.getString as well, so is there a specific reason that string(...) was used there?

ctx.go Outdated Show resolved Hide resolved
@dozheiny dozheiny requested a review from a user May 31, 2023 20:47
@dozheiny
Copy link
Contributor Author

dozheiny commented Jun 3, 2023

@leonklingele I revert two suggestion commits because Linter gets an error.

@ghost
Copy link

ghost commented Jun 3, 2023

I revert two suggestion commits because Linter gets an error.

Linter failed because of timeout, not because of the code.
https://github.com/gofiber/fiber/actions/runs/5164476070/jobs/9303393316#step:4:25
https://github.com/gofiber/fiber/actions/runs/5164478717/jobs/9303397503#step:4:25

@dozheiny
Copy link
Contributor Author

dozheiny commented Jun 3, 2023

I revert two suggestion commits because Linter gets an error.

Linter failed because of timeout, not because of the code. https://github.com/gofiber/fiber/actions/runs/5164476070/jobs/9303393316#step:4:25 https://github.com/gofiber/fiber/actions/runs/5164478717/jobs/9303397503#step:4:25

Good to know; thanks!

@ReneWerner87
Copy link
Member

  1. should we split the depth structures and overtake some functions from here

    fiber/ctx.go

    Lines 1133 to 1141 in c955d76

    if strings.Contains(k, "[") {
    k, err = parseParamSquareBrackets(k)
    }
    if strings.Contains(v, ",") && equalFieldType(out, reflect.Slice, k) {
    values := strings.Split(v, ",")
    for i := 0; i < len(values); i++ {
    data[k] = append(data[k], values[i])
    }

  2. why did you revert the performance optimization with the app.getString function?

@dozheiny
Copy link
Contributor Author

dozheiny commented Jun 5, 2023

  1. should we split the depth structures and overtake some functions from here

    fiber/ctx.go

    Lines 1133 to 1141 in c955d76

    if strings.Contains(k, "[") {
    k, err = parseParamSquareBrackets(k)
    }
    if strings.Contains(v, ",") && equalFieldType(out, reflect.Slice, k) {
    values := strings.Split(v, ",")
    for i := 0; i < len(values); i++ {
    data[k] = append(data[k], values[i])
    }
  2. why did you revert the performance optimization with the app.getString function?

As for your first question, I don't think it's needed.
And for the second question, I'm sorry! I commit this again.

@ReneWerner87
Copy link
Member

We should think about the first point, because if we release it first, a change would be a breaking change.

I assume that the users also assume that the array values are split and stored in a deep structure, because otherwise you would have to do it yourself.
With the queryparser many also assumed this

@dozheiny
Copy link
Contributor Author

dozheiny commented Jun 6, 2023

We should think about the first point, because if we release it first, a change would be a breaking change.

I assume that the users also assume that the array values are split and stored in a deep structure, because otherwise you would have to do it yourself. With the queryparser many also assumed this

You are right, but the output of this function is a map[string]string.
If we want to survey the array values of the queries, it replaces them with the latest value.
Like this:

c.Request().URI().SetQueryString("list=1,2,3")
queries := c.Queries()
queries["list"] // 3

Do you have any suggestions about the output of the queries function?

@ReneWerner87
Copy link
Member

would not ignore any values as no one expects this
we should then change the type of the map and parse infinitely deep

@gaby @efectn what do you think?

@efectn
Copy link
Member

efectn commented Jun 6, 2023

Returning map[string][]string seems better to me

@ReneWerner87
Copy link
Member

Or map[string]interface{}

@dozheiny
Copy link
Contributor Author

dozheiny commented Jun 6, 2023

Or map[string]interface{}

If that map[string]interface{}, a user should convert it into a string or array of series.
Returning map[string][]string makes more sense.

@ReneWerner87
Copy link
Member

And what happens for these examples?

GET /api/posts?filters.author.name=John&filters.category.name=Technology
GET /api/orders?filters[customer][name]=Alice&filters[status]=pending

@ReneWerner87
Copy link
Member

Some more interesting examples which we should consider

GET /api/search?filters[category]=electronics&filters[tags][]=apple&filters[tags][]=orange&filters[tags][]=banana
GET /api/search?tags=apple,orange,banana
GET /api/search?filters[tags]=apple,orange,banana&filters[category][name]=fruits
GET /api/search?filters.tags=apple,orange,banana&filters.category.name=fruits
GET /api/search?filters.tags.0=apple&filters.tags.1=orange&filters.tags.2=banana&filters.category.name=fruits
GET /api/search?filters[tags][0]=apple&filters[tags][1]=orange&filters[tags][2]=banana&filters[category][name]=fruits

Can you include these examples in the test and document the expectations

@dozheiny
Copy link
Contributor Author

dozheiny commented Jun 7, 2023

Sure, I add this test case and documents.

@dozheiny
Copy link
Contributor Author

dozheiny commented Jun 9, 2023

@ReneWerner87 shouldn't we make output of function to map[string][]string?
I'm just asking because you approve PR.

@ReneWerner87
Copy link
Member

Not really, I thought about it again and thought that it is ok, as long as we describe it well in the documentation.
Then you get exactly, the key value pairs which are to be recognized also in the query url, without any postprocessing

@ReneWerner87 ReneWerner87 merged commit d87065f into gofiber:master Jun 12, 2023
Skyenought pushed a commit to Skyenought/fiber that referenced this pull request Jun 15, 2023
* πŸš€ FEATURE: add queries method

* πŸ“š DOCS: add documents for queries method.

* 🩹 Fix: fix wrap error returned from Queries function

* 🚨 tests: add url encoded tests

* πŸ”₯ feature: add url encoded support for queries

* 🩹 Fix: fix wrap error returned from Queries function

* ♻️ Refactor: change error message of url.QueryUnescape

* ♻️ Refactor: refactor of mapping key and value queries

* 🚨 Test: Validate to fail parse queries

* 🚨 Test: Add benchmark test for Queries

* 🩹 Fix: remove parsing for encoded urls

* ♻️ Refactor: change string function to c.app.getString fucntion

Co-authored-by: cmd777 <[email protected]>

* ♻️ Refactor: change name of benchamark function ctx queries

Co-authored-by: leonklingele <[email protected]>

* ♻️ Refactor: remove empty lines

Co-authored-by: leonklingele <[email protected]>

* Revert "♻️ Refactor: change string function to c.app.getString fucntion"

This reverts commit 28febf9.

* πŸ“š Docs: add documents for queries method

* 🚨 Tests: add more tests for queries function

* ♻️ Refactor: change string function to c.app.getString function

* 🚨 Tests: add more test for queries function

* πŸ“š Docs: add more documents to queries function

---------

Co-authored-by: cmd777 <[email protected]>
Co-authored-by: leonklingele <[email protected]>
@dozheiny dozheiny deleted the queries branch November 5, 2023 20:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants