Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix NuGet Package API for $filter with Id equality #31188

Merged
merged 12 commits into from
Jun 4, 2024

Conversation

tdesveaux
Copy link
Contributor

Fixes issue when running choco info pkgname where pkgname is also a substring of another package Id.

Relates to #31168


This might fix the issue linked, but I'd like to test it with more choco commands before closing the issue in case I find other problems if that's ok.
I'm pretty inexperienced with Go, so feel free to nitpick things.

Not sure I handled this in the best way, so looking for feedback on if I should fix the underlying issue (nil might be a better default for Value?).

@GiteaBot GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label May 30, 2024
@pull-request-size pull-request-size bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 30, 2024
@github-actions github-actions bot added modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code labels May 30, 2024
@KN4CK3R
Copy link
Member

KN4CK3R commented May 31, 2024

Thank you for the fix. Isn't a change like this enough?

@@ -96,20 +96,34 @@ func FeedCapabilityResource(ctx *context.Context) {
        xmlResponse(ctx, http.StatusOK, Metadata)
 }
 
-var searchTermExtract = regexp.MustCompile(`'([^']+)'`)
+var (
+       searchTermExtract = regexp.MustCompile(`'([^']+)'`)
+       searchTermExact   = regexp.MustCompile(`\s+eq\s+'`)
+)
 
-func getSearchTerm(ctx *context.Context) string {
+func getSearchTerm(ctx *context.Context) packages_model.SearchValue {
        searchTerm := strings.Trim(ctx.FormTrim("searchTerm"), "'")
-       if searchTerm == "" {
-               // $filter contains a query like:
-               // (((Id ne null) and substringof('microsoft',tolower(Id)))
-               // We don't support these queries, just extract the search term.
-               match := searchTermExtract.FindStringSubmatch(ctx.FormTrim("$filter"))
-               if len(match) == 2 {
-                       searchTerm = strings.TrimSpace(match[1])
+       if searchTerm != "" {
+               return packages_model.SearchValue{
+                       Value:      searchTerm,
+                       ExactMatch: false,
+               }
+       }
+
+       // $filter contains a query like:
+       // (((Id ne null) and substringof('microsoft',tolower(Id)))
+       // https://www.odata.org/documentation/odata-version-2-0/uri-conventions/ section 4.5
+       // We don't support these queries, just extract the search term.
+       filter := ctx.FormTrim("$filter")
+       match := searchTermExtract.FindStringSubmatch(filter)
+       if len(match) == 2 {
+               return packages_model.SearchValue{
+                       Value:      strings.TrimSpace(match[1]),
+                       ExactMatch: searchTermExact.MatchString(filter),
                }
        }
-       return searchTerm
+
+       return packages_model.SearchValue{}
 }
 
 // https://github.com/NuGet/NuGet.Client/blob/dev/src/NuGet.Core/NuGet.Protocol/LegacyFeed/V2FeedQueryBuilder.cs
@@ -118,11 +132,9 @@ func SearchServiceV2(ctx *context.Context) {
        paginator := db.NewAbsoluteListOptions(skip, take)
 
        pvs, total, err := packages_model.SearchLatestVersions(ctx, &packages_model.PackageSearchOptions{
-               OwnerID: ctx.Package.Owner.ID,
-               Type:    packages_model.TypeNuGet,
-               Name: packages_model.SearchValue{
-                       Value: getSearchTerm(ctx),
-               },
+               OwnerID:    ctx.Package.Owner.ID,
+               Type:       packages_model.TypeNuGet,
+               Name:       getSearchTerm(ctx),
                IsInternal: optional.Some(false),
                Paginator:  paginator,
        })
@@ -169,10 +181,8 @@ func SearchServiceV2(ctx *context.Context) {
 // http://docs.oasis-open.org/odata/odata/v4.0/errata03/os/complete/part2-url-conventions/odata-v4.0-errata03-os-part2-url-conventions-complete.html#_Toc453752351
 func SearchServiceV2Count(ctx *context.Context) {
        count, err := nuget_model.CountPackages(ctx, &packages_model.PackageSearchOptions{
-               OwnerID: ctx.Package.Owner.ID,
-               Name: packages_model.SearchValue{
-                       Value: getSearchTerm(ctx),
-               },
+               OwnerID:    ctx.Package.Owner.ID,
+               Name:       getSearchTerm(ctx),
                IsInternal: optional.Some(false),
        })
        if err != nil {

Could you test if this fixes your choco queries?

@tdesveaux
Copy link
Contributor Author

I added the queries choco make with it's commands in the issue here

I also tried a choco info '' and it failed before querying, so the edge case 'issue' I found can be ignore for this case.

Thank you for the fix. Isn't a change like this enough?

If I removed the edge-case handling of .Value = "" filter, and added the missing patch of Packages/$count (and code clarity).
The only difference is on the regex used to define if the search is an exact one.
I don't mind doing it this way, but any reason to only look for eq vs the pattern I used which is more cautious and will only match on Id?

Could you test if this fixes your choco queries?

The tests I added pass so I think it's fine.
I'll take time later to test the real choco query.

This is cleaner (and correctly fix the / route).
With the drawback of not correctly handling "$filter=Id eq ''" queries, so it's ignored in tests.
@tdesveaux
Copy link
Contributor Author

@KN4CK3R I updated the branch with your implementation suggestion.

There is still the edge-case of Id eq '' but it's an improbable query (at least from choco). So I simply ignored the test case.
I tested with a local build and choco works as expected now.

@GiteaBot GiteaBot added lgtm/need 1 This PR needs approval from one additional maintainer to be merged. and removed lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. labels Jun 3, 2024
Copy link
Member

@KN4CK3R KN4CK3R left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for the tests.

@GiteaBot GiteaBot added lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. and removed lgtm/need 1 This PR needs approval from one additional maintainer to be merged. labels Jun 3, 2024
@KN4CK3R KN4CK3R enabled auto-merge (squash) June 3, 2024 18:24
@lunny lunny added the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Jun 4, 2024
@KN4CK3R KN4CK3R merged commit c888c93 into go-gitea:main Jun 4, 2024
26 checks passed
@GiteaBot GiteaBot added this to the 1.23.0 milestone Jun 4, 2024
@GiteaBot GiteaBot removed the reviewed/wait-merge This pull request is part of the merge queue. It will be merged soon. label Jun 4, 2024
@tdesveaux
Copy link
Contributor Author

@KN4CK3R apologies for the ping.

Any change this would be backported to 1.22?

CONTRIBUTING.md mention that large PRs are not backport, and I don't know how the large qualifier is attributed, but isn't it set due to the changes in tests?

@tdesveaux tdesveaux deleted the fix/issue-31168 branch June 4, 2024 09:48
@KN4CK3R
Copy link
Member

KN4CK3R commented Jun 4, 2024

You can backport it to 1.22 because it's a bug fix.

tdesveaux added a commit to tdesveaux/gitea that referenced this pull request Jun 4, 2024
Fixes issue when running `choco info pkgname` where `pkgname` is also a
substring of another package Id.

Relates to go-gitea#31168

---

This might fix the issue linked, but I'd like to test it with more choco
commands before closing the issue in case I find other problems if
that's ok.

---------

Co-authored-by: KN4CK3R <[email protected]>
@silverwind
Copy link
Member

silverwind commented Jun 4, 2024

Should have automatically backported, but ok :)

@tdesveaux
Copy link
Contributor Author

Should have automatically backported, but ok :)

Might have misinterpreted KN4CK3R 's message above.
Should I have done something or waited on your side?

@KN4CK3R
Copy link
Member

KN4CK3R commented Jun 4, 2024

Everything fine, the Giteabot can create backports too.

@KN4CK3R KN4CK3R added backport/manual No power to the bots! Create your backport yourself! backport/v1.22 This PR should be backported to Gitea 1.22 labels Jun 4, 2024
@silverwind
Copy link
Member

silverwind commented Jun 4, 2024

It's preferably if the bot does it, because then we know the content is exactly the same. Manual backport like you did is only when the bot fails to do it because of merge conflict.

Let's see if the bot does it now...

lafriks pushed a commit that referenced this pull request Jun 4, 2024
Backport #31188

Fixes issue when running `choco info pkgname` where `pkgname` is also a
substring of another package Id.

Relates to #31168

---

This might fix the issue linked, but I'd like to test it with more choco
commands before closing the issue in case I find other problems if
that's ok.
I'm pretty inexperienced with Go, so feel free to nitpick things.

Not sure I handled
[this](https://github.com/tdesveaux/gitea/blob/70f87e11b5caf1ee441ae71c7eba1831f45897d4/routers/api/packages/nuget/nuget.go#L135-L137)
in the best way, so looking for feedback on if I should fix the
underlying issue (`nil` might be a better default for `Value`?).

Co-authored-by: KN4CK3R <[email protected]>
@silverwind silverwind added backport/done All backports for this PR have been created and removed backport/v1.22 This PR should be backported to Gitea 1.22 backport/manual No power to the bots! Create your backport yourself! labels Jun 4, 2024
@silverwind
Copy link
Member

It didn't because of the backport/manual flag. Next time try via bot first.

tdesveaux added a commit to dontnod/fork-gitea that referenced this pull request Jun 4, 2024
Fixes issue when running `choco info pkgname` where `pkgname` is also a
substring of another package Id.

Relates to go-gitea#31168

---

This might fix the issue linked, but I'd like to test it with more choco
commands before closing the issue in case I find other problems if
that's ok.

---------

Co-authored-by: KN4CK3R <[email protected]>
zjjhot added a commit to zjjhot/gitea that referenced this pull request Jun 5, 2024
* giteaofficial/main:
  Add `lint-go-gopls` (go-gitea#30729)
  Make blockquote attention recognize more syntaxes (go-gitea#31240)
  Fix admin oauth2 custom URL settings (go-gitea#31246)
  Replace `gt-word-break` with `tw-break-anywhere` (go-gitea#31183)
  Make pasted "img" tag has the same behavior as markdown image (go-gitea#31235)
  Remove .segment from .project-column (go-gitea#31204)
  Fix overflow on push notification (go-gitea#31179)
  Fix NuGet Package API for $filter with Id equality  (go-gitea#31188)
  Fix overflow on notifications (go-gitea#31178)
  Update chroma to v2.14.0 (go-gitea#31177)
  Update air package path (go-gitea#31233)
  Bump `@github/relative-time-element` to v4.4.1 (go-gitea#31232)
silverwind added a commit to silverwind/gitea that referenced this pull request Jun 6, 2024
* origin/main: (231 commits)
  Allow including `Reviewed-on`/`Reviewed-by` lines for custom merge messages (go-gitea#31211)
  Add `MAX_ROWS` option for CSV rendering (go-gitea#30268)
  Update `golang.org/x/net` (go-gitea#31260)
  Add replacement module for `mholt/archiver` (go-gitea#31267)
  Fix Activity Page Contributors dropdown (go-gitea#31264)
  Optimize runner-tags layout to enhance visual experience (go-gitea#31258)
  fix: allow actions artifacts storage migration to complete succesfully (go-gitea#31251)
  Add `lint-go-gopls` (go-gitea#30729)
  Make blockquote attention recognize more syntaxes (go-gitea#31240)
  Fix admin oauth2 custom URL settings (go-gitea#31246)
  Replace `gt-word-break` with `tw-break-anywhere` (go-gitea#31183)
  Make pasted "img" tag has the same behavior as markdown image (go-gitea#31235)
  Remove .segment from .project-column (go-gitea#31204)
  Fix overflow on push notification (go-gitea#31179)
  Fix NuGet Package API for $filter with Id equality  (go-gitea#31188)
  Fix overflow on notifications (go-gitea#31178)
  Update chroma to v2.14.0 (go-gitea#31177)
  Update air package path (go-gitea#31233)
  Bump `@github/relative-time-element` to v4.4.1 (go-gitea#31232)
  Add option for mailer to override mail headers (go-gitea#27860)
  ...
@go-gitea go-gitea locked as resolved and limited conversation to collaborators Sep 2, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
backport/done All backports for this PR have been created lgtm/done This PR has enough approvals to get merged. There are no important open reservations anymore. modifies/api This PR adds API routes or modifies them modifies/go Pull requests that update Go code size/L Denotes a PR that changes 100-499 lines, ignoring generated files. topic/packages type/bug
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants