Some Indexers don't return all results for id based searches

[ThatIcyPolarBear]

Is there an existing issue for this?

• I have searched the existing open and closed issues

Current Behavior

When using parameters to do a search, not al parameters are used in the link used to grab the results.

I'm not sure how I can explain it better so I'll just give an example:
When I search for a season of a show, sonarr searchs via prowlarr using the parameters: {ImdbId:tt1520211}{season:2}.
Those parameters get parsed correctly, as they're visible in prowlarrs log entry:
2024-04-01 23:02:48.7|Info|ReleaseSearchService|Searching indexer(s): [TheRARBG] for Term: [] | ID(s): IMDbId:[1520211] for Season / Episode:[S02], Offset: 0, Limit: 100, Categories: [2000, 5000, 5080]
However, when I go to the history tab and check the link it used to grab the results I see this:
https://t-rb.org/get-posts/order:-se:category:Movies:category:TV:category:Anime:category:Documentaries:keywords:tt1520211:paginate_by:100:format:json/
which only includes the ImdbId but not the season. This causes the results to vary wildly, they're all the right show of course but not a single one of the 100 results was of the correct season, which causes sonnar of course to fail completely.

If you click on the indexer you can see that the Search Capabilities do say that it supports those parameters, which makes sense because sonarr uses those parameters automatically for that indexer, and not for others that don't support it. Which makes it so only indexers who don't support it work correctly.

Reordering of the parameters has no effect, but removing the ImdbId does cause it to return results of Season 2, but this time of completely random shows.

I looked through the settings of sonarr and prowlarr but couldn't find any settings related to the parameters or Search Capabilities so I figured the next most sensible thing was opening a bug report.

Expected Behavior

When multiple parameters are given they should all be added to the link prowlarr generates.

Steps To Reproduce

1. Use an indexer that with Search Capabilities that support the parameters ImdbId and Season, or multiple parameters in general
2. Go to the search tab and in the query options change from Basic search to TV search
3. Do a search using that indexer and multiple parameters, for example {ImdbId:tt1520211}{season:2}
4. Watch it return results of the correct show but not of the correct season

Environment

- OS: Unraid 6.12.9
- Prowlarr: 1.15.0.4361
- Docker Install: Yes
- Using Reverse Proxy: No
- Browser: Firefox 124.0.1

What branch are you running?

Master

Trace Logs?

https://gist.github.com/ThatIcyPolarBear/e969e1e63cb02cfaeb6f673bac5dffa5

Trace Logs have been provided as applicable. Reports may be closed if the required logs are not provided.

• I have read and followed the steps in the wiki link above and provided the required trace logs - the logs contain trace - that are relevant and show this issue.

[mynameisbogdan]

This is most likely a limitation of the tracker no supporting both id and text to be used in the same time.

But Prowlarr works fine with multiple parameters.

[ThatIcyPolarBear]

This is most likely a limitation of the tracker no supporting both id and text to be used in the same time.

But Prowlarr works fine with multiple parameters.

Hmm I did some more snooping around and you seem to be at least mostly right.
I tried it again on another Indexer with the same Search Capabilities but got the same result. I don't have / know any other indexers with those search capabilities so I can't test if it's just an issue with those two.
However I did test it again using a query to get the right show and then two parameters for the season and the episode and this time it did return the correct results.

So it seems that it indeed is only an issue when an id is provided. Checking in the search box of the site of the first indexer seems to prove it.

However this doesn't fix that sonarr will automatically use the id parameter if prowlarr reports that it's supported by the indexer. Is there any possible fix that can be done?

[mynameisbogdan]

You can snoop the Indexers repository for the definitions for you indexers.

For THERARBG you can check it here where the condition is either one of them, not both in the same time:

Indexers/definitions/v9/therarbg.yml

Line 71 in 4861ac8

- path: "get-posts/order:{{ .Config.sort }}{{ range .Categories }}:category:{{.}}{{end}}{{ if or .Query.IMDBID .Keywords }}:keywords:{{ or .Query.IMDBID .Keywords }}{{ else }}{{ end }}:paginate_by:100:format:json/"

These are just search limitations of the sites.

[ilike2burnthing]

Prowlarr/Prowlarr#611

[mynameisbogdan]

I don't know if Prowlarr should disable id based searches for tvsearch same as Jackett...

[ilike2burnthing]

This was disabled in Jackett with the comment introduce setter so individual trackers can override. I think that's the right approach, default to false, as generally searching by both imdbid and keywords together isn't supported by trackers.

While some indexers would need reworked to support this, many are already set up to work with that simple setting change, e.g.:

https://github.com/Jackett/Jackett/blob/3b173e96e72dbfaf19fc800bb08854ded2f563af/src/Jackett.Common/Definitions/rotorrent.yml#L95

(linking to the Jackett version as it seems rotorrent.yml is missing here)

To save managing more differences between Jackett and Prowlarr definitions, this would probably be best implemented on both at the same time (I say that knowing that I won't be the one doing the work 😛).

[mynameisbogdan]

TBH considering the possible issues with the release titles the different languages, I would prefer id based searches returning all seasons instead of no results at all.

[ilike2burnthing]

The issue was that for trackers which only returned 40 or 20 results per page, even if we grabbed 2 pages there was still a chance that relevant results wouldn't be returned (e.g. a series has 6 seasons, user wants S02E13, but that doesn't appear until page 4 as results are sorted by descending time). However, because any results were returned, Sonarr doesn't perform a title search, and therefore Sonarr finds no relevant results. Even a tracker that returned 200 results still wouldn't show everything for a big enough series.

We have SupportsRawSearch/allowrawsearch, which we default to false, and then enable where appropriate. I think we could do the same here with SupportsImdbTVSearch (and an equivalent allowimdbtvsearch).

[mynameisbogdan]

Seems overkill, we can simply add or remove imdbid from tvsearch caps. 🤔

[ilike2burnthing]

~200 indexers currently list imdbid TV support, not all of which we have access to, so that seems like a lot of work as well. I'm not sure which would be more.

I'm also not sure if there are any other (non-arr) projects, or even just scripts, which rely on imdbid TV search, but don't check SupportsImdbTVSearch.

[bakerboy448]

I vote for the disable by default and do a new flag to enable it like was planned / not implemented for jackett.

perhaps we can use the # engine tag to help facilitate more bulk changes?

imdbid only for tv often causes more issues historically from a support view

[mynameisbogdan]

I disagree. 👺

[bakerboy448]

Soooo @mynameisbogdan @ilike2burnthing where we going with this?

[mynameisbogdan]

Status quo feels like the right thing.

[ilike2burnthing]

I wouldn't be the one implementing it, so I'll take Bogdan's lead.

Maybe we could add a comment to the indexers, for those trackers which technically support IMDB ID search for TV releases, but not with season and/or episode information? e.g.:

  modes:
    search: [q]
    tv-search: [q, season, ep] # tracker imdbid search cannot be used with season/ep
    movie-search: [q, imdbid]
    music-search: [q]

[bakerboy448]

Seems reasonable to me

[bakerboy448]

Related - Prowlarr/Prowlarr#611

[mynameisbogdan]

I wouldn't be the one implementing it, so I'll take Bogdan's lead.

What do you mean, a way to use tv-search: [q, season, ep, imdbid] but to use imdbid only if season/ep is empty?

[ilike2burnthing]

Yea, my suggested SupportsImdbTVSearch (might be better changed to something like SupportsImdbSeasonEpTVSearch) would indicate that IMDB ID can be used with season and episode information.

So for a tvsearch imdbid search, if:
=true - imdbid AND season AND/OR ep
=false (default) - imdbid

So =true would work fine with something UNIT3D > v6:

    name: "{{ .Keywords }}"
    seasonNumber: "{{ .Query.Season }}"
    episodeNumber: "{{ .Query.Ep }}"
    imdbId: "{{ .Query.IMDBIDShort }}"

and this Luminance example:

    searchtext: "{{ if .Query.IMDBID }}{{ .Query.IMDBID }}{{ else }}{{ end }}"
    title: "{{ if .Keywords }}{{ .Keywords }}{{ else }}{{ end }}"

whereas in this example we'd leave it to the default of =false:

    search_type: "{{ if .Query.IMDBID }}imdb{{ else }}title{{ end }}"
    search: "{{ if .Query.IMDBID }}{{ .Query.IMDBID }}{{ else }}{{ .Keywords }}{{ end }}"

[bakerboy448]

bumpy bumpy bumpy for Bogdan thoughts