Harvester results - pagination doesn't work

[josegar74]

Describe the bug

Pagination button in the harvester results doesn't work. Seem a issue of version 4.0.x, previous versions seem fine.

To Reproduce
Steps to reproduce the behavior:

1. Create a new CSW harvester
2. Execute it
3. Click the metadata results tab in the harvester page
4. Next page doesn't seem to do anything. Last page button issues a request that fails:

{
"servlet":"spring",
"message":"Error is: Bad Request.\nRequest:\n{"from":-20,"size":10,"sort":[{"resourceTitleObject.default.keyword":"asc"},"_score"],"query":{"function_score":{"boost":"5","functions":[{"filter":{"exists":{"field":"parentUuid"}},"weight":0.3},{"filter":{"match":{"cl_status.key":"obsolete"}},"weight":0.3},{"gauss":{"dateStamp":{"scale":"365d","offset":"90d","decay":0.5}}}],"score_mode":"multiply","query":{"bool":{"must":[{"terms":{"isTemplate":["y","n","s","t"]}},{"terms":{"harvesterUuid":["4226a948-591f-46ba-b797-7251f943b1b7"]}}],"filter":{"query_string":{"query":"* AND (draft:n OR draft:e)"}}}}}},"aggregations":{"availableInServices":{"filters":{"filters":{"availableInViewService":{"query_string":{"query":"+linkProtocol:/OGC:WMS.*/"}},"availableInDownloadService":{"query_string":{"query":"+linkProtocol:/OGC:WFS.*/"}}}}},"cl_hierarchyLevel.key":{"terms":{"field":"cl_hierarchyLevel.key"},"meta":{"vega":"arc"}},"tag.default":{"terms":{"field":"tag.default","size":10},"meta":{"vega":"arc"}}},"_source":{"includes":["id","uuid","overview.*","resource*","isTemplate","valid","op*"]},"track_total_hits":true}\n.\nError:\n{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"numHits must be > 0; please use TotalHitCountCollector if you just need the total hit count"}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"gn-records","node":"MyVyAUFfQdCxCWr9OVZWWg","reason":{"type":"illegal_argument_exception","reason":"numHits must be > 0; please use TotalHitCountCollector if you just need the total hit count"}}],"caused_by":{"type":"illegal_argument_exception","reason":"numHits must be > 0; please use TotalHitCountCollector if you just need the total hit count","caused_by":{"type":"illegal_argument_exception","reason":"numHits must be > 0; please use TotalHitCountCollector if you just need the total hit count"}}},"status":400}.",
"url":"/geonetwork/srv/api/search/records/_search",
"status":"400"
}

Expected behavior

Pagination buttons allow to display the result pages with the harvested metadata.

[tomkralidis]

We are seeing this issue in public deployments:

• https://metadata.bgs.ac.uk/geonetwork/srv/eng/csw?request=GetRecords&service=CSW&version=2.0.2&maxrecords=10
• https://metadata.bgs.ac.uk/geonetwork/srv/eng/csw?request=GetRecords&service=CSW&version=2.0.2&maxrecords=50
• https://metadata.bgs.ac.uk/geonetwork/srv/eng/csw?request=GetRecords&service=CSW&version=2.0.2&maxrecords=100

i.e. nextRecord is always zero, even though there are more records in the result set.

cc @kalxas @KoalaGeo

[kalxas]

This issue was picked up by OWSLib CI which is failing recently with the above urls

[kalxas]

I was able to reproduce this issue on a local deployment with GeoNetwork 4.4.3

[KoalaGeo]

We are seeing this issue in public deployments:

• https://metadata.bgs.ac.uk/geonetwork/srv/eng/csw?request=GetRecords&service=CSW&version=2.0.2&maxrecords=10
• https://metadata.bgs.ac.uk/geonetwork/srv/eng/csw?request=GetRecords&service=CSW&version=2.0.2&maxrecords=50
• https://metadata.bgs.ac.uk/geonetwork/srv/eng/csw?request=GetRecords&service=CSW&version=2.0.2&maxrecords=100

i.e. nextRecord is always zero, even though there are more records in the result set.

cc @kalxas @KoalaGeo

Cheers @tomkralidis for making the connection to the issue raised on gitter, and this existing issue.

Not sure what to do or where the fix is? Could maybe get our java team on it if you can point us in the right direction?

We've a number of stakeholders using this so looking for a quick fix...

[fxprunayre]

If they are running 4.4.3, it has been fixed in 4.4.4 https://docs.geonetwork-opensource.org/4.4/overview/change-log/version-4.4.4/, #7937. It was related to Elasticsearch 8 migration.

[KoalaGeo]

Yes 4.4.3, I'll upgrade to 4.4.4 today

[kalxas]

Thank you @fxprunayre
I will test again with 4.4.4

[KoalaGeo]

Working for me in staging: https://hosted-metadata-uat.bgs.ac.uk/geonetwork/srv/eng/csw?request=GetRecords&service=CSW&version=2.0.2&maxrecords=10

GN: 4.4.4
PostgreSQL: 16.2
ES: 8.11.3
OGCAPI-Records: 4.4.3

Going to roll out to production

[kalxas]

It is now working for me as well.
Thanks for the release!

[juanluisrp]

The pagination in the harvester console works at least in v4.2.9 and v4.4.4. Closing the issue.