OAI-PMH responses indicating errors should be processable by OAI-PMH clients #3797

tkgesis · 2017-04-26T17:17:35Z

A valid OAI request such as
https://dataverse.unc.edu/oai?verb=ListIdentifiers&set=census&metadataPrefix=dc
should return a response processable for OAI-PMH clients or HTTP 5XX.

pdurbin · 2017-04-26T17:22:44Z

@tkgesis thanks but when I run curl -i 'https://dataverse.unc.edu/oai?verb=ListIdentifiers&set=census&metadataPrefix=dc' I see HTTP/1.1 500 Internal Server Error rather than 200. That said, I do agree that it should return a processable response! Maybe we just need to change the subject/title of this issue.

donsizemore · 2017-04-26T17:32:04Z

[#|2017-04-26T13:29:06.671-0400|WARNING|
StandardWrapperValve[OAIServlet]: Servlet.service() for servlet OAIServlet threw exception
javax.servlet.ServletException: Unknown servlet exception in Get.

tkgesis · 2017-04-26T17:41:06Z

'dc' is not a supported metadataPrefix, while 'oai_dc' is.

tkgesis · 2017-04-26T17:49:08Z

According to http://www.openarchives.org/OAI/openarchivesprotocol.html#ErrorConditions
a XML response indicating the badArgument would do.

pdurbin · 2017-04-26T19:36:22Z

'dc' is not a supported metadataPrefix, while 'oai_dc' is.

Ah, so you get the same error at https://dataverse.unc.edu/oai?verb=ListIdentifiers&set=census&metadataPrefix=foobar presumably.

donsizemore · 2017-04-27T13:44:19Z

generated by 4.5 build 41-dc58ae1:

[#|2017-04-27T06:00:08.093-0400|WARNING|glassfish 4.1|javax.enterprise.web|_ThreadID=48;_ThreadName=jk-connector(2);_TimeMillis=1493287208093;_LevelValue=900;|
StandardWrapperValve[OAIServlet]: Servlet.service() for servlet OAIServlet threw exception
javax.servlet.ServletException: Unknown servlet exception in Get.
at edu.harvard.iq.dataverse.harvest.server.web.servlet.OAIServlet.doGet(OAIServlet.java:262)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
at org.apache.catalina.core.StandardWrapper.service(StandardWrapper.java:1682)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:344)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.glassfish.tyrus.servlet.TyrusServletFilter.doFilter(TyrusServletFilter.java:295)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.ocpsoft.rewrite.servlet.RewriteFilter.doFilter(RewriteFilter.java:205)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:256)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:214)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:316)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:160)
at org.apache.catalina.core.StandardPipeline.doInvoke(StandardPipeline.java:734)
at org.apache.catalina.core.StandardPipeline.invoke(StandardPipeline.java:673)
at com.sun.enterprise.web.WebPipeline.invoke(WebPipeline.java:99)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:174)
at org.apache.catalina.connector.CoyoteAdapter.doService(CoyoteAdapter.java:415)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:282)
at com.sun.enterprise.v3.services.impl.ContainerMapper$HttpHandlerCallable.call(ContainerMapper.java:459)
at com.sun.enterprise.v3.services.impl.ContainerMapper.service(ContainerMapper.java:167)
at org.glassfish.grizzly.http.server.HttpHandler.runService(HttpHandler.java:201)
at org.glassfish.grizzly.http.server.HttpHandler.doHandle(HttpHandler.java:175)
at org.glassfish.grizzly.http.server.HttpServerFilter.handleRead(HttpServerFilter.java:235)
at org.glassfish.grizzly.filterchain.ExecutorResolver$9.execute(ExecutorResolver.java:119)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeFilter(DefaultFilterChain.java:284)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.executeChainPart(DefaultFilterChain.java:201)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.execute(DefaultFilterChain.java:133)
at org.glassfish.grizzly.filterchain.DefaultFilterChain.process(DefaultFilterChain.java:112)
at org.glassfish.grizzly.ProcessorExecutor.execute(ProcessorExecutor.java:77)
at org.glassfish.grizzly.nio.transport.TCPNIOTransport.fireIOEvent(TCPNIOTransport.java:561)
at org.glassfish.grizzly.strategies.AbstractIOStrategy.fireIOEvent(AbstractIOStrategy.java:112)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.run0(WorkerThreadIOStrategy.java:117)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy.access$100(WorkerThreadIOStrategy.java:56)
at org.glassfish.grizzly.strategies.WorkerThreadIOStrategy$WorkerThreadRunnable.run(WorkerThreadIOStrategy.java:137)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.doWork(AbstractThreadPool.java:565)
at org.glassfish.grizzly.threadpool.AbstractThreadPool$Worker.run(AbstractThreadPool.java:545)
at java.lang.Thread.run(Thread.java:748)
|#]

pdurbin · 2017-04-27T13:49:51Z

@donsizemore thanks for the stacktrace and the version of Dataverse you're running. Here's where "Unknown servlet exception in Get." is coming from: https://github.com/IQSS/dataverse/blob/v4.5.1/src/main/java/edu/harvard/iq/dataverse/harvest/server/web/servlet/OAIServlet.java#L262

pdurbin · 2018-06-01T14:19:15Z

@tkgesis @donsizemore has this been fixed? I just went to https://dataverse.unc.edu/oai?verb=ListIdentifiers&set=census&metadataPrefix=dc and now it says "HTTP Status 503 - Sorry. OAI Service is disabled on this Dataverse node" like this:

Previously it was a 500 error: #3797 (comment)

pdurbin · 2022-04-12T20:17:02Z

Just noting that I get a 500 on demo running 5.10.1 with this:

curl -i 'https://demo.dataverse.org/oai?verb=ListIdentifiers&set=census&metadataPrefix=dc'

donsizemore · 2022-11-18T20:12:31Z

@pdurbin I just tried, and I get: <OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd"><responseDate>2022-11-18T20:11:43Z</responseDate><request>https://demo.dataverse.org/oai</request><error code="noRecordsMatch">Requested set 'census' does not exist</error></OAI-PMH

pdurbin · 2022-11-18T20:47:55Z

@donsizemore nice! Same! Here it is with a little pretty printing:

$ curl -s 'https://demo.dataverse.org/oai?verb=ListIdentifiers&set=census&metadataPrefix=dc' | xmllint -format -
<?xml version="1.0"?>
<OAI-PMH xmlns="http://www.openarchives.org/OAI/2.0/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openarchives.org/OAI/2.0/ http://www.openarchives.org/OAI/2.0/OAI-PMH.xsd">
  <responseDate>2022-11-18T20:46:56Z</responseDate>
  <request>https://demo.dataverse.org/oai</request>
  <error code="noRecordsMatch">Requested set 'census' does not exist</error>
</OAI-PMH>

Closing! (@tkgesis please let us know if it still isn't fixed for you.)

pdurbin · 2022-11-18T20:51:37Z

Oh, wait... a 200 response...

$ curl -i 'https://demo.dataverse.org/oai?verb=ListIdentifiers&set=census&metadataPrefix=dc'
HTTP/1.1 200 OK
Date: Fri, 18 Nov 2022 20:50:34 GMT
Server: Apache/2.4.37 (rocky) OpenSSL/1.1.1k mod_apreq2-20101207/2.8.1
Content-Type: text/xml;charset=UTF-8
Transfer-Encoding: chunked

I assume we want non-200 like @tkgesis asked for. Re-opening. Sorry for the noise!

sekmiller · 2022-11-18T20:53:37Z

Right. He's looking for an error response.

pdurbin · 2022-11-18T20:57:51Z

Ok, at least we already return error code="noRecordsMatch", (it's something 😄 ):

  <error code="noRecordsMatch">Requested set 'census' does not exist</error>

poikilotherm · 2022-11-19T00:55:00Z

Note: you might want to open an issue at gdcc/xoai and tag me.

landreev · 2022-11-21T16:25:14Z

I just mentioned it during the standup - there's a good chance this has already been addressed, during the revamping of the XOAI library by @poikilotherm. demo.dataverse.org just happens to be running a version of Dataverse from before that PR got merged (I'm going to go ahead and upgrade it to v5.12.1, and then this could be re-tested).

The OAI protocol description can be found here: http://www.openarchives.org/OAI/openarchivesprotocol.html (it's really straightforward). I'm fairly positive that the 200/OK response is correct. I.e., the protocol always returns 200 on the HTTP level - "yes, your request has been processed" - and then supplies the description of the result (such as "set does not exist" or "no records found", etc.) inside the body of the response. (but I encourage everybody to double-check things like this in the spec).

sekmiller · 2022-11-28T18:55:02Z

Latest library from XOAI (deployed to demo) fixes this issue.

poikilotherm · 2022-11-29T16:16:38Z

Now I'm glad investing all the work for updating the lib and how we use it actually pays off! 🥰

pdurbin · 2024-01-04T22:01:47Z

OAI-PMH error handling: meaningful errors in XML for verb processing #10205

pdurbin added Feature: Harvesting Type: Bug a defect labels Apr 26, 2017

tkgesis changed the title ~~OAI RESPONSE shows an error, but returns HTTP 200~~ OAI-PMH responses indicating error should be processable by OAI-PMH clients Apr 26, 2017

tkgesis changed the title ~~OAI-PMH responses indicating error should be processable by OAI-PMH clients~~ OAI-PMH responses indicating errors should be processable by OAI-PMH clients Apr 26, 2017

pdurbin added the User Role: Sysadmin Installs, upgrades, and configures the system, connects via ssh label Jul 4, 2017

pdurbin mentioned this issue Apr 13, 2022

Spike: Inventory and prioritize all existing Harvesting related issues IQSS/dataverse-pm#24

Closed

3 tasks

mreekie moved this to NIH (Stefano) in IQSS Dataverse Project Nov 2, 2022

mreekie added this to IQSS Dataverse Project Nov 2, 2022

mreekie moved this from NIH (Stefano) to 💨Ready (Stefano) in IQSS Dataverse Project Nov 2, 2022

mreekie moved this from 💨Ready (Stefano) to Sprint in IQSS Dataverse Project Nov 2, 2022

sekmiller self-assigned this Nov 17, 2022

pdurbin closed this as completed Nov 18, 2022

Repository owner moved this from 2️⃣📃✔▶ SPRINT PREP to 🏁COMPLETED in IQSS Dataverse Project Nov 18, 2022

pdurbin reopened this Nov 18, 2022

Repository owner moved this from 🏁COMPLETED to New in IQSS Dataverse Project Nov 18, 2022

sekmiller closed this as completed Nov 28, 2022

Repository owner moved this from New to 🏁In a Sprint or Completed in IQSS Dataverse Project Nov 28, 2022

sekmiller mentioned this issue Nov 28, 2022

OAI server: metadataPrefix unknown: Internal server error #3741

Closed

pdurbin added this to the 5.12.1 milestone Nov 28, 2022

sync-by-unito bot mentioned this issue Mar 3, 2023

4 | 1.4.1 | Resolve OAI-PMH harvesting issues | 5 IQSS/dataverse-pm#10

Closed

3 tasks

mreekie moved this to 🚮Clear of the Backlog in IQSS Dataverse Project Jan 28, 2023

mreekie added pm.GREI-d-1.4.1 NIH, yr1, aim4, task1: Resolve OAI-PMH harvesting issues pm.GREI-d-1.4.2 NIH, yr1, aim4, task2: Create working group on packaging standards labels Mar 20, 2023

pdurbin mentioned this issue Jan 4, 2024

OAI-PMH error handling: meaningful errors in XML for verb processing #10205

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

OAI-PMH responses indicating errors should be processable by OAI-PMH clients #3797

OAI-PMH responses indicating errors should be processable by OAI-PMH clients #3797

tkgesis commented Apr 26, 2017

pdurbin commented Apr 26, 2017

donsizemore commented Apr 26, 2017

tkgesis commented Apr 26, 2017

tkgesis commented Apr 26, 2017

pdurbin commented Apr 26, 2017

donsizemore commented Apr 27, 2017 •

edited

Loading

pdurbin commented Apr 27, 2017

pdurbin commented Jun 1, 2018

pdurbin commented Apr 12, 2022

donsizemore commented Nov 18, 2022

pdurbin commented Nov 18, 2022

pdurbin commented Nov 18, 2022

sekmiller commented Nov 18, 2022

pdurbin commented Nov 18, 2022

poikilotherm commented Nov 19, 2022

landreev commented Nov 21, 2022

sekmiller commented Nov 28, 2022

poikilotherm commented Nov 29, 2022

pdurbin commented Jan 4, 2024

OAI-PMH responses indicating errors should be processable by OAI-PMH clients #3797

OAI-PMH responses indicating errors should be processable by OAI-PMH clients #3797

Comments

tkgesis commented Apr 26, 2017

pdurbin commented Apr 26, 2017

donsizemore commented Apr 26, 2017

tkgesis commented Apr 26, 2017

tkgesis commented Apr 26, 2017

pdurbin commented Apr 26, 2017

donsizemore commented Apr 27, 2017 • edited Loading

pdurbin commented Apr 27, 2017

pdurbin commented Jun 1, 2018

pdurbin commented Apr 12, 2022

donsizemore commented Nov 18, 2022

pdurbin commented Nov 18, 2022

pdurbin commented Nov 18, 2022

sekmiller commented Nov 18, 2022

pdurbin commented Nov 18, 2022

poikilotherm commented Nov 19, 2022

landreev commented Nov 21, 2022

sekmiller commented Nov 28, 2022

poikilotherm commented Nov 29, 2022

pdurbin commented Jan 4, 2024

donsizemore commented Apr 27, 2017 •

edited

Loading