iBabs requests sometimes fail

[breyten]

Describe the bug

iBabs requests sometimes fail:

loader_1    | [trace] ERROR Task ocd_backend.pipeline.setup_pipeline[c6b15bf9-c9ae-4727-8258-7860aa0f8908] raised unexpected: HTTPError('403 Client Error: Forbidden for url: https://wcf.ibabs.eu/api/Public.svc?singleWsdl')
loader_1    | Traceback (most recent call last):
loader_1    |   File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 385, in trace_task
loader_1    |     R = retval = fun(*args, **kwargs)
loader_1    |   File "/usr/local/lib/python3.8/site-packages/celery/app/trace.py", line 650, in __protected_call__
loader_1    |     return self.run(*args, **kwargs)
loader_1    |   File "/opt/ori/ocd_backend/pipeline.py", line 108, in setup_pipeline
loader_1    |     for item in pipeline_extractors[pipeline['id']](
loader_1    |   File "/opt/ori/ocd_backend/extractors/ibabs.py", line 51, in __init__
loader_1    |     self.client = Client(ibabs_wsdl,
loader_1    |   File "/usr/local/lib/python3.8/site-packages/zeep/client.py", line 68, in __init__
loader_1    |     self.wsdl = Document(wsdl, self.transport, settings=self.settings)
loader_1    |   File "/usr/local/lib/python3.8/site-packages/zeep/wsdl/wsdl.py", line 80, in __init__
loader_1    |     document = self._get_xml_document(location)
loader_1    |   File "/usr/local/lib/python3.8/site-packages/zeep/wsdl/wsdl.py", line 142, in _get_xml_document
loader_1    |     return load_external(
loader_1    |   File "/usr/local/lib/python3.8/site-packages/zeep/loader.py", line 78, in load_external
loader_1    |     content = transport.load(url)
loader_1    |   File "/usr/local/lib/python3.8/site-packages/zeep/transports.py", line 110, in load
loader_1    |     content = self._load_remote_data(url)
loader_1    |   File "/usr/local/lib/python3.8/site-packages/zeep/transports.py", line 127, in _load_remote_data
loader_1    |     response.raise_for_status()
loader_1    |   File "/usr/local/lib/python3.8/site-packages/requests/models.py", line 940, in raise_for_status
loader_1    |     raise HTTPError(http_error_msg, response=self)
loader_1    | requests.exceptions.HTTPError: 403 Client Error: Forbidden for url: https://wcf.ibabs.eu/api/Public.svc?singleWsdl

Requesting wdsl sometimes gives weird results:

 curl -I 'https://wcf.ibabs.eu/api/Public.svc?singleWsdl'
HTTP/2 400 
date: Thu, 01 Feb 2024 10:01:37 GMT
content-length: 0
cache-control: private
set-cookie: ASP.NET_SessionId=mhbchhg45255anwygjcqe2nx; path=/; HttpOnly; SameSite=Lax
x-aspnet-version: 4.0.30319
request-context: appId=cid-v1:7ca5096e-f618-4d0f-9436-b9cc2f68c1f1
access-control-expose-headers: Request-Context
x-permitted-cross-domain-policies: none
x-custom: 0200-22
x-azure-ref: 20240201T100136Z-0wptpfn6g10vhdx8kurupu945000000005qg00000000kwes
x-cache: CONFIG_NOCACHE

URL / Environment
https://

Steps to reproduce

1. Go to '...'
2. Click on '....'
3. Scroll down to '....'
4. See error

Expected behavior

Requests should not fail -- perhaps space more evenly?

Screenshots

If applicable, add screenshots to help explain your problem.

Additional context

Add any other context about the problem here. Machine, OS, browser, etc.

[breyten]

It seems that these are maybe triggered with municipalities with large report lists? Limburg has list of thousands, as well as utrecht. At a certain point some kind of memory fault is introduced, thereby invalidating the run, so we don't get these items.