Overcoming the limitation of http get-based DALResults from_result_url method

[almicol]

At ESO we have a datalink service that associates calibration files to science raw files.
For efficiency, it is better (much better) to pass to such service a big number of science raw files IDs (up to several thousands), instead of splitting the request into many chunks. Passing so many IDs is not possible using an http get, http post must be used instead.

The from_result_url() method is therefore of no use here, as it uses a "session.get", hence limiting to few hundreds the max number of IDs passed at once.

The gain in efficiency depends on the instrument and mode, I have seen cases whereby a POST returns in few minutes while the equivalent set of chunked GET requests takes several hours. On top, after those several hours, the software has still to merge the results of each chunked GET.

I'm aware of the existing DatalinkResultsMixin() method; that would (partially) remove the management of merging results, but it cannot overcome the mentioned efficiency issue, as, using GET, it cannot make use of the optimised internal logic of the calibration service when used on a big number of IDs.

Could you implement a pyvo method that would implement the equivalent of from_result_url using POST instead of GET?

Thanks!

[almicol]

And who is ArthurMLi ? Why should I follow that link? It looks like phishing to me.

[almicol]

Ah, the ArthurMLi's answer with a phishing link has just disappeared!

[bsipocz]

Yes, unfortunately GitHub has experienced a large spamming/scamming campaign for a few weeks now, I suppose this was part of that (comments tend to come after new issues, but now there some automated cleanups in place).