Add EGI pulsar endpoint

[sebastian-luna-valero]

Based on: usegalaxy-eu/infrastructure-playbook#872

Please configure the same credentials as the ones used for the production instance of Galaxy.

[sanjaysrikakulam]

Hey Sebastian!

A pulsar instance can only be connected to one Galaxy instance. If you would like to move your short-lived pulsar test instance to this one then I will share with you the mq server details and a new password.

[sebastian-luna-valero]

Hi Sanjay!

ok, let's have separate credentials for this test instance of Galaxy.

Thanks!

[sebastian-luna-valero]

xref: sebastian-luna-valero#1

[sebastian-luna-valero]

Here is my plan for testing the TPV API metascheduler.

• Pulsar deployment in Italy: we connected it to UseGalaxy.eu. I would like to deploy a second pulsar app on the same HTCondor virtual cluster in Italy and connect it to the ESG Galaxy with this PR. Therefore the PR #902 in the infrastructure-playbook repo could be closed.

• Pulsar deployment in Slovakia: After this PR has been merged, I would like to connect a second pulsar deployment in Slovakia to the ESG Galaxy by opening a PR like this one here.

Please let me know if you agree (when you get back from your holidays) and we proceed exchanging credentials.

[bgruening]

Please ignore this comment when I understand something wrong.
For me, the idea of the endpoint was that we can test the endpoint simply via CURL, without Galaxy. We can through all the different parameters (fake S3, fake Pulsar geo locations etc) and should get a meaningful destination back.

This gives us a lot of flexibility, without having real deployments of all of that and we can iterate much faster. So no one should ever feel blocked.

Of course in the end we want to have those real deployments, but for simple testing of the API this should not be needed. Imho. Does that make sense?

[bgruening]

There is also a merge conflict here in this PR.

[sebastian-luna-valero]

Hi @bgruening

Thanks for your feedback! Yes, it makes absolute sense.

I do like the idea of the standalone metascheduler endpoint to avoid blocking other people/activities, but I also see a benefit on integrating it now into the ESG Galaxy instance to confirm that everything works as expected and iterate fast otherwise to avoid last minute surprises down the road.

Happy to pause this activity if you think it's the best option at the moment.