Add a pulsar endpoint for EGI deployed instance

[enolfc]

This will be a Pulsar instance supported by the BYOC task of EuroScienceGateway deployed automatically by IM.

[bgruening]

Do you already have your password :)

[kysrpex]

Do you already have your password :)

How should I share the password with you, @enolfc?

[enolfc]

Do you already have your password :)

How should I share the password with you, @enolfc?

How do you normally do? If I send a signed email, would you be able to send it back encrypted?

[kysrpex]

Do you already have your password :)

How should I share the password with you, @enolfc?

How do you normally do? If I send a signed email, would you be able to send it back encrypted?

@bgruening I do not know what is the usual procedure, but I don't have the toolchain needed for signed emails installed at the moment, so maybe let's just use GPG? I trust your GitHub account and I assume you can access a gpg binary :)

Generate a keypair (that we will use only for this and only once).

export GNUPGHOME="$(mktemp -d)"
cat > $GNUPGHOME/oneoff <<EOF
     %echo One-off OpenPGP key
     Key-Type: RSA
     Key-Length: 4096
     Subkey-Type: RSA
     Subkey-Length: 4096
     Name-Real: One-off key
     Name-Comment: One-off key
     Name-Email: [email protected]
     Expire-Date: 2023-09-30
     Passphrase: password
     %commit
EOF
gpg --batch --generate-key $GNUPGHOME/oneoff

Get the public key using the command below and paste the output here on GitHub.

gpg --armor --export [email protected]

Do not close the terminal (because the key is in a temporary directory). With this output I can encrypt a message and paste it here on GitHub. Then you can copy it to a file (e.g. "my_password.txt.encrypted").

If you run

gpg --output my_password.txt --decrypt my_password.txt.encrypted

you will be prompted for a password. Enter "password" and the decrypted message will be available on my_password.txt :)

---

Of course all this only works if you can use gpg haha

[bgruening]

I also don't know what the current procedure is. Mira is currently on vacation, but any encrypted channel works imho. And Matrix is e2e. You both have a matrix account correct?

[kysrpex]

I also don't know what the current procedure is. Mira is currently on vacation, but any encrypted channel works imho. And Matrix is e2e. You both have a matrix account correct?

Yes I do. @enolfc I just sent you a message.

[sanjaysrikakulam]

Mira used to share the password via Matrix (encrypted chat).

[kysrpex]

Can this be merged? (in other words: is the Pulsar endpoint ready?)

[sebastian-luna-valero]

At the moment we have a virtual HTCondor cluster with 6 compute nodes, 8 vCPUs and 16 GB each.

In the max_accepted_cores and max_accepted_mem do you include the total per node or per cluster?

[kysrpex]

max_accepted_cores: Number of cores of the machine with the most cores in your cluster (8)
max_accepted_mem: Number of cores of the machine with the most memory in your cluster (15 looks good to me because you are going to need some memory for the OS, etc)

[sebastian-luna-valero]

thanks, then it looks correct

[sebastian-luna-valero]

As discussed in the context of the EuroScienceGateway project, the EGI pulsar endpoint is volatile as we are deploying/destroying trying to troubleshoot issues. If that's not an issue, I am happy with this to be merged and run tests.

[kysrpex]

Shouldn't then this be "merged" to @sanjaysrikakulam's test instance instead?

[sanjaysrikakulam]

I am almost there with a test instance for ESG (https://usegalaxy.esgwps.uno). I will fix the last few things and run a couple of tests, and then we can start integrating ESG test pulsar instances to the ESG test Galaxy instance. I was on vacation last week so things got a bit delayed.

[bgruening]

As long as it is an opt-in feature that does not impact any user, I'm perfectly fine with getting this deployed.