Store remote dataset credentials separately

[frcroth]

URL of deployed dev instance (used for testing):

• https://storecredentialsseparately.webknossos.xyz/

Steps to test:

• Add a dataset with credentials (S3 or HTTP Basic Auth) via exploration
• Test that the dataset can be viewed
• Test that the following other scenarios are still possible
  • Viewing a dataset without credentials
  • Viewing a dataset with legacy credentials in the json
• Test the POST route to create a credential (This route is currently not used)

TODO

• Do not put passwords in log (this is currently the case because the body of the rpc request is printed)
• cache bucket providers – they are no longer vals in layer, but now defs, so there might be new ones created all the time (I can do this –@fm3)

Issues:

• fixes Store and manage credentials separate from datasets #6612

---

(Please delete unneeded items, merge only when none are left open)

• Updated (unreleased) changelog
• Updated (unreleased) migration guide if applicable
• Needs datastore update
• Ready for review

[frcroth]

Waiting for #6679 to fix the s3 file system.

[fm3]

Cool stuff, works really well :) I added a few refactoring comments, but nothing major.

[fm3]

This could possibly use the CredentialType enum values (with toString if needed) to avoid bare unnamed strings here.

[frcroth]

[error]       case CredentialType.HTTP_Basic_Auth.toString => parseAsHttpBasicAuthCredential(r)
[error]                                           ^
[error] /home/felix/scm/webknossos/app/models/binary/credential/CredentialDAO.scala:68:41: stable identifier required, but CredentialType.S3_Access_Key.toString found.
[error]       case CredentialType.S3_Access_Key.toString   => parseAsS3AccessKeyCredential(r)

[fm3]

LGTM :) Please have a look at the merge conflicts, including evolution numbering.

Note that refresh-schema will delete the stored credentials for local datasets, so streaming test datasets become unusable then. I’m not sure how to manage this for our local development. May not be a serious problem. The old format still works, right? So maybe we could manually edit the json for a local dataset to include the credentials again, to keep it locally for testing across schema refreshs.