Update the authentication process for remote storages #4241

shinyhappydan · 2023-09-03T08:25:22Z

Fixes #4063

Remaining tasks:

Config option for the existing approach (token)
Possible change to construction of KeyValueStore
Possible move of construction into sdk module

...orage/src/main/scala/ch/epfl/bluebrain/nexus/delta/plugins/storage/StoragePluginModule.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthenticateAs.scala

tests/docker/config/delta-postgres.conf

tests/src/test/resources/kg/storages/remote-disk-response.json

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/kernel/src/main/scala/ch/epfl/bluebrain/nexus/delta/kernel/cache/KeyValueStore.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/KeycloakAuthService.scala

shinyhappydan · 2023-09-03T19:44:22Z

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/error/TokenError.scala

+    Encoder.AsObject.instance[TokenError] {
+      case TokenHttpError(r)           =>
+        JsonObject(keywords.tpe := "TokenHttpError", "reason" := r.reason)
+      case TokenNotFoundInResponse(r)  =>
+        JsonObject(keywords.tpe -> "TokenNotFoundInResponse".asJson, "reason" := r.message)
+      case ExpiryNotFoundInResponse(r) =>
+        JsonObject(keywords.tpe -> "ExpiryNotFoundInResponse".asJson, "reason" := r.message)
+    }


I tried semi-automatic derivation, but the error messages are bad and I couldn't see the problem

What do you mean ?
Also if you only use the message/reason from DecodingFailure/ HttpClientError because it is what we are interested in here, you should not run into this

When using semi-automatic derivation for an Encoder[TokenError], there were errors which were unclear. I tried changing some obvious things, but could not fix the issue. I noticed that we had used this method elsewhere in the codebase, so I did that.

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

imsdu · 2023-09-05T09:06:12Z

...orage/src/main/scala/ch/epfl/bluebrain/nexus/delta/plugins/storage/StoragePluginModule.scala

@@ -146,8 +147,12 @@ class StoragePluginModule(priority: Int) extends ModuleDef {

  many[ResourceShift[_, _, _]].ref[Storage.Shift]

-  make[AuthTokenProvider].from { (cfg: StorageTypeConfig) =>
-    AuthTokenProvider(cfg)
+  make[KeycloakAuthService].from { (httpClient: HttpClient @Id("storage"), realms: Realms, clock: Clock[UIO]) =>


This needs to go in sdk and use the same http client as Identities

I don't see any modules inside of sdk? Also it's using the config from the storage part

delta/kernel/src/main/scala/ch/epfl/bluebrain/nexus/delta/kernel/cache/KeyValueStore.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthenticateAs.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/KeycloakAuthService.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

imsdu · 2023-09-11T09:12:30Z

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/Credentials.scala

+  case class Credentials(user: String, password: Secret[String], realm: Label) extends AuthMethod
+  object Credentials    {
+    @nowarn("cat=unused")
+    implicit private val labelConfigReader: ConfigReader[Label] = ConfigReader.fromString(str =>


We don't already have one for Label ?

If we do I don't know where it is

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/Credentials.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

...e/src/main/scala/ch/epfl/bluebrain/nexus/delta/plugins/storage/storages/StoragesConfig.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/KeycloakAuthService.scala

...orage/src/main/scala/ch/epfl/bluebrain/nexus/delta/plugins/storage/StoragePluginModule.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/OpenIdAuthService.scala

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/error/AuthTokenError.scala

docs/src/main/paradox/docs/getting-started/running-nexus/configuration/index.md

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/OpenIdAuthService.scala

imsdu · 2023-09-12T13:49:01Z

docs/src/main/paradox/docs/releases/v1.9-release-notes.md

@@ -114,6 +114,41 @@ Storages can no longer be created with credentials that would get stored:

 These should instead be defined in the Delta configuration.

+#### Remote storages


To put in the storages page and a link here to the chapter in the storage page

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala

delta/kernel/src/main/scala/ch/epfl/bluebrain/nexus/delta/kernel/cache/KeyValueStore.scala

shinyhappydan · 2023-09-13T21:29:29Z

authenticate using keycloak

b630a35

shinyhappydan commented Sep 3, 2023

View reviewed changes

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala Outdated Show resolved Hide resolved

shinyhappydan and others added 6 commits September 3, 2023 10:40

refactor AuthTokenProvider

3817ee5

use specific token error

423a107

scalafmt

a054f25

clarify test usage for AuthTokenProvider

68032a0

cache the access tokens

e8d8f63

Merge branch 'master' into remote-storage-auth-changes

b09088d

shinyhappydan commented Sep 3, 2023

View reviewed changes

delta/kernel/src/main/scala/ch/epfl/bluebrain/nexus/delta/kernel/cache/KeyValueStore.scala Outdated Show resolved Hide resolved

shinyhappydan commented Sep 3, 2023

View reviewed changes

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala Outdated Show resolved Hide resolved

shinyhappydan commented Sep 3, 2023

View reviewed changes

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/KeycloakAuthService.scala Outdated Show resolved Hide resolved

shinyhappydan commented Sep 3, 2023

View reviewed changes

delta/sdk/src/main/scala/ch/epfl/bluebrain/nexus/delta/sdk/auth/AuthTokenProvider.scala Show resolved Hide resolved

imsdu requested changes Sep 5, 2023

View reviewed changes

shinyhappydan and others added 7 commits September 8, 2023 18:52

Merge branch 'master' into remote-storage-auth-changes

bb7a7ef

change migration code which was broken when merging master branch

5d4dd36

use label for realm

ce103a0

rename config class to 'Credentials'

9ab1af9

use the realm http client

be6404a

add anonymous as an authentication method

a1b3cb7

add legacy auth token method to auth options

bc07c05

shinyhappydan marked this pull request as ready for review September 10, 2023 11:49

shinyhappydan and others added 3 commits September 10, 2023 14:27

add docs

31a060c

Merge branch 'master' into remote-storage-auth-changes

6202ad9

rename variable

bbf1146

imsdu requested changes Sep 11, 2023

View reviewed changes

shinyhappydan added 3 commits September 11, 2023 12:01

credentials are not optional

9ec0090

rename AuthMethod to Credentials

dc353b7

TokenError -> AuthTokenError

527ce05

shinyhappydan added 3 commits September 11, 2023 12:22

KeycloakAuthService -> OpenIdAuthService

c58dcb0

add logging to caching open id auth token provider

5ea60c1

fix comment

1653973

imsdu requested changes Sep 11, 2023

View reviewed changes

pr feedback

402dd64

shinyhappydan changed the title ~~authenticate using keycloak~~ Update the authentication process for remote storages Sep 11, 2023

shinyhappydan and others added 4 commits September 12, 2023 10:08

add release notes

d35a18a

fix storage config

47b0c5e

check realm is not deprecated

b23ee33

Merge branch 'master' into remote-storage-auth-changes

4a78606

imsdu requested changes Sep 12, 2023

View reviewed changes

shinyhappydan and others added 5 commits September 13, 2023 10:01

use ParsedToken

f4cb360

Merge branch 'master' into remote-storage-auth-changes

0cd7d79

remove KeyValueStore.create

333fcfe

change AuthTokenProvider interface

3042bd4

move docs

0ad83cd

docs change

c6aa5ec

imsdu previously approved these changes Sep 14, 2023

View reviewed changes

add scaladoc

dee991e

shinyhappydan dismissed imsdu’s stale review via dee991e September 14, 2023 09:36

scalafmt

0e51145

imsdu approved these changes Sep 14, 2023

View reviewed changes

shinyhappydan merged commit 288460e into BlueBrain:master Sep 14, 2023
6 checks passed

shinyhappydan deleted the remote-storage-auth-changes branch September 14, 2023 11:42

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update the authentication process for remote storages #4241

Update the authentication process for remote storages #4241

shinyhappydan commented Sep 3, 2023 •

edited

Loading

shinyhappydan Sep 3, 2023

imsdu Sep 11, 2023

shinyhappydan Sep 11, 2023

imsdu Sep 5, 2023

shinyhappydan Sep 8, 2023

imsdu Sep 11, 2023

shinyhappydan Sep 11, 2023

imsdu Sep 12, 2023

shinyhappydan commented Sep 13, 2023

		@@ -114,6 +114,41 @@ Storages can no longer be created with credentials that would get stored:

		These should instead be defined in the Delta configuration.

		#### Remote storages

Update the authentication process for remote storages #4241

Update the authentication process for remote storages #4241

Conversation

shinyhappydan commented Sep 3, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shinyhappydan commented Sep 13, 2023

shinyhappydan commented Sep 3, 2023 •

edited

Loading