chore: Update BPDM version #156
Conversation
There was a problem hiding this comment.
Looks really good!
If you want you could also look into building back the workaround for the permission configuration discussed here. It should not be necessary anymore, because the CX-Central realm for the centralidp Keycloak should now contain all the configuration (relevant change eclipse-tractusx/portal-iam#155)
cc: @nicoprow
Hi @evegufy, I think I’ll need to enable "Cl25-CX-BPDM-Orchestrator" to delete the permissions configuration since this client has the right permissions set up. Let me know if that sounds right, or if there’s anything else I should double-check with the new CX-Central realm setup in centralidp Keycloak.
|
Hi @CDiezRodriguez thank you for look into this. @nicoprow could you please answer this? |
|
@nicoprow could you please provide feedback? |
@CDiezRodriguez @evegufy It shouldn't be necessary to activate the Orchestrator Client as we just use it to define the Keycloak roles there. The Orchestrator client is not directly used for authentication but just a container for the role definitions. BPDM uses technical users (service accounts) that are separately defined in Keycloak for authentication. Again, the BPDM Orchestrator references Cl25-CX-BPDM-Orchestrator to know where to look for roles in user token. Cl25-CX-BPDM-Orchestrator does not need to be active for this. |
@nicoprow My comment here was about building back the workaround in the helm values file, which shouldn't be needed anymore. My point was not about doing any change in the CX-Central realm config because that should now be as it's supposed to be. Could you please provide the input for building back the workaround in the helm values file for the (now resolved) permission issue? |
For 24.08. the portal-iam should contain the necessary clients in order to setup the golden record process. Here is the list of clients. Also all clients should have matching permissions with BPDM default roles. Therefore, you would need to remove the permission overrides and assign the new client credentials to the BPDM apps:
|
|
Hi @nicoprow and @evegufy, thank you for the support! I followed @nicoprow’s instructions, but the BPDM deployment failed because the pool, gate, and orchestrator components could not connect to the database. Below is the relevant log: 2024-11-15 09:18:56.202 ERROR [System ] [No Request] [main ] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Exception during pool initialization.
org.postgresql.util.PSQLException: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:342) ~[postgresql-42.6.2.jar!/:42.6.2]
at org.postgresql.core.ConnectionFactory.openConnection(ConnectionFactory.java:54) ~[postgresql-42.6.2.jar!/:42.6.2]
at org.postgresql.jdbc.PgConnection.<init>(PgConnection.java:263) ~[postgresql-42.6.2.jar!/:42.6.2]
at org.postgresql.Driver.makeConnection(Driver.java:443) ~[postgresql-42.6.2.jar!/:42.6.2]
at org.postgresql.Driver.connect(Driver.java:297) ~[postgresql-42.6.2.jar!/:42.6.2]
at com.zaxxer.hikari.util.DriverDataSource.getConnection(DriverDataSource.java:138) ~[HikariCP-5.0.1.jar!/:na]
at com.zaxxer.hikari.pool.PoolBase.newConnection(PoolBase.java:359) ~[HikariCP-5.0.1.jar!/:na]
at com.zaxxer.hikari.pool.PoolBase.newPoolEntry(PoolBase.java:201) ~[HikariCP-5.0.1.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.createPoolEntry(HikariPool.java:470) ~[HikariCP-5.0.1.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.checkFailFast(HikariPool.java:561) ~[HikariCP-5.0.1.jar!/:na]
at com.zaxxer.hikari.pool.HikariPool.<init>(HikariPool.java:100) ~[HikariCP-5.0.1.jar!/:na]
at com.zaxxer.hikari.HikariDataSource.getConnection(HikariDataSource.java:112) ~[HikariCP-5.0.1.jar!/:na]
at org.flywaydb.core.internal.jdbc.JdbcUtils.openConnection(JdbcUtils.java:48) ~[flyway-core-9.22.3.jar!/:na]
at org.flywaydb.core.internal.jdbc.JdbcConnectionFactory.<init>(JdbcConnectionFactory.java:74) ~[flyway-core-9.22.3.jar!/:na]
at org.flywaydb.core.FlywayExecutor.execute(FlywayExecutor.java:142) ~[flyway-core-9.22.3.jar!/:na]
at org.flywaydb.core.Flyway.migrate(Flyway.java:140) ~[flyway-core-9.22.3.jar!/:na]
at org.springframework.boot.autoconfigure.flyway.FlywayMigrationInitializer.afterPropertiesSet(FlywayMigrationInitializer.java:66) ~[spring-boot-autoconfigure-3.2.5.jar!/:3.2.5]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1833) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1782) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:600) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:522) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:326) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:324) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:313) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:200) ~[spring-beans-6.1.6.jar!/:6.1.6]
at org.springframework.context.support.AbstractApplicationContext.getBean(AbstractApplicationContext.java:1234) ~[spring-context-6.1.6.jar!/:6.1.6]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:952) ~[spring-context-6.1.6.jar!/:6.1.6]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:624) ~[spring-context-6.1.6.jar!/:6.1.6]
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:146) ~[spring-boot-3.2.5.jar!/:3.2.5]
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754) ~[spring-boot-3.2.5.jar!/:3.2.5]
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:456) ~[spring-boot-3.2.5.jar!/:3.2.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:334) ~[spring-boot-3.2.5.jar!/:3.2.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1354) ~[spring-boot-3.2.5.jar!/:3.2.5]
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343) ~[spring-boot-3.2.5.jar!/:3.2.5]
at org.eclipse.tractusx.bpdm.pool.ApplicationKt.main(Application.kt:36) ~[!/:6.1.0]
at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(Unknown Source) ~[na:na]
at java.base/java.lang.reflect.Method.invoke(Unknown Source) ~[na:na]
at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:91) ~[app.jar:6.1.0]
at org.springframework.boot.loader.launch.Launcher.launch(Launcher.java:53) ~[app.jar:6.1.0]
at org.springframework.boot.loader.launch.JarLauncher.main(JarLauncher.java:58) ~[app.jar:6.1.0]
Caused by: java.net.ConnectException: Connection refused
at java.base/sun.nio.ch.Net.pollConnect(Native Method) ~[na:na]
at java.base/sun.nio.ch.Net.pollConnectNow(Unknown Source) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.timedFinishConnect(Unknown Source) ~[na:na]
at java.base/sun.nio.ch.NioSocketImpl.connect(Unknown Source) ~[na:na]
at java.base/java.net.SocksSocketImpl.connect(Unknown Source) ~[na:na]
at java.base/java.net.Socket.connect(Unknown Source) ~[na:na]
at org.postgresql.core.PGStream.createSocket(PGStream.java:243) ~[postgresql-42.6.2.jar!/:42.6.2]
at org.postgresql.core.PGStream.<init>(PGStream.java:98) ~[postgresql-42.6.2.jar!/:42.6.2]
at org.postgresql.core.v3.ConnectionFactoryImpl.tryConnect(ConnectionFactoryImpl.java:132) ~[postgresql-42.6.2.jar!/:42.6.2]
at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:258) ~[postgresql-42.6.2.jar!/:42.6.2]
... 41 common frames omitted
2024-11-15 09:18:56.205 WARN [System ] [No Request] [main ] ConfigServletWebServerApplicationContext : Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Unable to obtain connection from database: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State : 08001
Error Code : 0
Message : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
2024-11-15 09:18:56.209 INFO [System ] [No Request] [main ] o.apache.catalina.core.StandardService : Stopping service [Tomcat]
2024-11-15 09:18:56.221 INFO [System ] [No Request] [main ] .s.b.a.l.ConditionEvaluationReportLogger :
Error starting ApplicationContext. To display the condition evaluation report re-run your application with 'debug' enabled.
2024-11-15 09:18:56.232 ERROR [System ] [No Request] [main ] o.s.boot.SpringApplication : Application run failed
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'flywayInitializer' defined in class path resource [org/springframework/boot/autoconfigure/flyway/FlywayAutoConfiguration$FlywayConfiguration.class]: Unable to obtain connection from database: Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
SQL State : 08001
Error Code : 0
Message : Connection to localhost:5432 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.After investigating, I noticed that the database service was missing. I added it as shown in this PR, and the deployment now works. However, a new issue has arisen. The onboarding process fails at the BPN step because the pool, gate, and cleaning service components cannot communicate with the orchestrator. Here’s an excerpt from the logs: 24-11-15 09:23:00.002 ERROR [System ] [No Request] [scheduling-1 ] o.e.t.b.c.service.CleaningServiceDummy : Error while processing cleaning task
org.springframework.web.reactive.function.client.WebClientRequestException: finishConnect(..) failed: Connection refused: /[0:0:0:0:0:0:0:1]:80
at org.springframework.web.reactive.function.client.ExchangeFunctions$DefaultExchangeFunction.lambda$wrapException$9(ExchangeFunctions.java:136) ~[spring-webflux-6.1.6.jar!/:6.1.6]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ Request to POST /v6/golden-record-tasks/step-reservations [DefaultWebClient]I suspect this might be happening because the components cannot locate the orchestrator. While checking the BPDM code, I found the following section in the application.yaml, which could be relevant. Could you help confirm if I’m on the right track or suggest what else I might be missing? Thanks in advance! Edit: I tried setting the orchestrator applicationConfig.bpdm.security.client-id to "Cl7-CX-BPDM", but it’s still not working. Edit 2: I just modified the orchestrator.base-url to point to the orchestrator service, but it didn’t work. client:
orchestrator:
base-url: http://umbrella-bpdm-orchestratorI received a 403 response, although the orchestrator did receive the request. Cleaning Service Log: -11-15 11:12:30.000 INFO [System ] [No Request] [scheduling-1 ] o.e.t.b.c.service.CleaningServiceDummy : Starting polling for cleaning tasks from Orchestrator... TaskStep CleanAndSync
2024-11-15 11:12:30.009 ERROR [System ] [No Request] [scheduling-1 ] o.e.t.b.c.service.CleaningServiceDummy : Error while processing cleaning task
org.springframework.web.reactive.function.client.WebClientResponseException$Forbidden: 403 Forbidden from POST http://umbrella-bpdm-orchestrator/v6/golden-record-tasks/step-reservations
at org.springframework.web.reactive.function.client.WebClientResponseException.create(WebClientResponseException.java:309) ~[spring-webflux-6.1.6.jar!/:6.1.6]
Suppressed: reactor.core.publisher.FluxOnAssembly$OnAssemblyException:
Error has been observed at the following site(s):
*__checkpoint ⇢ 403 FORBIDDEN from POST http://umbrella-bpdm-orchestrator/v6/golden-record-tasks/step-reservations [DefaultWebClient]Orchestrator Log: 2024-11-15 11:12:30.005 INFO [00003CRHK}] [c353fe62a1] [nio-8085-exec-7] o.e.t.b.common.config.UserLoggingFilter : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/step-reservations...
2024-11-15 11:12:30.006 INFO [00003CRHK}] [f2985a5b91] [nio-8085-exec-8] o.e.t.b.common.config.UserLoggingFilter : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/step-reservations...
2024-11-15 11:12:30.008 INFO [00003CRHK}] [415d87827b] [io-8085-exec-10] o.e.t.b.common.config.UserLoggingFilter : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks/state/search...
2024-11-15 11:12:30.027 INFO [00003CRHK}] [5c079af8b7] [nio-8085-exec-9] o.e.t.b.common.config.UserLoggingFilter : User '{bpn=BPNL00000003CRHK}' requests POST /v6/golden-record-tasks...
2024-11-15 11:12:32.452 INFO [Anonymous ] [0e322abd02] [nio-8085-exec-2] o.e.t.b.common.config.UserLoggingFilter : User 'Anonymous' requests GET /actuator/health/liveness...
2024-11-15 11:12:32.452 INFO [Anonymous ] [a132a3a62c] [nio-8085-exec-1] o.e.t.b.common.config.UserLoggingFilter : User 'Anonymous' requests GET /actuator/health/readiness...
2024-11-15 11:12:32.453 INFO [Anonymous ] [a132a3a62c] [nio-8085-exec-1] o.e.t.b.common.config.UserLoggingFilter : Response with status 200
2024-11-15 11:12:32.453 INFO [Anonymous ] [0e322abd02] [nio-8085-exec-2] o.e.t.b.common.config.UserLoggingFilter : Response with status 200 |
|
@nicoprow could you please have a look at the latest comment from @CDiezRodriguez and support? |
Under normal circumstances the BPDM charts can infer the Postgres dependency name and the BPDM client service names directly from the given name and fullnameOverrides in the values. However, due to a bug this does not work when using this version of BPDM as a subchart. I was not aware that this bug is present in version 5.1.0. But instead of letting the BPDM Charts infer the names you can just straight up directly set them, as you have already done. However, I propose these changes:
Regarding the 403 I just double checked the configuration. I forgot to mention that the Orchestrator's client-id ( |
|
Thanks to @nicoprow's hints, it now works fine. I tested it successfully with the onboarding. |
Description
Update the BPDM version to 5.1.0 #140
Pre-review checks
Please ensure to do as many of the following checks as possible, before asking for committer review: