Link two registry schema #293

aniket-tarento · 2022-11-25T14:10:04Z

aniket-tarento
Nov 25, 2022

ref: #163 #166 We have two schema 'Organization' and 'User', how to link these two schema so that organization can be created by registry admin and the users can be created by org admin and later verify that the particular user belong to that organization. Also, how to enforce the entity creation in third schema, so that only the users created in previous step with certain role can create an entity in third schema. Please, guide us on the role mappings between schemas. We tried with below schemas, but getting following error while inviting organization:

"errmsg":"Exception executing consequence for rule \"Create entity owner for newly added owner fields\" in dev.sunbirdrc.workflow: [Error: keycloakAdminUtil.createUser(stateDefinition.getEntityName(), stateDefinition.getUpdated().get(\"userId\").textValue(), stateDefinition.getUpdated().get(\"email\").textValue(), stateDefinition.getUpdated().get(\"mobile\").textValue()): javax.ws.rs.NotAuthorizedException: HTTP 401 Unauthorized]\n[Near : {... String owner = keycloakAdminUt ....}]\n ^\n[Line: 1, Column: 1]"}

Schemas:

organization.json:
{
	"$schema": "http://json-schema.org/draft-07/schema",
	"type": "object",
	"properties": {
		"Organization": {
			"$ref": "#/definitions/Organization"
		}
	},
	"required": ["Organization"],
	"title": "Organization",
	"definitions": {
		"Organization": {
			"$id": "#/properties/Organization",
			"type": "object",
			"title": "Organization schema",
			"required": ["name", "type", "location"],
			"uniqueIndexFields": ["name"],
			"properties": {
				"name": { "type": "string" },
				"type": { "type": "string" },
				"location": { "type": "string" },
				"phone": { "type": "string" },
				"email": { "type": "string" }
			  }
		}
	},
	"_osConfig": {
		"ownershipAttributes": [
			{
				"email": "/email",
				"mobile": "/phone",
				"userId": "/phone"
			}
		],
		"inviteRoles": ["anonymous"],
		"enableLogin": true,
		"systemFields": ["osCreatedAt", "osUpdatedAt"]
	}
}

user.json:
{
	"$schema": "http://json-schema.org/draft-07/schema",
	"type": "object",
	"properties": { "User": { "$ref": "#/definitions/User" } },
	"required": ["User"],
	"title": "User",
	"definitions": {
		"User": {
			"$id": "#/properties/User",
			"type": "object",
			"title": "User schema",
			"required": ["name", "phone", "organization", "roles"],
			"uniqueIndexFields": ["phone"],
			"properties": {
				"name": { "type": "string" },
				"phone": { "type": "string" },
				"email": { "type": "string" },
				"organization": {
				  "$ref": "organization.json/#/definitions/Organization"
				},
				"roles": {
				  "type": "string",
				  "enum": ["admin", "cert-issuer"]
				}
			  }
		}
	},
	"_osConfig": {
		"ownershipAttributes": [
			{
				"email": "/email",
				"mobile": "/phone",
				"userId": "/phone"
			}
		],
		"inviteRoles": ["organization"],
		"enableLogin": true,
		"systemFields": ["osCreatedAt", "osUpdatedAt", "osCreatedBy", "osUpdatedBy"]
	}
}

cc: @dileepbapat @tejash-jl

tejash-jl · 2022-11-28T09:49:46Z

tejash-jl
Nov 28, 2022
Collaborator

@aniket-tarento I tested with the above schemas, and I could invite an organization successfully. Could you share the curl, compose file and logs.

0 replies

aniket-tarento · 2022-11-28T11:22:01Z

aniket-tarento
Nov 28, 2022
Author

@tejash-jl Is there a dependency on 'teacher' and 'student' schema in the registry instance? Because when I deleted these two schemas from schema folder, I wasn't able to create 'organization' due to authorization error. But after having it back, the org entity is getting created successfully.
Also, how can we assign a custom role (eg. - issuer, org-admin, etc.) to an entity while creation?

0 replies

aniket-tarento · 2022-11-29T05:49:03Z

aniket-tarento
Nov 29, 2022
Author

@tejash-jl Started facing unauthorized error again while inviting organization. Please, find the following curl, compose file and logs:

curl:

curl --location --request POST 'http://localhost:8081/api/v1/Organization/invite' \
--header 'content-type: application/json' \
--header 'Cookie: JSESSIONID=BCE6B6AECCCC578EC30CB27E3CF8FF0E; OAuth_Token_Request_State=a456282b-48f9-4644-8dfc-7d327dfb53b5' \
--data-raw '{
    "name": "Test Org 3",
    "type": "education",
    "location": "bangalore",
    "phone": "5566778899",
    "email": "[email protected]"
}'

docker-compose.yaml

version: '2.4'

services:
  es:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.10.1
    environment:
      - discovery.type=single-node
      - 'ES_JAVA_OPTS=-Xms512m -Xmx512m'
    healthcheck:
      test: ['CMD', 'curl', '-f', 'localhost:9200/_cluster/health']
      interval: 30s
      timeout: 10s
      retries: 4
    ports:
      - '9200:9200'
      - '9300:9300'
  db:
    image: postgres
    ports:
      - '5432:5432'
    environment:
      - POSTGRES_DB=registry
      - POSTGRES_USER=postgres
      - POSTGRES_PASSWORD=postgres
  rg:
    image: dockerhub/sunbird-rc-core
    volumes:
      - ./config/schemas:/home/sunbirdrc/config/public/_schemas
    environment:
      - connectionInfo_uri=jdbc:postgresql://db:5432/registry
      - connectionInfo_username=postgres
      - connectionInfo_password=postgres
      - elastic_search_connection_url=es:9200
      - search_provider=dev.sunbirdrc.registry.service.ElasticSearchService
      - sunbird_sso_realm=sunbird-rc
      - sunbird_sso_url=http://kc:8080/auth
      - sunbird_sso_admin_client_id=admin-api
      - sunbird_sso_client_id=registry-frontend
      - sunbird_sso_admin_client_secret=0e85ccd2-aa4b-4b3e-b439-1d1b6acffd83
      - sunbird_keycloak_user_set_password=true
      - sunbird_keycloak_user_password=test
      - claims_url=http://cs:8082
      - sign_url=http://certificate-signer:8079/sign
      - sign_health_check_url=http://certificate-signer:8079/health
      - signature_enabled=true
      - pdf_url=http://certificate-api:8078/api/v1/certificatePDF
      - template_base_url=http://rg:8081/api/v1/templates/
    ports:
      - '8081:8081'
    depends_on:
      es:
        condition: service_healthy
      db:
        condition: service_started
      kc:
        condition: service_healthy
  kc:
    image: dockerhub/ndear-keycloak
    volumes:
      - ./config/keycloak:/opt/jboss/keycloak/imports
    environment:
      - DB_VENDOR=postgres
      - DB_ADDR=db
      - DB_PORT=5432
      - DB_DATABASE=registry
      - DB_USER=postgres
      - DB_PASSWORD=postgres
      - KEYCLOAK_USER=admin
      - KEYCLOAK_PASSWORD=admin
      - KEYCLOAK_IMPORT=/opt/jboss/keycloak/imports/realm-export.json
      - KEYCLOAK_FRONTEND_URL=http://localhost:8080/auth
    healthcheck:
      test: ['CMD-SHELL', 'curl -f http://localhost:9990/ || exit 1']
      interval: 30s
      timeout: 10s
      retries: 5
    ports:
      - '8080:8080'
      - '9990:9990'
    depends_on:
      db:
        condition: service_started
  cs:
    image: dockerhub/open-saber-claim-ms
    environment:
      - connectionInfo_uri=jdbc:postgresql://db:5432/registry
      - connectionInfo_username=postgres
      - connectionInfo_password=postgres
      - sunbirdrc_url=http://rg:8081
    ports:
      - '8082:8082'
    depends_on:
      db:
        condition: service_started
      rg:
        condition: service_started
  certificate-signer:
    image: dockerhub/sunbird-rc-certificate-signer
    environment:
      - PORT=8079
    ports:
      - "8079:8079"
    volumes:
      - ./imports:/etc/signer
  certificate-api:
    image: dockerhub/sunbird-rc-certificate-api
    environment:
      - PORT=8078
      - QR_TYPE=URL-W3C-VC
      - CERTIFICATE_DOMAIN_URL=http://localhost:4000
    ports:
      - "8078:8078"

registry-logs.txt

1 reply

tejash-jl Nov 29, 2022
Collaborator

@aniket-tarento can you check if you're using the latest version of registry-cli

sachinsahu731 · 2022-11-29T07:47:20Z

sachinsahu731
Nov 29, 2022

@tejash-jl I am also having the same issue with creating a teacher . Kindly look at priority it will block us.

12 replies

aniket-tarento Dec 2, 2022
Author

@tejash-jl We've upgraded to registry cli version - 1.6.2 and tried instantiating the registry. After that we put our schema files to '/config/schema' folder and did a registry restart but this command stuck in terminal and not ending. However, we are able to see 14/15 services are running except 'kafka'. Below are the terminal output :

anike@DESKTOP-HOGOC5R MINGW64 ~/OneDrive/Desktop/DIKSHA/Registries/ndear-stars
$ registry init
- Loading...

Registry Setup

- Checking environment...
✔︎ All necessary tools installed!

√ Enter the name of the registry · ndear-stars
? Enter the name of the Keycloak realm to create » sunbird-rc
√ Enter the name of the Keycloak realm to create · sunbird-rc
? Enter the ID to assign to the admin client in Keycloak » admin-api
√ Enter the ID to assign to the admin client in Keycloak · admin-api
? Enter the ID to assign to the frontend client in Keycloak » registry-frontend
√ Enter the ID to assign to the frontend client in Keycloak · registry-frontend
? Enter a username for the admin account in Keycloak » admin
√ Enter a username for the admin account in Keycloak · admin
? Enter a password for the admin account in Keycloak » admin
√ Enter a password for the admin account in Keycloak · admin
? Enter the path to a directory containing entity schemas » use-example-config
√ Enter the path to a directory containing entity schemas · use-example-config
√ Enter the path to a file containing consent configuration · use-example-config


- Copying over necessary files...
✔︎ Successfully copied over necessary files
- Starting all services...
✔︎ Successfully started all services!
- Configuring keycloak...
✔︎ Successfully configured keycloak!
- Setting up consent provider...
✔︎ Successfully setup keycloak as a consent provider!
- Restarting all services...
✔︎ Created new registry instance successfully!

Sunbird-RC is configured with test/default keys for signing. It is required to be updated `imports/config.json` before going live/production

anike@DESKTOP-HOGOC5R MINGW64 ~/OneDrive/Desktop/DIKSHA/Registries/ndear-stars
$ registry restart
- Loading...

ndear-stars

- Checking environment...
✔︎ All necessary tools installed!
- Restarting all registry containers...

Also, we checked the secret in keycloak as well as docker-compose file and found out both are same (attaching ss). We're still not able to invite either Teacher or Organization. Attaching the logs.

registry-logs.txt

aniket-tarento Dec 2, 2022
Author

@tejash-jl Here's the reponse from health api :

{
    "id": "sunbird-rc.registry.health",
    "ver": "1.0",
    "ets": 1669977568003,
    "params": {
        "resmsgid": "",
        "msgid": "bcb1564e-6d65-46dc-babc-38752520fcf7",
        "err": "",
        "status": "SUCCESSFUL",
        "errmsg": ""
    },
    "responseCode": "OK",
    "result": {
        "name": "sunbirdrc-registry-api",
        "healthy": true,
        "checks": [
            {
                "name": "sunbird.notification.service",
                "healthy": true,
                "err": "NOTIFICATION_ENABLED",
                "errmsg": "false"
            },
            {
                "name": "sunbird.encryption.service",
                "healthy": true,
                "err": "ENCRYPTION_ENABLED",
                "errmsg": "false"
            },
            {
                "name": "sunbird.certificate-api.service",
                "healthy": true,
                "err": "",
                "errmsg": ""
            },
            {
                "name": "sunbird.signature.service",
                "healthy": true,
                "err": "",
                "errmsg": ""
            },
            {
                "name": "sunbird.elastic.service",
                "healthy": true,
                "err": "",
                "errmsg": ""
            },
            {
                "name": "sunbird.file-storage.service",
                "healthy": true,
                "err": "",
                "errmsg": ""
            },
            {
                "name": "sunbird.keycloak.service",
                "healthy": true,
                "err": "",
                "errmsg": ""
            }
        ]
    }
}

tejash-jl Dec 5, 2022
Collaborator

@aniket-tarento can you ping me on discord, we can connect over a call and resolve this today
https://discord.gg/Q5mvw2mGC8

aniket-tarento Dec 13, 2022
Author

@tejash-jl I have upgraded to registry cli version 1.6.4, but still facing the same unauthorized issue while creating entity. The registry init command is not working as expected and getting ended with following error :

anike@DESKTOP-HOGOC5R MINGW64 ~/OneDrive/Desktop/DIKSHA/Registries/ndear-stars
$ registry init
- Loading...

Registry Setup

- Checking environment...
✔︎ All necessary tools installed!

√ Enter the name of the registry · ndear-stars
? Enter the name of the Keycloak realm to create » sunbird-rc
√ Enter the name of the Keycloak realm to create · sunbird-rc
? Enter the ID to assign to the admin client in Keycloak » admin-api
√ Enter the ID to assign to the admin client in Keycloak · admin-api
? Enter the ID to assign to the frontend client in Keycloak » registry-frontend
√ Enter the ID to assign to the frontend client in Keycloak · registry-frontend
? Enter a username for the admin account in Keycloak » admin
√ Enter a username for the admin account in Keycloak · admin
? Enter a password for the admin account in Keycloak » admin
√ Enter a password for the admin account in Keycloak · admin
? Enter the path to a directory containing entity schemas » use-example-config
√ Enter the path to a directory containing entity schemas · use-example-config
√ Enter the path to a file containing consent configuration · use-example-config


- Copying over necessary files...
✔︎ Successfully copied over necessary files
- Starting all services...
ⅹ An unexpected error occurred while starting the registry: Command failed with exit code 1: docker compose up -d Error: Command failed with exit code 1: docker compose up -d
    at makeError (C:\Users\anike\AppData\Roaming\npm\node_modules\registry-cli\node_modules\execa\lib\error.js:56:11)
    at handlePromise (C:\Users\anike\AppData\Roaming\npm\node_modules\registry-cli\node_modules\execa\index.js:114:26)
    at processTicksAndRejections (node:internal/process/task_queues:96:5)


anike@DESKTOP-HOGOC5R MINGW64 ~/OneDrive/Desktop/DIKSHA/Registries/ndear-stars
$ registry version
- Loading...

cli  1.6.4
node 16.14.0
os   win32 x64

Due to this the keycloak and other configurations are not getting done successfully is what I believe.
Please look into this. here's the logs from registry :
registry-logs.txt

tejash-jl Dec 16, 2022
Collaborator

@aniket-tarento
i think some services are not getting started,
can you delete all files and restart

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Sunbird Registries & Credentials

Link two registry schema #293

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 4 comments 13 replies

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

Select a reply

Sunbird Registries & Credentials

Link two registry schema #293

aniket-tarento Nov 25, 2022

Replies: 4 comments · 13 replies

tejash-jl Nov 28, 2022 Collaborator

aniket-tarento Nov 28, 2022 Author

aniket-tarento Nov 29, 2022 Author

tejash-jl Nov 29, 2022 Collaborator

sachinsahu731 Nov 29, 2022

aniket-tarento Dec 2, 2022 Author

aniket-tarento Dec 2, 2022 Author

tejash-jl Dec 5, 2022 Collaborator

aniket-tarento Dec 13, 2022 Author

tejash-jl Dec 16, 2022 Collaborator

aniket-tarento
Nov 25, 2022

Replies: 4 comments 13 replies

tejash-jl
Nov 28, 2022
Collaborator

aniket-tarento
Nov 28, 2022
Author

aniket-tarento
Nov 29, 2022
Author

tejash-jl Nov 29, 2022
Collaborator

sachinsahu731
Nov 29, 2022

aniket-tarento Dec 2, 2022
Author

aniket-tarento Dec 2, 2022
Author

tejash-jl Dec 5, 2022
Collaborator

aniket-tarento Dec 13, 2022
Author

tejash-jl Dec 16, 2022
Collaborator