Unable to sync Azure groups after updating Operator to v0.0.23

[coseom47]

I1222 16:54:59.494394 1 request.go:601] Waited for 1.040893871s due to client-side throttling, not priority and fairness, request: GET:https://172.30.0.1:443/apis/network.openshift.io/v1?timeout=32s
1.6717281019386058e+09 INFO controller-runtime.metrics Metrics server is starting to listen {"addr": "127.0.0.1:8080"}
1.671728101939002e+09 INFO setup starting manager
1.6717281019395614e+09 INFO Starting server {"path": "/metrics", "kind": "metrics", "addr": "127.0.0.1:8080"}
I1222 16:55:01.939681 1 leaderelection.go:248] attempting to acquire leader lease group-sync-operator/085c249a.redhat.io...
1.671728101939729e+09 INFO Starting server {"kind": "health probe", "addr": "[::]:8081"}
I1222 16:56:11.850650 1 leaderelection.go:258] successfully acquired lease group-sync-operator/085c249a.redhat.io
1.6717281718509688e+09 INFO Starting EventSource {"controller": "groupsync", "controllerGroup": "redhatcop.redhat.io", "controllerKind": "GroupSync", "source": "kind source: *v1alpha1.GroupSync"}
1.6717281718510113e+09 INFO Starting Controller {"controller": "groupsync", "controllerGroup": "redhatcop.redhat.io", "controllerKind": "GroupSync"}
1.6717281718507807e+09 DEBUG events Normal {"object": {"kind":"Lease","namespace":"group-sync-operator","name":"085c249a.redhat.io","uid":"35b84246-9a55-43b1-afb2-82acad22f6b0","apiVersion":"coordination.k8s.io/v1","resourceVersion":"6777972"}, "reason": "LeaderElection", "message": "group-sync-operator-controller-manager-764dbc8b48-m9jdb_f295766d-cdd7-4479-85f5-f38d4a7cda25 became leader"}
1.6717281719518456e+09 INFO Starting workers {"controller": "groupsync", "controllerGroup": "redhatcop.redhat.io", "controllerKind": "GroupSync", "worker count": 1}
1.6717281720531447e+09 INFO controllers.GroupSync Beginning Sync {"groupsync": "group-sync-operator/groupsync", "Provider": "azure"}
1.6717281792370646e+09 ERROR syncer_azure Failed to get Group members for Group {"Group":
github.com/redhat-cop/group-sync-operator/pkg/syncer.(*AzureSyncer).Sync
/home/runner/work/group-sync-operator/group-sync-operator/pkg/syncer/azure.go:278
github.com/redhat-cop/group-sync-operator/controllers.(*GroupSyncReconciler).Reconcile
/home/runner/work/group-sync-operator/group-sync-operator/controllers/groupsync_controller.go:112
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234
1.6717281792372632e+09 ERROR controllers.GroupSync Failed to Complete Sync {"groupsync": "group-sync-operator/groupsync", "Provider": "azure", "error": "error status code received from the API"}

[tmanor-redhat]

Adding in the remainder of the error from the stack....

sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:121
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:320
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234
1.671728193008269e+09 DEBUG events Warning {"object": {"kind":"GroupSync","namespace":"group-sync-operator","name":"groupsync","uid":"ff4a2a77-3ef9-4e65-ad1e-ce1f68a48d5e","apiVersion":"redhatcop.redhat.io/v1alpha1","resourceVersion":"6778373"}, "reason": "ProcessingError", "message": "error status code received from the API"}
1.6717281930195937e+09 ERROR Reconciler error {"controller": "groupsync", "controllerGroup": "redhatcop.redhat.io", "controllerKind": "GroupSync", "groupSync": {"name":"groupsync","namespace":"group-sync-operator"}, "namespace": "group-sync-operator", "name": "groupsync", "reconcileID": "04b98b05-8d12-42ce-905c-4072a0f763a1", "error": "error status code received from the API"}
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:273
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2
/home/runner/go/pkg/mod/sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:234

[sabre1041]

@coseom47 @tmanor-redhat would it be possible to share the GroupSync resource as well as how groups are structured in AAD?

• Are there subgroups involved?
• Are there a combination of users and subgroup(s) in the same group?

[coseom47]

apiVersion: redhatcop.redhat.io/v1alpha1
kind: GroupSync
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: >
{"apiVersion":"redhatcop.redhat.io/v1alpha1","kind":"GroupSync","metadata":{"annotations":{},"labels":{"app.kubernetes.io/instance":"group-sync-operator"},"name":"groupsync","namespace":"group-sync-operator"},"spec":{"providers":[{"azure":{"baseGroups":["test1","test2","test3","test4","test5","test6","test7","test8"],"credentialsSecret":{"kind":"Secret","name":"azure-group-sync","namespace":"group-sync-operator"},"userNameAttributes":["userPrincipalName"]},"name":"azure"}]}}
resourceVersion: '165991906'
name: groupsync
uid: 01434171-618f-4cb1-b93e-28acefe8846c
creationTimestamp: '2022-11-07T19:52:21Z'
generation: 3
managedFields:
- apiVersion: redhatcop.redhat.io/v1alpha1
fieldsType: FieldsV1
fieldsV1:
'f:metadata':
'f:annotations':
.: {}
'f:kubectl.kubernetes.io/last-applied-configuration': {}
'f:labels':
.: {}
'f:app.kubernetes.io/instance': {}
'f:spec':
.: {}
'f:providers': {}
manager: argocd-application-controller
operation: Update
time: '2022-11-07T19:52:21Z'
- apiVersion: redhatcop.redhat.io/v1alpha1
fieldsType: FieldsV1
fieldsV1:
'f:status':
.: {}
'f:conditions':
.: {}
'k:{"type":"ReconcileError"}':
.: {}
'f:lastTransitionTime': {}
'f:message': {}
'f:observedGeneration': {}
'f:reason': {}
'f:status': {}
'f:type': {}
'k:{"type":"ReconcileSuccess"}':
.: {}
'f:lastTransitionTime': {}
'f:message': {}
'f:observedGeneration': {}
'f:reason': {}
'f:status': {}
'f:type': {}
'f:lastSyncSuccessTime': {}
manager: manager
operation: Update
subresource: status
time: '2022-12-11T19:13:28Z'
namespace: group-sync-operator
labels:
app.kubernetes.io/instance: group-sync-operator
spec:
providers:
- azure:
baseGroups:
- test1
- test2
- test3
- test4
- test5
- test6
- test7
- test8
credentialsSecret:
kind: Secret
name: azure-group-sync
namespace: group-sync-operator
userNameAttributes:
- userPrincipalName
name: azure
status:
conditions:
- lastTransitionTime: '2022-12-12T20:49:25Z'
message: ''
observedGeneration: 3
reason: LastReconcileCycleSucceded
status: 'True'
type: ReconcileSuccess
- lastTransitionTime: '2022-12-27T14:36:43Z'
message: error status code received from the API
observedGeneration: 3
reason: LastReconcileCycleFailed
status: 'True'
type: ReconcileError
lastSyncSuccessTime: '2022-12-12T20:49:25Z'

[coseom47]

@sabre1041 I am guessing what I pasted above is what you are looking for? Also, I am working on getting the information you asked for, related to the following:

Are there subgroups involved?
Are there a combination of users and subgroup(s) in the same group?

I am hoping to get answers to your questions this week, however a lot of people are out due to the holidays.

[coseom47]

@sabre1041 here are the answers to your questions.

Are there subgroups involved? No
Are there a combination of users and subgroup(s) in the same group? No

All groups are direct membership only.