Sync issue happened for wg-manifests and wg-deployment

[PatrickXYS]

After this PR merged,

we should have wg-manifests and wg-deployment Github Team established,

given this query, https://github.com/orgs/kubeflow/teams?query=wg-

It's obviously sync script does not work, I'm guessing the issue happened before, happened again.

/cc @chensun @jlewi @Bobgy

[Bobgy]

/assign
I'll investigate now

[Bobgy]

Found error

ValueError: chensun is not in the allowed set of admins. Allowed admins is Bobgy, google-admin, googlebot, james-jwu, jlewi, k8s-ci-robot, richardsliu, rmgogogo, theadactyl

I'll prioritize #346 after immediate work items for Kubeflow 1.2

[Bobgy]

Hmm, the change was correct in #376, but the running job hasn't been updated. I'll investigate further.

[Bobgy]

/cc @jlewi

[PatrickXYS]

Maybe adding @chensun to the kubeflow org, and then check out the cron job to see how it goes.

[PatrickXYS]

I think we can improve the process, find a way to expose logs / errors / issues to non-google folks, thus, it will save time for google-admin to check those logs as well

[Bobgy]

OK, I found the problem, the validate.py change in #376 needs to be manually updated by

internal-acls/github-orgs/manifests/Makefile

Line 10 in e4ad312

create-config-map:

I've updated that, the next sync should work

[Bobgy]

I think we can improve the process, find a way to expose logs / errors / issues to non-google folks, thus, it will save time for google-admin to check those logs as well

I think the blocker is that fixing anything there is high privilege, so you'll need to wait for a google-admin to do that, just exposing the logs might not be enough, but exposing logs might be helpful to let community members find a fix and contribute a PR...

Any ideas for a good way to expose those logs?

[Bobgy]

There's one more problem

{"component":"peribolos","file":"prow/cmd/peribolos/main.go:202","func":"main.main","level":"fatal","msg":"Configuration failed: failed to configure kubeflow members: kubeflow must specify {[jlewi james-jwu google-admin googlebot] true} as admins, missing [jlewi]","severity":"fatal","time":"2020-11-05T02:50:10Z"}

[Bobgy]

/reopen
Still not working

{"component":"peribolos","file":"prow/cmd/peribolos/main.go:202","func":"main.main","level":"fatal","msg":"Configuration failed: failed to configure kubeflow members: authenticated user jlewi is not an admin of kubeflow","severity":"fatal","time":"2020-11-05T03:10:19Z"}

[k8s-ci-robot]

@Bobgy: Reopened this issue.

In response to this:

/reopen
Still not working

{"component":"peribolos","file":"prow/cmd/peribolos/main.go:202","func":"main.main","level":"fatal","msg":"Configuration failed: failed to configure kubeflow members: authenticated user jlewi is not an admin of kubeflow","severity":"fatal","time":"2020-11-05T03:10:19Z"}

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[Bobgy]

The sync was successful

[PatrickXYS]

I think we can improve the process, find a way to expose logs / errors / issues to non-google folks, thus, it will save time for google-admin to check those logs as well

I think the blocker is that fixing anything there is high privilege, so you'll need to wait for a google-admin to do that, just exposing the logs might not be enough, but exposing logs might be helpful to let community members find a fix and contribute a PR...

Any ideas for a good way to expose those logs?

I can come up with some ideas including giving viewer access, but looks like the GKE cluster cronjob is running is not used internally, I don't have a good idea