Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Error when upgrading to v0.34 and up - "panic: recursive set of Transformers detected" #1769

Closed
HenriWilliams opened this issue Jun 1, 2023 · 10 comments
Closed

Error when upgrading to v0.34 and up - "panic: recursive set of Transformers detected" #1769

HenriWilliams opened this issue Jun 1, 2023 · 10 comments
Labels
bug Something isn't working

Comments

@HenriWilliams
Copy link

HenriWilliams commented Jun 1, 2023 •
edited
Loading

What happened?

When upgrading Crossplane from v0.33.0 to v0.34.0 (using the command: kubectl patch provider.pkg crossplane-provider-aws -p '{"spec":{"package": "crossplanecontrib/provider-aws:v0.34.0"}}' --type=merge), the crossplane-provider-aws pod goes into a CrashLoopBackoff with the following error:

panic: recursive set of Transformers detected:
        Transformer(cmpopts.SortSlices, cmpopts.sliceSorter.sort): interface {} => interface {}
consider using cmpopts.AcyclicTransformer

goroutine 3178 [running]:
github.com/google/go-cmp/cmp.(*recChecker).Check(0x0?, {0xc019980000, 0x10000, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:630 +0x1bf
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7590?})
        github.com/google/[email protected]/cmp/compare.go:244 +0x18f
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d5574d0?, {0x5066ec0?, 0xc00d5572f0?, 0x5?}, {0x5066ec0?, 0xc00d5574d0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d5572f0?, 0x0?}, {0x5066ec0?, 0xc00d5574d0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7ac80?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fdbe140?, 0x0?}, {0x5442280?, 0xc00fdbe160?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7540?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d557098?, {0x5066ec0?, 0xc00d556ed0?, 0x5?}, {0x5066ec0?, 0xc00d557098?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d556ed0?, 0x0?}, {0x5066ec0?, 0xc00d557098?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7ac40?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fdbe0e0?, 0x0?}, {0x5442280?, 0xc00fdbe100?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff74f0?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d556c60?, {0x5066ec0?, 0xc00d556a98?, 0x5?}, {0x5066ec0?, 0xc00d556c60?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d556a98?, 0x0?}, {0x5066ec0?, 0xc00d556c60?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7ac00?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fdbe080?, 0x0?}, {0x5442280?, 0xc00fdbe0a0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff74a0?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d556840?, {0x5066ec0?, 0xc00d556678?, 0x5?}, {0x5066ec0?, 0xc00d556840?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d556678?, 0x0?}, {0x5066ec0?, 0xc00d556840?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7abc0?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fdbe020?, 0x0?}, {0x5442280?, 0xc00fdbe040?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7450?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d556420?, {0x5066ec0?, 0xc00d556258?, 0x5?}, {0x5066ec0?, 0xc00d556420?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d556258?, 0x0?}, {0x5066ec0?, 0xc00d556420?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7ab80?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbfa0?, 0x0?}, {0x5442280?, 0xc00fcfbfd0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7400?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54ffb0?, {0x5066ec0?, 0xc00d54fe00?, 0x5?}, {0x5066ec0?, 0xc00d54ffb0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54fe00?, 0x0?}, {0x5066ec0?, 0xc00d54ffb0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7ab40?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbf10?, 0x0?}, {0x5442280?, 0xc00fcfbf40?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff73b0?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54fbc0?, {0x5066ec0?, 0xc00d54fa10?, 0x5?}, {0x5066ec0?, 0xc00d54fbc0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54fa10?, 0x0?}, {0x5066ec0?, 0xc00d54fbc0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7ab00?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbe80?, 0x0?}, {0x5442280?, 0xc00fcfbeb0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7360?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54f7d0?, {0x5066ec0?, 0xc00d54f620?, 0x5?}, {0x5066ec0?, 0xc00d54f7d0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54f620?, 0x0?}, {0x5066ec0?, 0xc00d54f7d0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7aac0?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbdf0?, 0x0?}, {0x5442280?, 0xc00fcfbe20?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7310?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54f3e0?, {0x5066ec0?, 0xc00d54f230?, 0x5?}, {0x5066ec0?, 0xc00d54f3e0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54f230?, 0x0?}, {0x5066ec0?, 0xc00d54f3e0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7aa80?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbd60?, 0x0?}, {0x5442280?, 0xc00fcfbd90?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7270?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54eff0?, {0x5066ec0?, 0xc00d54ee28?, 0x5?}, {0x5066ec0?, 0xc00d54eff0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54ee28?, 0x0?}, {0x5066ec0?, 0xc00d54eff0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7aa40?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbcd0?, 0x0?}, {0x5442280?, 0xc00fcfbd00?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7220?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54ebe8?, {0x5066ec0?, 0xc00d54ea38?, 0x5?}, {0x5066ec0?, 0xc00d54ebe8?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54ea38?, 0x0?}, {0x5066ec0?, 0xc00d54ebe8?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7aa00?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbc40?, 0x0?}, {0x5442280?, 0xc00fcfbc70?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff71d0?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54e7f8?, {0x5066ec0?, 0xc00d54e648?, 0x5?}, {0x5066ec0?, 0xc00d54e7f8?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54e648?, 0x0?}, {0x5066ec0?, 0xc00d54e7f8?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a9c0?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbbb0?, 0x0?}, {0x5442280?, 0xc00fcfbbe0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7180?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54e3f0?, {0x5066ec0?, 0xc00d54e258?, 0x5?}, {0x5066ec0?, 0xc00d54e3f0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54e258?, 0x0?}, {0x5066ec0?, 0xc00d54e3f0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a980?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfbb20?, 0x0?}, {0x5442280?, 0xc00fcfbb50?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7130?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54bfe0?, {0x5066ec0?, 0xc00d54be60?, 0x5?}, {0x5066ec0?, 0xc00d54bfe0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54be60?, 0x0?}, {0x5066ec0?, 0xc00d54bfe0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a940?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfba90?, 0x0?}, {0x5442280?, 0xc00fcfbac0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff70e0?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54bbf0?, {0x5066ec0?, 0xc00d54ba58?, 0x5?}, {0x5066ec0?, 0xc00d54bbf0?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54ba58?, 0x0?}, {0x5066ec0?, 0xc00d54bbf0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a900?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfba00?, 0x0?}, {0x5442280?, 0xc00fcfba30?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7090?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54b800?, {0x5066ec0?, 0xc00d54b650?, 0x5?}, {0x5066ec0?, 0xc00d54b800?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54b650?, 0x0?}, {0x5066ec0?, 0xc00d54b800?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a8c0?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfb970?, 0x0?}, {0x5442280?, 0xc00fcfb9a0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff7040?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54b410?, {0x5066ec0?, 0xc00d54b260?, 0x5?}, {0x5066ec0?, 0xc00d54b410?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54b260?, 0x0?}, {0x5066ec0?, 0xc00d54b410?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a880?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfb8e0?, 0x0?}, {0x5442280?, 0xc00fcfb910?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff6ff0?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54b020?, {0x5066ec0?, 0xc00d54ae70?, 0x5?}, {0x5066ec0?, 0xc00d54b020?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54ae70?, 0x0?}, {0x5066ec0?, 0xc00d54b020?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a840?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfb850?, 0x0?}, {0x5442280?, 0xc00fcfb880?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff6fa0?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54ac18?, {0x5066ec0?, 0xc00d54aa68?, 0x5?}, {0x5066ec0?, 0xc00d54ac18?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54aa68?, 0x0?}, {0x5066ec0?, 0xc00d54ac18?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d98, 0xc010b7a800?})
        github.com/google/[email protected]/cmp/compare.go:259 +0x439
github.com/google/go-cmp/cmp.(*state).compareInterface(0xc009db7400?, {0x6a0c010?, 0x5442280?}, {0x5442280?, 0xc00fcfb7c0?, 0x0?}, {0x5442280?, 0xc00fcfb7f0?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:577 +0x3a9
github.com/google/go-cmp/cmp.(*state).compareAny(0xc009db7400, {0x69f6d68, 0xc010ff6f50?})
        github.com/google/[email protected]/cmp/compare.go:295 +0xab6
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0012abfc0, 0xc00d54a828?, {0x5066ec0?, 0xc00d54a678?, 0x5?}, {0x5066ec0?, 0xc00d54a828?, 0x6a0c010?})
        github.com/google/[email protected]/cmp/options.go:323 +0x205
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc009db7400, {0x6a0c010, 0x5066ec0}, {0x5066ec0?, 0xc00d54a678?, 0x0?}, {0x5066ec0?, 0xc00d54a828?, 0x0?})
        github.com/google/[email protected]/cmp/compare.go:304 +0x11a
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2
        sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223 +0x31c

How can we reproduce it?

This consistently happens for us when upgrading from version v0.33.0 to version v0.34.0 and is resolved when downgrading back to v0.33.0

What environment did it happen in?

Crossplane version: v0.34+
Kubernetes version: 1.24.13
@HenriWilliams HenriWilliams added the bug Something isn't working label Jun 1, 2023
@MisterMX
Copy link
Collaborator

MisterMX commented Jun 2, 2023

Have you tried upgrading to another version, like v0.35.0 or higher?

@HenriWilliams
Copy link
Author

Yes, the same error seems to be present in all versions above v0.34.0

@MisterMX
Copy link
Collaborator

MisterMX commented Jun 5, 2023

Can you provide the YAML resources you have used to reproduce this issue?

@HenriWilliams
Copy link
Author

Unfortunately I have around 1000 different YAML resources and currently don't know which of them are causing the issue

@MisterMX
Copy link
Collaborator

MisterMX commented Jun 9, 2023

Then I would suggest, you debug the provider locally against your cluster and see where exactly it panics. cmpopts.SortSlices is used numerous times in many different resources.

@haarchri
Copy link
Member

any option to update to a newer version ?

@MisterMX
Copy link
Collaborator

MisterMX commented Aug 4, 2023

Closing due to staleness. Feel free to reopen if something changes.

@MisterMX MisterMX closed this as completed Aug 4, 2023
@fabioaraujopt
Copy link

This is happening to us as well but only for the "Role" kind, did you end up finding what was causing your issue?

@MisterMX
Copy link
Collaborator

@fabioaraujopt can you give us a YAML that is causing the crash at your end?

@eapri
Copy link

eapri commented Mar 5, 2024 •
edited
Loading

I also see this kind of logging in the crossplane-contrib/provider-aws pod logging. We are currently on version v0.46.0.

E0305 13:44:13.761614       1 runtime.go:77] Observed a panic: recursive set of Transformers detected:
	Transformer(cmpopts.SortSlices, cmpopts.sliceSorter.sort): interface {} => interface {}
consider using cmpopts.AcyclicTransformer
goroutine 4048 [running]:
k8s.io/apimachinery/pkg/util/runtime.logPanic({0x5adc5a0?, 0xc008b0c9c0})
	k8s.io/[email protected]/pkg/util/runtime/runtime.go:75 +0x85
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile.func1()
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:108 +0xb2
panic({0x5adc5a0?, 0xc008b0c9c0?})
	runtime/panic.go:914 +0x21f
github.com/google/go-cmp/cmp.(*recChecker).Check(0x0?, {0xc00feaa000, 0x10000, 0x0?})
	github.com/google/[email protected]/cmp/compare.go:632 +0x459
github.com/google/go-cmp/cmp.(*state).compareAny(0xc006681d60, {0x786af30, 0xc009899bd0?})
	github.com/google/[email protected]/cmp/compare.go:246 +0x24f
github.com/google/go-cmp/cmp.(*transformer).apply(0xc0082baa80, 0xc0074b4d80?, {0x59f8520?, 0xc0074b4a50?, 0x5?}, {0x59f8520?, 0xc0074b4d80?, 0x7894ba8?})
	github.com/google/[email protected]/cmp/options.go:328 +0x228
github.com/google/go-cmp/cmp.(*state).tryOptions(0xc006681d60, {0x7894ba8, 0x59f8520}, {0x59f8520?, 0xc0074b4a50?, 0x7894ba8?}, {0x59f8520?, 0xc0074b4d80?, 0x4dea35?})
	github.com/google/[email protected]/cmp/compare.go:306 +0x109
github.com/google/go-cmp/cmp.(*state).compareAny(0xc006681d60, {0x786af00,

...

github.com/google/go-cmp/cmp.(*state).comparePtr(0xc006681d60, {0x7894ba8, 0x5fbb860}, {0x5fbb860?, 0xc00dd49420?, 0x4539ef?}, {0x5fbb860?, 0xc00dd493b0?, 0xc0082baa01?})
	github.com/google/[email protected]/cmp/compare.go:566 +0x391
github.com/google/go-cmp/cmp.(*state).compareAny(0xc006681d60, {0x786af60, 0xc0082bab80?})
	github.com/google/[email protected]/cmp/compare.go:295 +0xb92
github.com/google/go-cmp/cmp.Diff({0x5fbb860, 0xc00dd49420}, {0x5fbb860, 0xc00dd493b0}, {0xc0082bab00?, 0xc00ad7ffa0?, 0xc00ad7ffb0?})
	github.com/google/[email protected]/cmp/compare.go:122 +0x75
github.com/crossplane-contrib/provider-aws/pkg/clients/iam.IsRoleUpToDate({{0xc00319af00, 0x2d1}, 0xc00ad7ff90, 0xc002d4192c, 0xc00ad7ffa0, 0xc00ad7ffb0, {0xc003df8a00, 0x10, 0x10}}, {0xc00cfc3620, ...})
	github.com/crossplane-contrib/provider-aws/pkg/clients/iam/role.go:191 +0x3a8
github.com/crossplane-contrib/provider-aws/pkg/controller/iam/role.(*external).Observe(0xc00de6ec20, {0x786c098, 0xc00873aa10}, {0x78a91e0?, 0xc00b14a960})
	github.com/crossplane-contrib/provider-aws/pkg/controller/iam/role/controller.go:144 +0x55e
github.com/crossplane/crossplane-runtime/pkg/reconciler/managed.(*Reconciler).Reconcile(0xc00094a1e0, {0x786bff0, 0xc0015ffdd0}, {{{0x0, 0x0}, {0xc001d8d060, 0x1c}}})
	github.com/crossplane/[email protected]/pkg/reconciler/managed/reconciler.go:903 +0x2b08
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Reconcile(0x78728b0?, {0x786bff0?, 0xc0015ffdd0?}, {{{0x0?, 0xb?}, {0xc001d8d060?, 0x0?}}})
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:119 +0xb7
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).reconcileHandler(0xc000922f00, {0x786c028, 0xc0005dd9a0}, {0x64cf060?, 0xc00317a020?})
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:316 +0x3c5
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).processNextWorkItem(0xc000922f00, {0x786c028, 0xc0005dd9a0})
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:266 +0x1c9
sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2.2()
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:227 +0x79
created by sigs.k8s.io/controller-runtime/pkg/internal/controller.(*Controller).Start.func2 in goroutine 755
	sigs.k8s.io/[email protected]/pkg/internal/controller/controller.go:223 +0x565

note this part:

github.com/crossplane-contrib/provider-aws/pkg/clients/iam.IsRoleUpToDate({{0xc00319af00, 0x2d1}, 0xc00ad7ff90, 0xc002d4192c, 0xc00ad7ffa0, 0xc00ad7ffb0, {0xc003df8a00, 0x10, 0x10}}, {0xc00cfc3620, ...})
	github.com/crossplane-contrib/provider-aws/pkg/clients/iam/role.go:191 +0x3a8

maybe it hints to location of the problem

The pods keeps running, when provided with resources, but is becoming a problem for our logging license.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
5 participants
@eapri @MisterMX @haarchri @fabioaraujopt @HenriWilliams