-
Notifications
You must be signed in to change notification settings - Fork 4.3k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Address a data race issue within identity_store_util::processLocalAlias #13093
Merged
stevendpclark
merged 1 commit into
main
from
stevendpclark/fix-data-race-alias-identity-store
Nov 9, 2021
Merged
Address a data race issue within identity_store_util::processLocalAlias #13093
stevendpclark
merged 1 commit into
main
from
stevendpclark/fix-data-race-alias-identity-store
Nov 9, 2021
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- When loading an existing alias within processLocalAlias we aren't cloning the object from the memory store. There seems to be a data race within the function when calling entity.UpsertAlias and a concurrent invalidation routine. ================== WARNING: DATA RACE Read at 0x00c00bd03d08 by goroutine 94: google.golang.org/protobuf/internal/impl.pointer.Elem() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/pointer_unsafe.go:118 +0x2b3 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:76 +0x265 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:56 +0x12a google.golang.org/protobuf/internal/impl.(*MessageInfo).size() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:40 +0x95 google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:33 +0x6c google.golang.org/protobuf/proto.MarshalOptions.marshal() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:153 +0x1f3 google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:122 +0xa5 github.com/golang/protobuf/proto.marshalAppend() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:40 +0xe4 github.com/golang/protobuf/proto.Marshal() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:23 +0x64 github.com/hashicorp/vault/helper/identity.(*Entity).Clone() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150 github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306 github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate() /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm() /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey() /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey() <autogenerated>:1 +0x7d github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390 github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453() /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71 Previous write at 0x00c00bd03d08 by goroutine 52: github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271 github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672 github.com/hashicorp/vault/vault.possiblyForwardEntityCreation() /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286 github.com/hashicorp/vault/vault.(*Core).handleLoginRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11 github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5 github.com/hashicorp/vault/vault.(*Core).HandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2 github.com/hashicorp/vault/http.request() /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1 github.com/hashicorp/vault/http.handleLogicalInternal.func1() /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.handleRequestForwarding.func1() /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.(*ServeMux).ServeHTTP() /usr/local/go/src/net/http/server.go:2424 +0xc5 github.com/hashicorp/vault/http.wrapHelpHandler.func1() /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapCORSHandler.func1() /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1() /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1() /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapGenericHandler.func1() /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1() /go/pkg/mod/github.com/hashicorp/[email protected]/handlers.go:42 +0xc1 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.serverHandler.ServeHTTP() /usr/local/go/src/net/http/server.go:2878 +0x89a net/http.initALPNRequest.ServeHTTP() /usr/local/go/src/net/http/server.go:3479 +0x34d net/http.(*initALPNRequest).ServeHTTP() <autogenerated>:1 +0x8f net/http.Handler.ServeHTTP-fm() /usr/local/go/src/net/http/server.go:87 +0x75 net/http.(*http2serverConn).runHandler() /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd net/http.(*http2serverConn).processHeaders·dwrap·31() /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64
ncabatoff
approved these changes
Nov 9, 2021
vishalnayak
approved these changes
Nov 9, 2021
stevendpclark
deleted the
stevendpclark/fix-data-race-alias-identity-store
branch
November 9, 2021 15:00
stevendpclark
added a commit
that referenced
this pull request
Nov 9, 2021
…as (#13093) - When loading an existing alias within processLocalAlias we aren't cloning the object from the memory store. There seems to be a data race within the function when calling entity.UpsertAlias and a concurrent invalidation routine. ================== WARNING: DATA RACE Read at 0x00c00bd03d08 by goroutine 94: google.golang.org/protobuf/internal/impl.pointer.Elem() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/pointer_unsafe.go:118 +0x2b3 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:76 +0x265 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:56 +0x12a google.golang.org/protobuf/internal/impl.(*MessageInfo).size() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:40 +0x95 google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:33 +0x6c google.golang.org/protobuf/proto.MarshalOptions.marshal() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:153 +0x1f3 google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:122 +0xa5 github.com/golang/protobuf/proto.marshalAppend() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:40 +0xe4 github.com/golang/protobuf/proto.Marshal() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:23 +0x64 github.com/hashicorp/vault/helper/identity.(*Entity).Clone() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150 github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306 github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate() /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm() /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey() /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey() <autogenerated>:1 +0x7d github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390 github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453() /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71 Previous write at 0x00c00bd03d08 by goroutine 52: github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271 github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672 github.com/hashicorp/vault/vault.possiblyForwardEntityCreation() /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286 github.com/hashicorp/vault/vault.(*Core).handleLoginRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11 github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5 github.com/hashicorp/vault/vault.(*Core).HandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2 github.com/hashicorp/vault/http.request() /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1 github.com/hashicorp/vault/http.handleLogicalInternal.func1() /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.handleRequestForwarding.func1() /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.(*ServeMux).ServeHTTP() /usr/local/go/src/net/http/server.go:2424 +0xc5 github.com/hashicorp/vault/http.wrapHelpHandler.func1() /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapCORSHandler.func1() /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1() /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1() /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapGenericHandler.func1() /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1() /go/pkg/mod/github.com/hashicorp/[email protected]/handlers.go:42 +0xc1 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.serverHandler.ServeHTTP() /usr/local/go/src/net/http/server.go:2878 +0x89a net/http.initALPNRequest.ServeHTTP() /usr/local/go/src/net/http/server.go:3479 +0x34d net/http.(*initALPNRequest).ServeHTTP() <autogenerated>:1 +0x8f net/http.Handler.ServeHTTP-fm() /usr/local/go/src/net/http/server.go:87 +0x75 net/http.(*http2serverConn).runHandler() /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd net/http.(*http2serverConn).processHeaders·dwrap·31() /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64
stevendpclark
added a commit
that referenced
this pull request
Nov 9, 2021
stevendpclark
added a commit
that referenced
this pull request
Nov 9, 2021
stevendpclark
added a commit
that referenced
this pull request
Nov 9, 2021
Changelog was added within #13095 |
stevendpclark
added a commit
that referenced
this pull request
Nov 9, 2021
…rocessLocalAlias (#13093) (#13094) * Address a data race issue within identity_store_util::processLocalAlias (#13093) - When loading an existing alias within processLocalAlias we aren't cloning the object from the memory store. There seems to be a data race within the function when calling entity.UpsertAlias and a concurrent invalidation routine. ================== WARNING: DATA RACE Read at 0x00c00bd03d08 by goroutine 94: google.golang.org/protobuf/internal/impl.pointer.Elem() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/pointer_unsafe.go:118 +0x2b3 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:76 +0x265 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:56 +0x12a google.golang.org/protobuf/internal/impl.(*MessageInfo).size() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:40 +0x95 google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:33 +0x6c google.golang.org/protobuf/proto.MarshalOptions.marshal() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:153 +0x1f3 google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:122 +0xa5 github.com/golang/protobuf/proto.marshalAppend() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:40 +0xe4 github.com/golang/protobuf/proto.Marshal() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:23 +0x64 github.com/hashicorp/vault/helper/identity.(*Entity).Clone() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150 github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306 github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate() /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm() /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey() /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey() <autogenerated>:1 +0x7d github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390 github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453() /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71 Previous write at 0x00c00bd03d08 by goroutine 52: github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271 github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672 github.com/hashicorp/vault/vault.possiblyForwardEntityCreation() /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286 github.com/hashicorp/vault/vault.(*Core).handleLoginRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11 github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5 github.com/hashicorp/vault/vault.(*Core).HandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2 github.com/hashicorp/vault/http.request() /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1 github.com/hashicorp/vault/http.handleLogicalInternal.func1() /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.handleRequestForwarding.func1() /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.(*ServeMux).ServeHTTP() /usr/local/go/src/net/http/server.go:2424 +0xc5 github.com/hashicorp/vault/http.wrapHelpHandler.func1() /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapCORSHandler.func1() /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1() /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1() /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapGenericHandler.func1() /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1() /go/pkg/mod/github.com/hashicorp/[email protected]/handlers.go:42 +0xc1 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.serverHandler.ServeHTTP() /usr/local/go/src/net/http/server.go:2878 +0x89a net/http.initALPNRequest.ServeHTTP() /usr/local/go/src/net/http/server.go:3479 +0x34d net/http.(*initALPNRequest).ServeHTTP() <autogenerated>:1 +0x8f net/http.Handler.ServeHTTP-fm() /usr/local/go/src/net/http/server.go:87 +0x75 net/http.(*http2serverConn).runHandler() /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd net/http.(*http2serverConn).processHeaders·dwrap·31() /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64 * Add missing changelog for pr #13093 (#13095)
stevendpclark
added a commit
that referenced
this pull request
Dec 20, 2021
- Testcase TestIdentityStore_LocalAliasInvalidations identified a data race issue. - This reverts the previous attempt to address the issue from #13093
mlevitt
pushed a commit
to mlevitt/vault
that referenced
this pull request
Dec 23, 2021
…rocessLocalAlias (hashicorp#13093) (hashicorp#13094) * Address a data race issue within identity_store_util::processLocalAlias (hashicorp#13093) - When loading an existing alias within processLocalAlias we aren't cloning the object from the memory store. There seems to be a data race within the function when calling entity.UpsertAlias and a concurrent invalidation routine. ================== WARNING: DATA RACE Read at 0x00c00bd03d08 by goroutine 94: google.golang.org/protobuf/internal/impl.pointer.Elem() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/pointer_unsafe.go:118 +0x2b3 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointerSlow() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:76 +0x265 google.golang.org/protobuf/internal/impl.(*MessageInfo).sizePointer() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:56 +0x12a google.golang.org/protobuf/internal/impl.(*MessageInfo).size() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:40 +0x95 google.golang.org/protobuf/internal/impl.(*MessageInfo).size-fm() /go/pkg/mod/google.golang.org/[email protected]/internal/impl/encode.go:33 +0x6c google.golang.org/protobuf/proto.MarshalOptions.marshal() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:153 +0x1f3 google.golang.org/protobuf/proto.MarshalOptions.MarshalAppend() /go/pkg/mod/google.golang.org/[email protected]/proto/encode.go:122 +0xa5 github.com/golang/protobuf/proto.marshalAppend() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:40 +0xe4 github.com/golang/protobuf/proto.Marshal() /go/pkg/mod/github.com/golang/[email protected]/proto/wire.go:23 +0x64 github.com/hashicorp/vault/helper/identity.(*Entity).Clone() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:34 +0x150 github.com/hashicorp/vault/vault.(*IdentityStore).MemDBEntitiesByBucketKeyInTxn() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:1214 +0x306 github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate() /go/src/github.com/hashicorp/vault/vault/identity_store.go:216 +0xd6c github.com/hashicorp/vault/vault.(*IdentityStore).Invalidate-fm() /go/src/github.com/hashicorp/vault/vault/identity_store.go:160 +0x6d github.com/hashicorp/vault/sdk/framework.(*Backend).InvalidateKey() /go/src/github.com/hashicorp/vault/sdk/framework/backend.go:347 +0x8a github.com/hashicorp/vault/vault.(*IdentityStore).InvalidateKey() <autogenerated>:1 +0x7d github.com/hashicorp/vault/vault.(*Core).asyncInvalidateKey() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:58 +0x390 github.com/hashicorp/vault/vault.(*Core).asyncInvalidateHandler() /go/src/github.com/hashicorp/vault/vault/replication_invalidation_ent.go:71 +0x9b github.com/hashicorp/vault/vault.startReplicationEnt·dwrap·453() /go/src/github.com/hashicorp/vault/vault/replication_util_ent.go:331 +0x71 Previous write at 0x00c00bd03d08 by goroutine 52: github.com/hashicorp/vault/helper/identity.(*Entity).UpsertAlias() /go/src/github.com/hashicorp/vault/helper/identity/identity.go:55 +0x271 github.com/hashicorp/vault/vault.(*IdentityStore).processLocalAlias() /go/src/github.com/hashicorp/vault/vault/identity_store_util.go:720 +0x672 github.com/hashicorp/vault/vault.possiblyForwardEntityCreation() /go/src/github.com/hashicorp/vault/vault/request_handling_util_ent.go:230 +0x286 github.com/hashicorp/vault/vault.(*Core).handleLoginRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:1345 +0x234a github.com/hashicorp/vault/vault.(*Core).handleCancelableRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:607 +0x1a11 github.com/hashicorp/vault/vault.(*Core).switchedLockHandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:442 +0x5b5 github.com/hashicorp/vault/vault.(*Core).HandleRequest() /go/src/github.com/hashicorp/vault/vault/request_handling.go:408 +0xf2 github.com/hashicorp/vault/http.request() /go/src/github.com/hashicorp/vault/http/handler.go:953 +0xb1 github.com/hashicorp/vault/http.handleLogicalInternal.func1() /go/src/github.com/hashicorp/vault/http/logical.go:341 +0xca net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.handleRequestForwarding.func1() /go/src/github.com/hashicorp/vault/http/handler.go:887 +0x4eb net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.(*ServeMux).ServeHTTP() /usr/local/go/src/net/http/server.go:2424 +0xc5 github.com/hashicorp/vault/http.wrapHelpHandler.func1() /go/src/github.com/hashicorp/vault/http/help.go:23 +0x281 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapCORSHandler.func1() /go/src/github.com/hashicorp/vault/http/cors.go:29 +0xb0e net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.rateLimitQuotaWrapping.func1() /go/src/github.com/hashicorp/vault/http/util.go:97 +0xf28 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapDRSecondaryHandler.func1() /go/src/github.com/hashicorp/vault/http/util_ent.go:81 +0x7e3 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/vault/http.wrapGenericHandler.func1() /go/src/github.com/hashicorp/vault/http/handler.go:465 +0x1843 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d github.com/hashicorp/go-cleanhttp.PrintablePathCheckHandler.func1() /go/pkg/mod/github.com/hashicorp/[email protected]/handlers.go:42 +0xc1 net/http.HandlerFunc.ServeHTTP() /usr/local/go/src/net/http/server.go:2046 +0x4d net/http.serverHandler.ServeHTTP() /usr/local/go/src/net/http/server.go:2878 +0x89a net/http.initALPNRequest.ServeHTTP() /usr/local/go/src/net/http/server.go:3479 +0x34d net/http.(*initALPNRequest).ServeHTTP() <autogenerated>:1 +0x8f net/http.Handler.ServeHTTP-fm() /usr/local/go/src/net/http/server.go:87 +0x75 net/http.(*http2serverConn).runHandler() /usr/local/go/src/net/http/h2_bundle.go:5832 +0xdd net/http.(*http2serverConn).processHeaders·dwrap·31() /usr/local/go/src/net/http/h2_bundle.go:5562 +0x64 * Add missing changelog for pr hashicorp#13093 (hashicorp#13095)
heppu
pushed a commit
to heppu/vault
that referenced
this pull request
Jan 13, 2022
…ashicorp#13476) * Attempt to address a data race issue within identity store * Testcase TestIdentityStore_LocalAliasInvalidations identified a data race issue. * This reverts the previous attempt to address the issue from hashicorp#13093
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
cloning the object from the memory store. There seems to be a data
race within the function when calling entity.UpsertAlias and
a concurrent invalidation routine.