Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feature: multiplexing support for database plugins #14033

Merged
merged 23 commits into from
Feb 17, 2022
Merged
Show file tree
Hide file tree
Changes from 22 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
b1037ba
feat: DB plugin multiplexing (#13734)
fairclothjm Feb 3, 2022
4782d45
feat: plugin multiplexing: handle plugin client cleanup (#13896)
fairclothjm Feb 10, 2022
25620a1
feature: multiplexing: fix unit tests (#14007)
fairclothjm Feb 11, 2022
c4ec473
feat: multiplexing: handle v5 plugin compiled with new sdk
fairclothjm Feb 11, 2022
51eabad
add mux supported flag and increase test coverage
fairclothjm Feb 11, 2022
2fc2de5
set multiplexingSupport field in plugin server
fairclothjm Feb 11, 2022
774606a
remove multiplexingSupport field in sdk
fairclothjm Feb 12, 2022
82813ee
revert postgres to non-multiplexed
fairclothjm Feb 12, 2022
a8e4cba
add comments on grpc server fields
fairclothjm Feb 12, 2022
f4a11ed
use pointer receiver on grpc server methods
fairclothjm Feb 14, 2022
909f23a
add changelog
fairclothjm Feb 14, 2022
efaa9e7
use pointer for grpcserver instance
fairclothjm Feb 14, 2022
1a5a609
Use a gRPC server to determine if a plugin should be multiplexed
Feb 14, 2022
c0aaec2
Apply suggestions from code review
fairclothjm Feb 15, 2022
b18bdc6
add lock to removePluginClient
fairclothjm Feb 15, 2022
a46478c
add multiplexingSupport field to externalPlugin struct
fairclothjm Feb 15, 2022
189382d
do not send nil to grpc MultiplexingSupport
fairclothjm Feb 15, 2022
49fab35
Merge branch 'feature-multiplexing-server' into feature-multiplexing
fairclothjm Feb 15, 2022
451d9c7
check err before logging
fairclothjm Feb 15, 2022
baa01f0
handle locking scenario for cleanupFunc
fairclothjm Feb 15, 2022
d4d20c7
allow ServeConfigMultiplex to dispense v5 plugin
fairclothjm Feb 15, 2022
3c9cd0d
reposition structs, add err check and comments
fairclothjm Feb 16, 2022
c60d273
add comment on locking for cleanupExternalPlugin
fairclothjm Feb 16, 2022
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -194,6 +194,7 @@ proto: bootstrap
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative sdk/database/dbplugin/*.proto
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative sdk/database/dbplugin/v5/proto/*.proto
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative sdk/plugin/pb/*.proto
protoc --go_out=. --go_opt=paths=source_relative --go-grpc_out=. --go-grpc_opt=paths=source_relative sdk/helper/pluginutil/*.proto

# No additional sed expressions should be added to this list. Going forward
# we should just use the variable names choosen by protobuf. These are left
Expand Down
1 change: 1 addition & 0 deletions builtin/logical/database/backend.go
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,7 @@ func Backend(conf *logical.BackendConfig) *databaseBackend {
}

type databaseBackend struct {
// connections holds configured database connections by config name
connections map[string]*dbPluginInstance
logger log.Logger

Expand Down
5 changes: 5 additions & 0 deletions builtin/logical/database/path_config_connection.go
Original file line number Diff line number Diff line change
Expand Up @@ -329,6 +329,8 @@ func (b *databaseBackend) connectionWriteHandler() framework.OperationFunc {
}
config.ConnectionDetails = initResp.Config

b.Logger().Debug("created database object", "name", name, "plugin_name", config.PluginName)

b.Lock()
defer b.Unlock()

Expand Down Expand Up @@ -365,6 +367,9 @@ func (b *databaseBackend) connectionWriteHandler() framework.OperationFunc {
"Vault (or the sdk if using a custom plugin) to gain password policy support", config.PluginName))
}

if len(resp.Warnings) == 0 {
return nil, nil
}
return resp, nil
}
}
Expand Down
3 changes: 3 additions & 0 deletions changelog/14033.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
```release-note:feature
**Database plugin multiplexing**: manage multiple database connections with a single plugin process
```
4 changes: 2 additions & 2 deletions helper/builtinplugins/registry.go
Original file line number Diff line number Diff line change
Expand Up @@ -150,8 +150,8 @@ type registry struct {
logicalBackends map[string]logical.Factory
}

// Get returns the BuiltinFactory func for a particular backend plugin
// from the plugins map.
// Get returns the Factory func for a particular backend plugin from the
// plugins map.
func (r *registry) Get(name string, pluginType consts.PluginType) (func() (interface{}, error), bool) {
switch pluginType {
case consts.PluginTypeCredential:
Expand Down
2 changes: 1 addition & 1 deletion helper/forwarding/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion helper/identity/mfa/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion helper/identity/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion helper/storagepacker/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion physical/raft/types.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 0 additions & 1 deletion plugins/database/postgresql/postgresql.go
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,6 @@ var (
singleQuotedPhrases = regexp.MustCompile(`('.*?')`)
)

// New implements builtinplugins.BuiltinFactory
func New() (interface{}, error) {
db := new()
// Wrap the plugin with middleware to sanitize errors
Expand Down
2 changes: 1 addition & 1 deletion sdk/database/dbplugin/database.pb.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

30 changes: 26 additions & 4 deletions sdk/database/dbplugin/v5/grpc_database_plugin.go
Original file line number Diff line number Diff line change
Expand Up @@ -5,21 +5,25 @@ import (

"github.com/hashicorp/go-plugin"
"github.com/hashicorp/vault/sdk/database/dbplugin/v5/proto"
"github.com/hashicorp/vault/sdk/helper/pluginutil"
"google.golang.org/grpc"
)

// handshakeConfigs are used to just do a basic handshake between
// a plugin and host. If the handshake fails, a user friendly error is shown.
// This prevents users from executing bad plugins or executing a plugin
// directory. It is a UX feature, not a security feature.
var handshakeConfig = plugin.HandshakeConfig{
ProtocolVersion: 5,
var HandshakeConfig = plugin.HandshakeConfig{
MagicCookieKey: "VAULT_DATABASE_PLUGIN",
MagicCookieValue: "926a0820-aea2-be28-51d6-83cdf00e8edb",
}

// Factory is the factory function to create a dbplugin Database.
type Factory func() (interface{}, error)

type GRPCDatabasePlugin struct {
Impl Database
FactoryFunc Factory
Impl Database

// Embeding this will disable the netRPC protocol
plugin.NetRPCUnsupportedPlugin
Expand All @@ -31,7 +35,25 @@ var (
)

func (d GRPCDatabasePlugin) GRPCServer(_ *plugin.GRPCBroker, s *grpc.Server) error {
proto.RegisterDatabaseServer(s, gRPCServer{impl: d.Impl})
var server gRPCServer

if d.Impl != nil {
server = gRPCServer{singleImpl: d.Impl}
} else {
// multiplexing is supported
server = gRPCServer{
factoryFunc: d.FactoryFunc,
instances: make(map[string]Database),
}

// Multiplexing is enabled for this plugin, register the server so we
// can tell the client in Vault.
pluginutil.RegisterPluginMultiplexingServer(s, pluginutil.PluginMultiplexingServerImpl{
Supported: true,
fairclothjm marked this conversation as resolved.
Show resolved Hide resolved
})
}

proto.RegisterDatabaseServer(s, &server)
return nil
}

Expand Down
Loading