Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Avoid generating incomplete configurations in autodiscover #20898

Merged
merged 5 commits into from
Sep 2, 2020
Merged

Avoid generating incomplete configurations in autodiscover #20898

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
8f5e7e4
Avoid generating incomplete configurations in autodiscover
jsoriano Sep 1, 2020
98fdab6
Fix changelog
jsoriano Sep 1, 2020
76b79bb
Take into account resolution of environment variables
jsoriano Sep 2, 2020
a83dca6
Revert unnecesary change
jsoriano Sep 2, 2020
9412722
After returning our own error at the end of resolvers, we don't need …
jsoriano Sep 2, 2020
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.next.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- Ensure dynamic template names are unique for the same field. {pull}18849[18849]
- Remove the deprecated `xpack.monitoring.*` settings. Going forward only `monitoring.*` settings may be used. {issue}9424[9424] {pull}18608[18608]
- Added `certificate` TLS verification mode to ignore server name mismatch. {issue}12283[12283] {pull}20293[20293]
- Autodiscover doesn't generate any configuration when a variable is missing. Previously it generated an incomplete configuration. {pull}20898[20898]

*Auditbeat*

Expand Down Expand Up @@ -167,6 +168,7 @@ https://github.com/elastic/beats/compare/v7.0.0-alpha2...master[Check the HEAD d
- [Metricbeat][Kubernetes] Change cluster_ip field from ip to keyword. {pull}20571[20571]
- Rename cloud.provider `az` value to `azure` inside the add_cloud_metadata processor. {pull}20689[20689]
- Add missing country_name geo field in `add_host_metadata` and `add_observer_metadata` processors. {issue}20796[20796] {pull}20811[20811]
- Explicitly detect missing variables in autodiscover configuration, log them at the debug level. {issue}20568[20568] {pull}20898[20898]

*Auditbeat*

Expand Down
16 changes: 14 additions & 2 deletions libbeat/autodiscover/template/config.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,10 @@
package template

import (
"fmt"

"github.com/elastic/go-ucfg"
"github.com/elastic/go-ucfg/parse"

"github.com/elastic/beats/v7/libbeat/common"
"github.com/elastic/beats/v7/libbeat/common/bus"
Expand Down Expand Up @@ -123,11 +126,20 @@ func ApplyConfigTemplate(event bus.Event, configs []*common.Config, options ...u
if err != nil {
logp.Err("Error building config: %v", err)
}

opts := []ucfg.Option{
ucfg.PathSep("."),
ucfg.Env(vars),
ucfg.ResolveEnv,
ucfg.VarExp,

// Catch-all resolve function to log fields not resolved in any other way.
ucfg.Resolve(func(name string) (string, parse.Config, error) {
jsoriano marked this conversation as resolved.
Show resolved Hide resolved
// Logging here the whole name of the field because it is not included in
// the error reported by Unpack.
logp.Debug("autodiscover", "Configuration includes a field not available in event: %s", name)
return "", parse.Config{}, fmt.Errorf("unavailable field %s", name)
}),
}
opts = append(opts, options...)

Expand All @@ -139,9 +151,9 @@ func ApplyConfigTemplate(event bus.Event, configs []*common.Config, options ...u
}
// Unpack config to process any vars in the template:
var unpacked map[string]interface{}
c.Unpack(&unpacked, opts...)
err = c.Unpack(&unpacked, opts...)
if err != nil {
logp.Err("Error unpacking config: %v", err)
logp.Debug("autodiscover", "Configuration template cannot be resolved: %v", err)
continue
}
// Repack again:
Expand Down
14 changes: 14 additions & 0 deletions libbeat/autodiscover/template/config_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,12 @@ import (
"github.com/elastic/beats/v7/libbeat/common"
"github.com/elastic/beats/v7/libbeat/common/bus"
"github.com/elastic/beats/v7/libbeat/keystore"
"github.com/elastic/beats/v7/libbeat/logp"
)

func TestConfigsMapping(t *testing.T) {
logp.TestingSetup()

config, _ := common.NewConfigFrom(map[string]interface{}{
"correct": "config",
})
Expand Down Expand Up @@ -111,6 +114,17 @@ func TestConfigsMapping(t *testing.T) {
},
expected: []*common.Config{configPorts},
},
// Missing variable, config is not generated
{
mapping: `
- config:
- module: something
hosts: ["${not.exists.host}"]`,
event: bus.Event{
"host": "1.2.3.4",
},
expected: nil,
},
}

for _, test := range tests {
Expand Down