Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ssh exporter #2138

Open
wants to merge 5 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
14 changes: 8 additions & 6 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Main (unreleased)

- Add `otelcol.receiver.solace` component to receive traces from a Solace broker. (@wildum)

- Added `prometheus.exporter.ssh` custom metrics via ssh for remote hosts. (@EHSchmitt4395)

### Enhancements

- Add second metrics sample to the support bundle to provide delta information (@dehaansa)
Expand All @@ -33,7 +35,7 @@ Main (unreleased)

- Fixed an issue in the `otelcol.processor.attribute` component where the actions `delete` and `hash` could not be used with the `pattern` argument. (@wildum)

- Fixed a race condition that could lead to a deadlock when using `import` statements, which could lead to a memory leak on `/metrics` endpoint of an Alloy instance. (@thampiotr)
- Fixed a race condition that could lead to a deadlock when using `import` statements, which could lead to a memory leak on `/metrics` endpoint of an Alloy instance. (@thampiotr)

### Other changes

Expand Down Expand Up @@ -86,7 +88,7 @@ v1.5.0
- Add support for relative paths to `import.file`. This new functionality allows users to use `import.file` blocks in modules
imported via `import.git` and other `import.file`. (@wildum)

- `prometheus.exporter.cloudwatch`: The `discovery` block now has a `recently_active_only` configuration attribute
- `prometheus.exporter.cloudwatch`: The `discovery` block now has a `recently_active_only` configuration attribute
to return only metrics which have been active in the last 3 hours.

- Add Prometheus bearer authentication to a `prometheus.write.queue` component (@freak12techno)
Expand All @@ -99,9 +101,9 @@ v1.5.0

- Fixed a bug in `import.git` which caused a `"non-fast-forward update"` error message. (@ptodev)

- Do not log error on clean shutdown of `loki.source.journal`. (@thampiotr)
- Do not log error on clean shutdown of `loki.source.journal`. (@thampiotr)

- `prometheus.operator.*` components: Fixed a bug which would sometimes cause a
- `prometheus.operator.*` components: Fixed a bug which would sometimes cause a
"failed to create service discovery refresh metrics" error after a config reload. (@ptodev)

### Other changes
Expand Down Expand Up @@ -140,7 +142,7 @@ v1.4.3

- `pyroscope.scrape` no longer tries to scrape endpoints which are not active targets anymore. (@wildum @mattdurham @dehaansa @ptodev)

- Fixed a bug with `loki.source.podlogs` not starting in large clusters due to short informer sync timeout. (@elburnetto-intapp)
- Fixed a bug with `loki.source.podlogs` not starting in large clusters due to short informer sync timeout. (@elburnetto-intapp)

- `prometheus.exporter.windows`: Fixed bug with `exclude` regular expression config arguments which caused missing metrics. (@ptodev)

Expand All @@ -159,7 +161,7 @@ v1.4.2
- Fix parsing of the Level configuration attribute in debug_metrics config block
- Ensure "optional" debug_metrics config block really is optional

- Fixed an issue with `loki.process` where `stage.luhn` and `stage.timestamp` would not apply
- Fixed an issue with `loki.process` where `stage.luhn` and `stage.timestamp` would not apply
default configuration settings correctly (@thampiotr)

- Fixed an issue with `loki.process` where configuration could be reloaded even if there
Expand Down
1 change: 1 addition & 0 deletions docs/sources/reference/compatibility/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -106,6 +106,7 @@ The following components, grouped by namespace, _export_ Targets.
- [prometheus.exporter.snmp](../components/prometheus/prometheus.exporter.snmp)
- [prometheus.exporter.snowflake](../components/prometheus/prometheus.exporter.snowflake)
- [prometheus.exporter.squid](../components/prometheus/prometheus.exporter.squid)
- [prometheus.exporter.ssh](../components/prometheus/prometheus.exporter.ssh)
- [prometheus.exporter.statsd](../components/prometheus/prometheus.exporter.statsd)
- [prometheus.exporter.unix](../components/prometheus/prometheus.exporter.unix)
- [prometheus.exporter.windows](../components/prometheus/prometheus.exporter.windows)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
---
canonical: https://grafana.com/docs/alloy/latest/reference/components/prometheus/prometheus.exporter.ssh/
aliases:
- ../prometheus.exporter.ssh/ # /docs/alloy/latest/reference/components/prometheus.exporter.ssh/
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aliases are not need for this component because it's brand new (this was used for mapping after the structure of the doc was changed)

description: Learn about prometheus.exporter.ssh
title: prometheus.exporter.ssh
---

# prometheus.exporter.ssh

The `prometheus.exporter.ssh` component embeds an SSH exporter for collecting metrics from remote servers over SSH and exporting them as Prometheus metrics.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add the experimental snippet to document the fact that the component is experimental

## Usage

```alloy
prometheus.exporter.ssh "LABEL" {
// Configuration options
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Configuration options
targets {
...
}

All the mandatory arguments and blocks need to be shown in the Usage section.

}
```

## Arguments

The following arguments can be used to configure the exporter's behavior.
All arguments are optional unless specified. Omitted fields take their default values.
Comment on lines +23 to +24
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Observation: In reviewing this I've noticed that the wording of some sections in the Prometheus topics are not consistent with the rest of the components. For example, Arguments in other sections simply state: The following arguments are supported:

I'll create an Issue to follow up on this. No action or suggestions for this PR.


| Name | Type | Description | Default | Required |
| ----------------- | -------- | -------------------------------------------------- | ------- | -------- |
| `verbose_logging` | `bool` | Enable verbose logging for debugging purposes. | `false` | no |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really need such a argument? Normally users would configure the logging level through the logging block.

| `targets` | `block` | One or more target configurations for SSH metrics. | | yes |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
| `targets` | `block` | One or more target configurations for SSH metrics. | | yes |

We don't list blocks in the table in the Arguments section. They are listed in the table in the Blocks section.


## Blocks

The following blocks are supported inside the definition of `prometheus.exporter.ssh`:

| Block | Description | Required |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add/include the Hierarchy column in this table?

| -------------- | ----------------------------------------------------------- | -------- |
| `targets` | Configures an SSH target to collect metrics from. | yes |
| `custom_metrics` | Defines custom metrics to collect from the target server. | yes |

### targets block

The `targets` block defines the remote servers to connect to and the metrics to collect. It supports the following arguments:

| Name | Type | Description | Default | Required |
| ----------------- | --------------------- | ---------------------------------------------------------------------- | ------- | -------- |
| `address` | `string` | The IP address or hostname of the target server. | | yes |
| `port` | `int` | SSH port number. | `22` | no |
| `username` | `string` | SSH username for authentication. | | yes |
| `password` | `secret` | Password for password-based SSH authentication. | | no |
| `key_file` | `string` | Path to the private key file for key-based SSH authentication. | | no |
| `command_timeout` | `int` | Timeout in seconds for each command execution over SSH. | `30` | no |
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be a duration? We rarely use dedicated seconds.

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is the expected behavior if command timeout > scrape timeout?

| `custom_metrics` | `block` | One or more custom metrics to collect from the target server. | | yes |

#### Authentication

You must provide either `password` or `key_file` for SSH authentication. If both are provided, `key_file` will be used.

### custom_metrics block

The `custom_metrics` block defines the metrics to collect from the target server. It supports the following arguments:

| Name | Type | Description | Default | Required |
| -------------- | --------------------- | ---------------------------------------------------------------------------- | ------- | -------- |
| `name` | `string` | The name of the metric. | | yes |
| `command` | `string` | The command to execute over SSH to collect the metric. | | yes |
| `type` | `string` | The type of the metric (`gauge` or `counter`). | | yes |
| `help` | `string` | Help text for the metric. | | no |
| `labels` | `map(string, string)` | Key-value pairs of labels to associate with the metric. | `{}` | no |
| `parse_regex` | `string` | Regular expression to parse the command output and extract the metric value. | | no |

#### Metric Types

- `gauge`: Represents a numerical value that can go up or down.
- `counter`: Represents a cumulative value that only increases.

#### parse_regex
Comment on lines +71 to +76
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
#### Metric Types
- `gauge`: Represents a numerical value that can go up or down.
- `counter`: Represents a cumulative value that only increases.
#### parse_regex
The `type` argument can have either of the following values:
- `gauge`: Represents a numerical value that can go up or down.
- `counter`: Represents a cumulative value that only increases.

We usually don't use sub-headings under the tables, unless there's too much text and there's no other way to structure the information.


If the command output is not a simple numeric value, use `parse_regex` to extract the numeric value from the output.

## Exported fields

{{< docs/shared lookup="reference/components/exporter-component-exports.md" source="alloy" version="<ALLOY_VERSION>" >}}

## Component health

`prometheus.exporter.ssh` is only reported as unhealthy if given an invalid configuration. In those cases, exported fields retain their last healthy values.

## Debug information

`prometheus.exporter.ssh` doesn't expose any component-specific debug information.

## Debug metrics

`prometheus.exporter.ssh` doesn't expose any component-specific debug metrics.

## Example

This example uses a [`prometheus.scrape` component][scrape] to collect metrics from `prometheus.exporter.ssh`:

```alloy
prometheus.exporter.ssh "example" {
verbose_logging = true

targets {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of using Alloy syntax arguments and blocks, I wonder if we should just have a targets argument which contains all the endpoint and auth information, similar to the targets argument for prometheus.exporter.snmp. It could work better with discovery components since they also output a targts array of type list(map(string)).

I'm not sure what is the best way to go yet... will need to think about it. Using Alloy syntax might be ok with some upcoming features like the foreach block
cc @wildum

address = "192.168.1.10"
port = 22
username = "admin"
password = "password"
command_timeout = 10

custom_metrics {
name = "load_average"
command = "cat /proc/loadavg | awk '{print $1}'"
type = "gauge"
help = "Load average over 1 minute"
}
}

targets {
address = "192.168.1.11"
port = 22
username = "monitor"
key_file = "/path/to/private.key"
command_timeout = 15

custom_metrics {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does the metric come with labels too? E.g. the address? If it does, it would be good to document it.

name = "disk_usage"
command = "df / | tail -1 | awk '{print $5}'"
type = "gauge"
help = "Disk usage percentage"
parse_regex = "(\\d+)%"
}
}
}

// Configure a prometheus.scrape component to collect SSH metrics.
prometheus.scrape "demo" {
targets = prometheus.exporter.ssh.example.targets
forward_to = [prometheus.remote_write.demo.receiver]
}

prometheus.remote_write "demo" {
endpoint {
url = PROMETHEUS_REMOTE_WRITE_URL

basic_auth {
username = USERNAME
password = PASSWORD
}
}
}
```

Replace the following:

- `PROMETHEUS_REMOTE_WRITE_URL`: The URL of the Prometheus remote_write-compatible server to send metrics to.
- `USERNAME`: The username to use for authentication to the `remote_write` API.
- `PASSWORD`: The password to use for authentication to the `remote_write` API.

[scrape]: ../prometheus.scrape/

<!-- START GENERATED COMPATIBLE COMPONENTS -->

## Compatible components

`prometheus.exporter.ssh` has exports that can be consumed by the following components:

- Components that consume [Targets](../../../compatibility/#targets-consumers)

{{< admonition type="note" >}}
Connecting some components may not be sensible or components may require further configuration to make the connection work correctly.
Refer to the linked documentation for more details.
{{< /admonition >}}

<!-- END GENERATED COMPATIBLE COMPONENTS -->
1 change: 1 addition & 0 deletions internal/component/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -128,6 +128,7 @@ import (
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/snmp" // Import prometheus.exporter.snmp
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/snowflake" // Import prometheus.exporter.snowflake
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/squid" // Import prometheus.exporter.squid
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/ssh" // Import prometheus.exporter.ssh
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/statsd" // Import prometheus.exporter.statsd
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/unix" // Import prometheus.exporter.unix
_ "github.com/grafana/alloy/internal/component/prometheus/exporter/windows" // Import prometheus.exporter.windows
Expand Down
131 changes: 131 additions & 0 deletions internal/component/prometheus/exporter/ssh/collector.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,131 @@
package ssh

import (
"errors"

"github.com/grafana/alloy/internal/component"
"github.com/grafana/alloy/internal/component/prometheus/exporter"
"github.com/grafana/alloy/internal/featuregate"
"github.com/grafana/alloy/internal/static/integrations"
"github.com/grafana/alloy/internal/static/integrations/ssh_exporter"
)

func init() {
component.Register(component.Registration{
Name: "prometheus.exporter.ssh",
Stability: featuregate.StabilityExperimental,
Args: Arguments{},
Exports: exporter.Exports{},
Build: exporter.New(createExporter, "ssh"),
})
}

func createExporter(opts component.Options, args component.Arguments, defaultInstanceKey string) (integrations.Integration, string, error) {
a := args.(Arguments)
return integrations.NewIntegrationWithInstanceKey(opts.Logger, a.Convert(), defaultInstanceKey)
}

type Arguments struct {
VerboseLogging bool `alloy:"verbose_logging,attr,optional"`
Targets []Target `alloy:"targets,block"`
}

func (a *Arguments) Validate() error {
if len(a.Targets) == 0 {
return errors.New("at least one target must be specified")
}
for _, target := range a.Targets {
if err := target.Validate(); err != nil {
return err
}
}
return nil
}

func (a *Arguments) Convert() *ssh_exporter.Config {
targets := make([]ssh_exporter.Target, len(a.Targets))
for i, t := range a.Targets {
targets[i] = t.Convert()
}
return &ssh_exporter.Config{
VerboseLogging: a.VerboseLogging,
Targets: targets,
}
}

type Target struct {
Address string `alloy:"address,attr"`
Port int `alloy:"port,attr,optional"`
Username string `alloy:"username,attr,optional"`
Password string `alloy:"password,attr,optional"`
KeyFile string `alloy:"key_file,attr,optional"`
CommandTimeout int `alloy:"command_timeout,attr,optional"`
CustomMetrics []CustomMetric `alloy:"custom_metrics,block,optional"`
}

func (t *Target) Validate() error {
if t.Address == "" {
return errors.New("target address cannot be empty")
}
if t.Port <= 0 || t.Port > 65535 {
return errors.New("invalid port")
}
if t.Username == "" {
return errors.New("username cannot be empty")
}
for _, cm := range t.CustomMetrics {
if err := cm.Validate(); err != nil {
return err
}
}
return nil
}

func (t *Target) Convert() ssh_exporter.Target {
customMetrics := make([]ssh_exporter.CustomMetric, len(t.CustomMetrics))
for i, cm := range t.CustomMetrics {
customMetrics[i] = cm.Convert()
}
return ssh_exporter.Target{
Address: t.Address,
Port: t.Port,
Username: t.Username,
Password: t.Password,
KeyFile: t.KeyFile,
CommandTimeout: t.CommandTimeout,
CustomMetrics: customMetrics,
}
}

type CustomMetric struct {
Name string `alloy:"name,attr"`
Command string `alloy:"command,attr"`
Type string `alloy:"type,attr"`
Help string `alloy:"help,attr,optional"`
Labels map[string]string `alloy:"labels,attr,optional"`
ParseRegex string `alloy:"parse_regex,attr,optional"`
}

func (cm *CustomMetric) Validate() error {
if cm.Name == "" {
return errors.New("custom metric name cannot be empty")
}
if cm.Command == "" {
return errors.New("custom metric command cannot be empty")
}
if cm.Type != "gauge" && cm.Type != "counter" {
return errors.New("unsupported metric type")
}
return nil
}

func (cm *CustomMetric) Convert() ssh_exporter.CustomMetric {
return ssh_exporter.CustomMetric{
Name: cm.Name,
Command: cm.Command,
Type: cm.Type,
Help: cm.Help,
Labels: cm.Labels,
ParseRegex: cm.ParseRegex,
}
}
Loading