Skip to content

Commit

Permalink
[Filebeat] Rename googlecloud module to gcp module (#22214)
Browse files Browse the repository at this point in the history
  • Loading branch information
kaiyan-sheng authored Nov 25, 2020
1 parent 96974f2 commit 113afdc
Show file tree
Hide file tree
Showing 42 changed files with 675 additions and 491 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.next.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -720,6 +720,7 @@ from being added to events by default. {pull}18159[18159]
- Add SSL option to checkpoint module {pull}19560[19560]
- Add max_number_of_messages config into s3 input. {pull}21993[21993]
- Update Okta documentation for new stateful restarts. {pull}22091[22091]
- Rename googlecloud module to gcp module. {pull}22214[22214]
- Rename awscloudwatch input to aws-cloudwatch. {pull}22228[22228]
- Rename google-pubsub input to gcp-pubsub. {pull}22213[22213]
- Copy tag names from MISP data into events. {pull}21664[21664]
Expand Down
6 changes: 3 additions & 3 deletions filebeat/docs/fields.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ grouped in the following categories:
* <<exported-fields-envoyproxy>>
* <<exported-fields-f5>>
* <<exported-fields-fortinet>>
* <<exported-fields-googlecloud>>
* <<exported-fields-gcp>>
* <<exported-fields-gsuite>>
* <<exported-fields-haproxy>>
* <<exported-fields-host-processor>>
Expand Down Expand Up @@ -68452,8 +68452,8 @@ type: integer

--

[[exported-fields-googlecloud]]
== Google Cloud fields
[[exported-fields-gcp]]
== Google Cloud Platform (GCP) fields

Module for handling logs from Google Cloud.

Expand Down
2 changes: 2 additions & 0 deletions filebeat/docs/index.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -64,4 +64,6 @@ include::./faq.asciidoc[]

include::{libbeat-dir}/contributing-to-beats.asciidoc[]

include::redirects.asciidoc[]


Original file line number Diff line number Diff line change
Expand Up @@ -2,10 +2,10 @@
This file is generated! See scripts/docs_collector.py
////

[[filebeat-module-googlecloud]]
[[filebeat-module-gcp]]
[role="xpack"]

:modulename: googlecloud
:modulename: gcp
:has-dashboards: false

== Google Cloud module
Expand All @@ -29,18 +29,18 @@ include::../include/config-option-intro.asciidoc[]
==== `audit` fileset settings

[role="screenshot"]
image::./images/filebeat-googlecloud-audit.png[]
image::./images/filebeat-gcp-audit.png[]

Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
audit:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-audit
var.subscription_name: filebeat-googlecloud-audit-sub
var.topic: gcp-vpc-audit
var.subscription_name: filebeat-gcp-audit-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -80,12 +80,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
vpcflow:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-flowlogs
var.subscription_name: filebeat-googlecloud-vpc-flowlogs-sub
var.topic: gcp-vpc-flowlogs
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -125,12 +125,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
firewall:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-firewall
var.subscription_name: filebeat-googlecloud-vpc-firewall-sub
var.topic: gcp-vpc-firewall
var.subscription_name: filebeat-gcp-vpc-firewall-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -170,5 +170,5 @@ field. Defaults to `false`, meaning the original message is not saved.
=== Fields

For a description of each field in the module, see the
<<exported-fields-googlecloud,exported fields>> section.
<<exported-fields-gcp,exported fields>> section.

4 changes: 2 additions & 2 deletions filebeat/docs/modules_list.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ This file is generated! See scripts/docs_collector.py
* <<filebeat-module-envoyproxy>>
* <<filebeat-module-f5>>
* <<filebeat-module-fortinet>>
* <<filebeat-module-googlecloud>>
* <<filebeat-module-gcp>>
* <<filebeat-module-gsuite>>
* <<filebeat-module-haproxy>>
* <<filebeat-module-ibmmq>>
Expand Down Expand Up @@ -91,7 +91,7 @@ include::modules/elasticsearch.asciidoc[]
include::modules/envoyproxy.asciidoc[]
include::modules/f5.asciidoc[]
include::modules/fortinet.asciidoc[]
include::modules/googlecloud.asciidoc[]
include::modules/gcp.asciidoc[]
include::modules/gsuite.asciidoc[]
include::modules/haproxy.asciidoc[]
include::modules/ibmmq.asciidoc[]
Expand Down
10 changes: 10 additions & 0 deletions filebeat/docs/redirects.asciidoc
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
["appendix",role="exclude",id="redirects"]
= Deleted pages

The following pages have moved or been deleted.

[role="exclude",id="filebeat-module-googlecloud"]
== Google Cloud module

See <<filebeat-module-gcp>>.

73 changes: 65 additions & 8 deletions x-pack/filebeat/filebeat.reference.yml
Original file line number Diff line number Diff line change
Expand Up @@ -813,8 +813,8 @@ filebeat.modules:
# "+02:00" for GMT+02:00
# var.tz_offset: local

#----------------------------- Google Cloud Module -----------------------------
- module: googlecloud
#--------------------- Google Cloud Platform (GCP) Module ---------------------
- module: gcp
vpcflow:
enabled: true

Expand All @@ -823,11 +823,11 @@ filebeat.modules:

# Google Pub/Sub topic containing VPC flow logs. Stackdriver must be
# configured to use this topic as a sink for VPC flow logs.
var.topic: googlecloud-vpc-flowlogs
var.topic: gcp-vpc-flowlogs

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-googlecloud-vpc-flowlogs-sub
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub

# Credentials file for the service account with authorization to read from
# the subscription.
Expand All @@ -841,11 +841,11 @@ filebeat.modules:

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: googlecloud-vpc-firewall
var.topic: gcp-vpc-firewall

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-googlecloud-firewall-sub
var.subscription_name: filebeat-gcp-firewall-sub

# Credentials file for the service account with authorization to read from
# the subscription.
Expand All @@ -859,11 +859,68 @@ filebeat.modules:

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: googlecloud-vpc-audit
var.topic: gcp-vpc-audit

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-googlecloud-audit
var.subscription_name: filebeat-gcp-audit

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

#----------------------------- Googlecloud Module -----------------------------
# googlecloud module is deprecated, please use gcp instead
- module: gcp
vpcflow:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing VPC flow logs. Stackdriver must be
# configured to use this topic as a sink for VPC flow logs.
var.topic: gcp-vpc-flowlogs

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

firewall:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-firewall

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-firewall-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

audit:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-audit

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-audit

# Credentials file for the service account with authorization to read from
# the subscription.
Expand Down
2 changes: 1 addition & 1 deletion x-pack/filebeat/include/list.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

54 changes: 54 additions & 0 deletions x-pack/filebeat/module/gcp/_meta/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
- module: gcp
vpcflow:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing VPC flow logs. Stackdriver must be
# configured to use this topic as a sink for VPC flow logs.
var.topic: gcp-vpc-flowlogs

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

firewall:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-firewall

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-firewall-sub

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json

audit:
enabled: true

# Google Cloud project ID.
var.project_id: my-gcp-project-id

# Google Pub/Sub topic containing firewall logs. Stackdriver must be
# configured to use this topic as a sink for firewall logs.
var.topic: gcp-vpc-audit

# Google Pub/Sub subscription for the topic. Filebeat will create this
# subscription if it does not exist.
var.subscription_name: filebeat-gcp-audit

# Credentials file for the service account with authorization to read from
# the subscription.
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
[role="xpack"]

:modulename: googlecloud
:modulename: gcp
:has-dashboards: false

== Google Cloud module
Expand All @@ -24,18 +24,18 @@ include::../include/config-option-intro.asciidoc[]
==== `audit` fileset settings

[role="screenshot"]
image::./images/filebeat-googlecloud-audit.png[]
image::./images/filebeat-gcp-audit.png[]

Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
audit:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-audit
var.subscription_name: filebeat-googlecloud-audit-sub
var.topic: gcp-vpc-audit
var.subscription_name: filebeat-gcp-audit-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -75,12 +75,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
vpcflow:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-flowlogs
var.subscription_name: filebeat-googlecloud-vpc-flowlogs-sub
var.topic: gcp-vpc-flowlogs
var.subscription_name: filebeat-gcp-vpc-flowlogs-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down Expand Up @@ -120,12 +120,12 @@ Example config:

[source,yaml]
----
- module: googlecloud
- module: gcp
firewall:
enabled: true
var.project_id: my-gcp-project-id
var.topic: googlecloud-vpc-firewall
var.subscription_name: filebeat-googlecloud-vpc-firewall-sub
var.topic: gcp-vpc-firewall
var.subscription_name: filebeat-gcp-vpc-firewall-sub
var.credentials_file: ${path.config}/gcp-service-account-xyz.json
var.keep_original_message: false
----
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
- key: googlecloud
title: Google Cloud
- key: gcp
title: Google Cloud Platform (GCP)
description: >
Module for handling logs from Google Cloud.
fields:
Expand Down
Loading

0 comments on commit 113afdc

Please sign in to comment.