Skip to content

Commit

Permalink
Convert Filebeat nginx.access to ECS (#9081)
Browse files Browse the repository at this point in the history
- Convert most fields under `nginx.access.*` to ECS. Previous field names are field aliases towards the new corresponding ECS field:
  - nginx.access.remote_ip_list => network.forwarded_ip
  - nginx.access.user_name => user.name
  - nginx.access.response_code => http.response.status_code
  - nginx.access.referrer => http.request.referrer
  - nginx.access.method => http.request.method
  - nginx.access.url => url.original
  - nginx.access.http_version => http.version
  - nginx.access.agent => user_agent.original
  - nginx.access.geoip.* => source.geo.*
  - nginx.access.user_agent.* => user_agent.*
- Coerce status code and bytes into ints
- Replace one of 3 Mac/FF user agent with Mac/Chrome, to spice things up
  • Loading branch information
webmat authored Nov 27, 2018
1 parent 1411852 commit 0edd07a
Show file tree
Hide file tree
Showing 9 changed files with 397 additions and 394 deletions.
1 change: 1 addition & 0 deletions CHANGELOG.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -130,6 +130,7 @@ https://github.com/elastic/beats/compare/v6.5.0...v7.0.0-alpha1[View commits]
- Rename many `iis.access.*` fields to map to ECS. {pull}9084[9084]
- IIS module's user agent string is no longer encoded (`+` replaced with spaces). {pull}9084[9084]
- Rename many `haproxy.*` fields to map to ECS. {pull}9117[9117]
- Rename many `nginx.access.*` fields to map to ECS. {pull}9081[9081]
*Metricbeat*
Expand Down
77 changes: 77 additions & 0 deletions dev-tools/ecs-migration.yml
Original file line number Diff line number Diff line change
Expand Up @@ -252,3 +252,80 @@
to: source.geo.region_iso_code
alias: true
copy_to: false

- from: nginx.access.remote_ip_list
to: network.forwarded_ip
alias: true
copy_to: false

- from: nginx.access.user_name
to: user.name
alias: true
copy_to: false

- from: nginx.access.url
to: url.original
alias: true
copy_to: false

- from: nginx.access.agent
to: user_agent.original
alias: true
copy_to: false

# Note: `http` is not officially in ECS yet

- from: nginx.access.response_code
to: http.response.status_code
alias: true
copy_to: false

- from: nginx.access.referrer
to: http.request.referrer
alias: true
copy_to: false

- from: nginx.access.method
to: http.request.method
alias: true
copy_to: false

- from: nginx.access.http_version
to: http.version
alias: true
copy_to: false

- from: nginx.access.geoip.continent_name
to: source.geo.continent_name
alias: true
copy_to: false

- from: nginx.access.geoip.country_iso_code
to: source.geo.country_iso_code
alias: true
copy_to: false

- from: nginx.access.geoip.location
to: source.geo.location
alias: true
copy_to: false

- from: nginx.access.geoip.region_name
to: source.geo.region_name
alias: true
copy_to: false

- from: nginx.access.geoip.city_name
to: source.geo.city_name
alias: true
copy_to: false

- from: nginx.access.geoip.region_iso_code
to: source.geo.region_iso_code
alias: true
copy_to: false

- from: nginx.access.agent
to: user_agent.original
alias: true
copy_to: false
7 changes: 7 additions & 0 deletions filebeat/_meta/fields.common.yml
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,13 @@
Referrer for this HTTP request.
example: https://blog.example.com/

# Temporary fixes until ECS is reimported
- name: url.original
type: keyword
description: >
Full original url. The field is stored as keyword.
example: https://blog.example.com/

# Temporary fix to get 7.0 dashboards working
- name: fileset.name
type: alias
Expand Down
Loading

0 comments on commit 0edd07a

Please sign in to comment.