Move add-cloud-metadata to ECS

[ruflin]

Fields renamed:

• meta.cloud.instance_id -> cloud.instance.id
• meta.cloud.instance_name -> cloud.instance.name
• meta.cloud.machine_type -> cloud.machine.type
• meta.cloud.availability_zone -> cloud.availability_zone
• meta.cloud.project_id -> cloud.project.id
• meta.cloud.region -> cloud.region

Further changes:

• Added alias for old fields
• Update asset generation to include ECS assets in libbeat fields.yml. This is needed for testing as the aliases fields are in there.

[elasticmachine]

Pinging @elastic/infrastructure

[webmat]

Almost there. The only small problem I see is that the meta.cloud.provider has been deleted instead of being made an alias to cloud.provider (both in migration file and actual common fields).

Once that alias is back in, I'm good with this.

Oh and I discovered another name clash with ECS ;-)

[webmat]

meta.cloud.provider seems to be directly deleted, instead of becoming an alias to cloud.provider. Please add it back as an alias

[ruflin]

fixed

[webmat]

Make sure you add the cloud.provider alias here as well

[ruflin]

Done, not sure how that slipped through above :-(

[webmat]

Oh jeez. Another thing called ECS. After our schema and AWS' container service (yes, in that order). Moreover, Alibaba's thing is named "Elastic ..." as well.
💯 🤦‍♂️ 😆

[ruflin]

yep, that one we knew ;-)

[webmat]

Restarted the single Travis test that was failing, and it passed. Likely a transient issue.

Not familiar enough with the provider tests to understand the failure in the Alibaba test:

10:44:45 ok  	github.com/elastic/beats/libbeat/outputs/transport/transptest	1.309s	coverage: 47.5% of statements
10:45:06 ok  	github.com/elastic/beats/libbeat/processors/actions	19.241s	coverage: 58.3% of statements
10:45:06 ok  	github.com/elastic/beats/libbeat/processors	19.828s	coverage: 68.3% of statements
10:45:06 command [go test -race -cover -coverprofile /tmp/gotestcover-323955738 github.com/elastic/beats/libbeat/processors/add_cloud_metadata]: exit status 1
10:45:06 --- FAIL: TestRetrieveAlibabaCloudMetadata (19.20s)
10:45:06     <autogenerated>:1: 
10:45:06         	Error Trace:	provider_alibaba_cloud_test.go:85
10:45:06         	Error:      	Not equal: 
10:45:06         	            	expected: common.MapStr{"cloud":common.MapStr{"provider":"ecs", "instance":common.MapStr{"id":"i-wz9g2hqiikg0aliyun2b"}, "region":"cn-shenzhen", "availability_zone":"cn-shenzhen-a"}}
10:45:06         	            	actual  : common.MapStr{}
10:45:06         	            	
10:45:06         	            	Diff:
10:45:06         	            	--- Expected
10:45:06         	            	+++ Actual
10:45:06         	            	@@ -1,2 +1,2 @@
10:45:06         	            	-(common.MapStr) (len=1) {"cloud":{"availability_zone":"cn-shenzhen-a","instance":{"id":"i-wz9g2hqiikg0aliyun2b"},"provider":"ecs","region":"cn-shenzhen"}}
10:45:06         	            	+(common.MapStr) {}
10:45:06         	            	 
10:45:06         	Test:       	TestRetrieveAlibabaCloudMetadata
10:45:06 FAIL
10:45:06 coverage: 85.2% of statements
10:45:06 FAIL	github.com/elastic/beats/libbeat/processors/add_cloud_metadata	19.331s
10:45:10 ok  	github.com/elastic/beats/libbeat/processors/add_host_metadata	1.025s	coverage: 83.1% of statements

Test run

[ruflin]

@webmat Already tried to reproduce the above failing test locally but couldn't so far. Not sure how it's possible that it fails especially as it's based on local data. Will need to further investigate.

[webmat]

About the failing test: is there an actual API call being performed? Or is this based on a static fixture file of the API response?

May just be a network hiccup if there's an API call. I guess we'll see after this test run.

[webmat]

If tests pass, this looks good to me