Skip to content

Commit

Permalink
[O11y][GoLang] Resolve the conflict in host.ip field (#7494)
Browse files Browse the repository at this point in the history
* revert changes

* update PR link in changelog.yml file

* address review comments
  • Loading branch information
harnish-elastic authored Aug 29, 2023
1 parent 38c21e3 commit fe8cb5c
Show file tree
Hide file tree
Showing 6 changed files with 200 additions and 1 deletion.
94 changes: 94 additions & 0 deletions packages/golang/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,100 @@ You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommen

For step-by-step instructions on how to set up an integration, see the [Getting started](https://www.elastic.co/guide/en/welcome-to-elastic/current/getting-started-observability.html) guide.

### Troubleshooting

If host.ip is shown conflicted under ``logs-*`` data view, then this issue can be solved by reindexing the ``Heap`` and ``Expvar`` data stream's indices.
To reindex the data, the following steps must be performed.

1. Stop the data stream by going to `Integrations -> Golang -> Integration policies` open the configuration of Golang and disable the `Collect Golang metrics` toggle to reindex metrics data stream and save the integration.

2. Copy data into the temporary index and delete the existing data stream and index template by performing the following steps in the Dev tools.

```
POST _reindex
{
"source": {
"index": "<index_name>"
},
"dest": {
"index": "temp_index"
}
}
```
Example:
```
POST _reindex
{
"source": {
"index": "logs-golang.heap-default"
},
"dest": {
"index": "temp_index"
}
}
```

```
DELETE /_data_stream/<data_stream>
```
Example:
```
DELETE /_data_stream/logs-golang.heap-default
```

```
DELETE _index_template/<index_template>
```
Example:
```
DELETE _index_template/logs-golang.heap
```
3. Go to `Integrations -> Golang -> Settings` and click on `Reinstall Golang`.

4. Copy data from temporary index to new index by performing the following steps in the Dev tools.

```
POST _reindex
{
"conflicts": "proceed",
"source": {
"index": "temp_index"
},
"dest": {
"index": "<index_name>",
"op_type": "create"
}
}
```
Example:
```
POST _reindex
{
"conflicts": "proceed",
"source": {
"index": "temp_index"
},
"dest": {
"index": "logs-golang.heap-default",
"op_type": "create"
}
}
```

5. Verify data is reindexed completely.

6. Start the data stream by going to the `Integrations -> Golang -> Integration policies` and open configuration of integration and enable the `Collect Golang metrics` toggle and save the integration.

7. Delete temporary index by performing the following step in the Dev tools.

```
DELETE temp_index
```

More details about reindexing can be found [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html).

## Logs reference

### expvar
Expand Down
5 changes: 5 additions & 0 deletions packages/golang/changelog.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
# newer versions go on top
- version: "0.4.1"
changes:
- description: Resolve the conflict in host.ip field.
type: bugfix
link: https://github.com/elastic/integrations/pull/7494
- version: "0.4.0"
changes:
- description: Rename ownership from obs-service-integrations to obs-infraobs-integrations
Expand Down
2 changes: 2 additions & 0 deletions packages/golang/data_stream/expvar/fields/ecs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,7 @@
name: event.original
- external: ecs
name: event.type
- external: ecs
name: host.ip
- external: ecs
name: service.address
2 changes: 2 additions & 0 deletions packages/golang/data_stream/heap/fields/ecs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -18,5 +18,7 @@
name: event.original
- external: ecs
name: event.type
- external: ecs
name: host.ip
- external: ecs
name: service.address
96 changes: 96 additions & 0 deletions packages/golang/docs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,6 +35,100 @@ You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommen

For step-by-step instructions on how to set up an integration, see the [Getting started](https://www.elastic.co/guide/en/welcome-to-elastic/current/getting-started-observability.html) guide.

### Troubleshooting

If host.ip is shown conflicted under ``logs-*`` data view, then this issue can be solved by reindexing the ``Heap`` and ``Expvar`` data stream's indices.
To reindex the data, the following steps must be performed.

1. Stop the data stream by going to `Integrations -> Golang -> Integration policies` open the configuration of Golang and disable the `Collect Golang metrics` toggle to reindex metrics data stream and save the integration.

2. Copy data into the temporary index and delete the existing data stream and index template by performing the following steps in the Dev tools.

```
POST _reindex
{
"source": {
"index": "<index_name>"
},
"dest": {
"index": "temp_index"
}
}
```
Example:
```
POST _reindex
{
"source": {
"index": "logs-golang.heap-default"
},
"dest": {
"index": "temp_index"
}
}
```

```
DELETE /_data_stream/<data_stream>
```
Example:
```
DELETE /_data_stream/logs-golang.heap-default
```

```
DELETE _index_template/<index_template>
```
Example:
```
DELETE _index_template/logs-golang.heap
```
3. Go to `Integrations -> Golang -> Settings` and click on `Reinstall Golang`.

4. Copy data from temporary index to new index by performing the following steps in the Dev tools.

```
POST _reindex
{
"conflicts": "proceed",
"source": {
"index": "temp_index"
},
"dest": {
"index": "<index_name>",
"op_type": "create"
}
}
```
Example:
```
POST _reindex
{
"conflicts": "proceed",
"source": {
"index": "temp_index"
},
"dest": {
"index": "logs-golang.heap-default",
"op_type": "create"
}
}
```

5. Verify data is reindexed completely.

6. Start the data stream by going to the `Integrations -> Golang -> Integration policies` and open configuration of integration and enable the `Collect Golang metrics` toggle and save the integration.

7. Delete temporary index by performing the following step in the Dev tools.

```
DELETE temp_index
```

More details about reindexing can be found [here](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html).

## Logs reference

### expvar
Expand Down Expand Up @@ -174,6 +268,7 @@ An example event for `expvar` looks as following:
| golang.expvar.obtained.total.bytes | The total bytes of memory obtained from the OS. | long | byte | gauge |
| golang.expvar.pointer.lookups | The number of pointer lookups performed by the runtime. | long | | gauge |
| golang.expvar.stack.bytes | Bytes in stack spans. | long | byte | gauge |
| host.ip | Host ip addresses. | ip | | |
| input.type | Type of Filebeat input. | keyword | | |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | | |
| tags | List of keywords used to tag each event. | keyword | | |
Expand Down Expand Up @@ -318,6 +413,7 @@ An example event for `heap` looks as following:
| golang.heap.system.released.bytes | Bytes of physical memory returned to the OS. | long | byte | gauge |
| golang.heap.system.stack.bytes | Bytes of stack memory obtained from the OS. | long | byte | gauge |
| golang.heap.system.total.bytes | Bytes of heap memory obtained from the OS. | long | byte | gauge |
| host.ip | Host ip addresses. | ip | | |
| input.type | Type of Filebeat input. | keyword | | |
| service.address | Address where data about this service was collected from. This should be a URI, network address (ipv4:port or [ipv6]:port) or a resource path (sockets). | keyword | | |
| tags | List of keywords used to tag each event. | keyword | | |
Expand Down
2 changes: 1 addition & 1 deletion packages/golang/manifest.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
format_version: 2.0.0
name: golang
title: Golang
version: "0.4.0"
version: "0.4.1"
description: This Elastic integration collects metrics from Golang applications.
type: integration
categories:
Expand Down

0 comments on commit fe8cb5c

Please sign in to comment.