[O11y][System] Rally benchmark `system.memory` #9296

ali786XI · 2024-03-07T08:20:26Z

Proposed commit message

This PR adds benchmarking templates to the memory data stream of System

Sample Response

sample_event.json

{
"@timestamp": "2024-03-12T09:03:56.592Z",
"agent": {
  "ephemeral_id": "22ed892c-43bd-408a-9121-65e2f5b6a56e",
  "id": "de42127b-4db8-4471-824e-a7b14f478663",
  "name": "system-scale-123456",
  "type": "metricbeat",
  "version": "8.8.0"
},
"benchmark_metadata": {
  "info": {
    "benchmark": "memory-benchmark",
    "run_id": "4fcdf3ec-7f80-4ec2-9e5f-bc6515e5caf9"
  }
},
"data_stream": {
  "dataset": "system.memory",
  "namespace": "ep",
  "type": "metrics"
},
"ecs": {
  "version": "8.0.0"
},
"elastic_agent": {
  "id": "404fc805-04b8-420a-9fcf-12d54b89d4fd",
  "snapshot": false,
  "version": "8.12.1"
},
"event": {
  "agent_id_status": "auth_metadata_missing",
  "dataset": "system.memory",
  "duration": 64088,
  "ingested": "2024-03-12T09:04:01Z",
  "module": "system"
},
"host": {
  "architecture": "x86_64",
  "containerized": true,
  "hostname": "docker-fleet-agent",
  "id": "96bd755dd266474fb86bd80a98cbaad5",
  "ip": "192.168.241.7",
  "mac": "02-42-C0-A8-F1-07",
  "name": "docker-fleet-agent",
  "os": {
    "codename": "focal",
    "family": "debian",
    "kernel": "3.10.0-1160.105.1.el7.x86_64",
    "name": "Ubuntu",
    "platform": "ubuntu",
    "type": "linux",
    "version": "20.04.6 LTS (Focal Fossa)"
  }
},
"metricset": {
  "name": "memory",
  "period": 10000
},
"service": {
  "type": "system"
},
"system": {
  "memory": {
    "actual": {
      "free": 4010742085,
      "used": {
        "bytes": 6897315189,
        "pct": 0.632
      }
    },
    "cached": 1193489350,
    "free": 2942436459,
    "swap": {
      "free": 3308840821,
      "total": 3665190130,
      "used": {
        "bytes": 356349309,
        "pct": 0.097
      }
    },
    "total": 11876868165,
    "used": {
      "bytes": 8934431706,
      "pct": 0.752
    }
  }
}
}

Checklist

I have reviewed tips for building integrations and this pull request is aligned with them.
I have verified that all data streams collect metrics or logs.

How to test this PR locally

Run this command from package root

elastic-package benchmark rally --benchmark memory-benchmark -v
elastic-package benchmark stream --benchmark memory-benchmark -v

Related issues

Relates [Meta] Add more benchmark schema templates to integrations #8636

Screenshots

--- Benchmark results for package: system - START ---
╭───────────────────────────────────────────────────────────────────────────────────╮
│ info                                                                              │
├────────────────────────┬──────────────────────────────────────────────────────────┤
│ benchmark              │                                         memory-benchmark │
│ description            │            Benchmark 20000 system.memory events ingested │
│ run ID                 │                     1fac31bf-a339-4b68-b253-15a761c5689f │
│ package                │                                                   system │
│ start ts (s)           │                                               1709799509 │
│ end ts (s)             │                                               1709799555 │
│ duration               │                                                      46s │
│ generated corpora file │ /root/.elastic-package/tmp/rally_corpus/corpus-227991648 │
╰────────────────────────┴──────────────────────────────────────────────────────────╯
╭──────────────────────────────────────────────────────────────────────╮
│ parameters                                                           │
├─────────────────────────────────┬────────────────────────────────────┤
│ package version                 │                             1.54.0 │
│ data_stream.name                │                             memory │
│ corpora.generator.total_events  │                              20000 │
│ corpora.generator.template.path │ ./memory-benchmark/template.ndjson │
│ corpora.generator.template.raw  │                                    │
│ corpora.generator.template.type │                             gotext │
│ corpora.generator.config.path   │      ./memory-benchmark/config.yml │
│ corpora.generator.config.raw    │                              map[] │
│ corpora.generator.fields.path   │      ./memory-benchmark/fields.yml │
│ corpora.generator.fields.raw    │                              map[] │
╰─────────────────────────────────┴────────────────────────────────────╯
╭───────────────────────╮
│ cluster info          │
├───────┬───────────────┤
│ name  │ elasticsearch │
│ nodes │             1 │
╰───────┴───────────────╯
╭───────────────────────────────────────╮
│ disk usage for index metrics-system.m │
│ emory-ep (for all fields)             │
├──────────────────────────────┬────────┤
│ total                        │ 5.0 MB │
│ inverted_index.total         │ 144 kB │
│ inverted_index.stored_fields │ 3.5 MB │
│ inverted_index.doc_values    │ 1.0 MB │
│ inverted_index.points        │ 342 kB │
│ inverted_index.norms         │    0 B │
│ inverted_index.term_vectors  │    0 B │
│ inverted_index.knn_vectors   │    0 B │
╰──────────────────────────────┴────────╯
╭─────────────────────────────────────────────────────────────────────────────────────────────╮
│ rally stats                                                                                 │
├────────────────────────────────────────────────────────────────┬────────────────────────────┤
│ Cumulative indexing time of primary shards                     │     0.5400833333333334 min │
│ Min cumulative indexing time across primary shards             │                      0 min │
│ Median cumulative indexing time across primary shards          │ 0.00038333333333333334 min │
│ Max cumulative indexing time across primary shards             │    0.25589999999999996 min │
│ Cumulative indexing throttle time of primary shards            │                      0 min │
│ Min cumulative indexing throttle time across primary shards    │                      0 min │
│ Median cumulative indexing throttle time across primary shards │                      0 min │
│ Max cumulative indexing throttle time across primary shards    │                      0 min │
│ Cumulative merge time of primary shards                        │                 0.0125 min │
│ Cumulative merge count of primary shards                       │                         23 │
│ Min cumulative merge time across primary shards                │                      0 min │
│ Median cumulative merge time across primary shards             │                      0 min │
│ Max cumulative merge time across primary shards                │                 0.0027 min │
│ Cumulative merge throttle time of primary shards               │                      0 min │
│ Min cumulative merge throttle time across primary shards       │                      0 min │
│ Median cumulative merge throttle time across primary shards    │                      0 min │
│ Max cumulative merge throttle time across primary shards       │                      0 min │
│ Cumulative refresh time of primary shards                      │    0.13541666666666666 min │
│ Cumulative refresh count of primary shards                     │                       1288 │
│ Min cumulative refresh time across primary shards              │                      0 min │
│ Median cumulative refresh time across primary shards           │ 0.00016666666666666666 min │
│ Max cumulative refresh time across primary shards              │                 0.0321 min │
│ Cumulative flush time of primary shards                        │     0.2996166666666667 min │
│ Cumulative flush count of primary shards                       │                        835 │
│ Min cumulative flush time across primary shards                │ 1.6666666666666667e-05 min │
│ Median cumulative flush time across primary shards             │  6.666666666666667e-05 min │
│ Max cumulative flush time across primary shards                │    0.04763333333333333 min │
│ Total Young Gen GC time                                        │                    0.031 s │
│ Total Young Gen GC count                                       │                          2 │
│ Total Old Gen GC time                                          │                        0 s │
│ Total Old Gen GC count                                         │                          0 │
│ Store size                                                     │    0.011841018684208393 GB │
│ Translog size                                                  │   0.0002580946311354637 GB │
│ Heap used for segments                                         │                       0 MB │
│ Heap used for doc values                                       │                       0 MB │
│ Heap used for terms                                            │                       0 MB │
│ Heap used for norms                                            │                       0 MB │
│ Heap used for points                                           │                       0 MB │
│ Heap used for stored fields                                    │                       0 MB │
│ Segment count                                                  │                        197 │
│ Total Ingest Pipeline count                                    │                      20003 │
│ Total Ingest Pipeline time                                     │                    0.884 s │
│ Total Ingest Pipeline failed                                   │                          0 │
│ Min Throughput                                                 │            34209.78 docs/s │
│ Mean Throughput                                                │            34209.78 docs/s │
│ Median Throughput                                              │            34209.78 docs/s │
│ Max Throughput                                                 │            34209.78 docs/s │
│ 50th percentile latency                                        │       555.3588673938066 ms │
│ 100th percentile latency                                       │       855.5337698198855 ms │
│ 50th percentile service time                                   │       555.3588673938066 ms │
│ 100th percentile service time                                  │       855.5337698198855 ms │
│ error rate                                                     │                     0.00 % │
╰────────────────────────────────────────────────────────────────┴────────────────────────────╯

--- Benchmark results for package: system - END   ---
Done

elasticmachine · 2024-03-07T08:36:17Z

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

niraj-elastic · 2024-03-12T06:43:03Z

@aliabbas-elastic Please add sample_event.json.

ali786XI · 2024-03-12T09:06:06Z

@aliabbas-elastic Please add sample_event.json.

Done. Please have a look

niraj-elastic

LGTM!

shmsr · 2024-03-12T09:55:51Z

packages/system/_dev/benchmark/rally/memory-benchmark/config.yml

+      min: 8000000000
+      max: 9000000000


How these ranges contribute to the benchmark? If I tune these parameters, will the result change?

These ranges just provide you the liberty to set fields with realistic values. Like for example here the values are correlated with each other. If you see the system_memory_actual_used_bytes ranges compared to this they would be lesser which will be the case in real environments as well. Modifying these wouldn't change the benchmarking stats.

shmsr · 2024-03-12T09:56:02Z

packages/system/_dev/benchmark/rally/memory-benchmark/config.yml

@@ -0,0 +1,35 @@
+fields:
+  - name: timestamp
+    period: -60m


Why - is prefixed?

This is in reference to this doc
https://github.com/elastic/elastic-integration-corpus-generator-tool/blob/main/docs/fields-configuration.md#:~:text=generator%20will%20stop.-,period,-optional%20(date
It would always generate documents before time.Now() so that the events would always be available in Last duration in Discover

… into system_benchmark_memory

elasticmachine · 2024-03-13T13:33:48Z

💚 Build Succeeded

Buildkite Build
Commit: dd3838b

History

💚 Build #9441 succeeded 1e55592
💚 Build #9401 succeeded 38be540

cc @aliabbas-elastic

elastic-sonarqube · 2024-03-13T13:33:50Z

Quality Gate passed

Kudos, no new issues were introduced!

0 New issues
0 Security Hotspots
No data about Coverage
No data about Duplication

See analysis details on SonarQube

rally benchmark system.memory

38be540

ali786XI added enhancement New feature or request Integration:system System labels Mar 7, 2024

ali786XI self-assigned this Mar 7, 2024

ali786XI mentioned this pull request Mar 7, 2024

[Meta] Add more benchmark schema templates to integrations #8636

Closed

29 tasks

fix some configs

1e55592

ali786XI marked this pull request as ready for review March 8, 2024 07:15

ali786XI requested a review from a team as a code owner March 8, 2024 07:15

ali786XI requested review from tommyers-elastic and niraj-elastic March 8, 2024 07:15

niraj-elastic approved these changes Mar 12, 2024

View reviewed changes

shmsr approved these changes Mar 12, 2024

View reviewed changes

shmsr approved these changes Mar 13, 2024

View reviewed changes

Merge branch 'main' of https://github.com/aliabbas-elastic/integrations…

dd3838b

… into system_benchmark_memory

ali786XI merged commit 01a436f into elastic:main Mar 25, 2024
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[O11y][System] Rally benchmark `system.memory` #9296

[O11y][System] Rally benchmark `system.memory` #9296

ali786XI commented Mar 7, 2024 •

edited

Loading

elasticmachine commented Mar 7, 2024 •

edited

Loading

niraj-elastic commented Mar 12, 2024

ali786XI commented Mar 12, 2024

niraj-elastic left a comment

shmsr Mar 12, 2024

ali786XI Mar 12, 2024

shmsr Mar 13, 2024

shmsr Mar 12, 2024

ali786XI Mar 12, 2024 •

edited

Loading

shmsr Mar 13, 2024

elasticmachine commented Mar 13, 2024

elastic-sonarqube bot commented Mar 13, 2024

[O11y][System] Rally benchmark system.memory #9296

[O11y][System] Rally benchmark system.memory #9296

Conversation

ali786XI commented Mar 7, 2024 • edited Loading

Proposed commit message

Sample Response

Checklist

How to test this PR locally

Related issues

Screenshots

elasticmachine commented Mar 7, 2024 • edited Loading

🚀 Benchmarks report

niraj-elastic commented Mar 12, 2024

ali786XI commented Mar 12, 2024

niraj-elastic left a comment

Choose a reason for hiding this comment

shmsr Mar 12, 2024

Choose a reason for hiding this comment

ali786XI Mar 12, 2024

Choose a reason for hiding this comment

shmsr Mar 13, 2024

Choose a reason for hiding this comment

shmsr Mar 12, 2024

Choose a reason for hiding this comment

ali786XI Mar 12, 2024 • edited Loading

Choose a reason for hiding this comment

shmsr Mar 13, 2024

Choose a reason for hiding this comment

elasticmachine commented Mar 13, 2024

💚 Build Succeeded

History

elastic-sonarqube bot commented Mar 13, 2024

Quality Gate passed

[O11y][System] Rally benchmark `system.memory` #9296

[O11y][System] Rally benchmark `system.memory` #9296

ali786XI commented Mar 7, 2024 •

edited

Loading

elasticmachine commented Mar 7, 2024 •

edited

Loading

ali786XI Mar 12, 2024 •

edited

Loading