Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Rename and map azure-eventhub input fields #20981

Closed
wants to merge 20 commits into from
Closed

Rename and map azure-eventhub input fields #20981

wants to merge 20 commits into from

Conversation

narph
Copy link
Contributor

@narph narph commented Sep 4, 2020

What does this PR do?

Renames and map azure-eventhub input fields

Why is it important?

Rename and map azure-eventhub input fields, preparation for migration.

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

Related issues

narph added 16 commits July 23, 2020 11:01
@narph
mofidy doc
938e66c
@narph
Merge branch 'master' of github.com:elastic/beats
4daef08
@narph
Merge branch 'master' of github.com:elastic/beats
b613178
@narph
Merge branch 'master' of github.com:elastic/beats
05364cf
@narph
Merge branch 'master' of github.com:elastic/beats
f147c4d
@narph
Merge branch 'master' of github.com:elastic/beats
4574718
@narph
Merge branch 'master' of github.com:elastic/beats
1e43077
@narph
Merge branch 'master' of github.com:elastic/beats
807cf06
@narph
Merge branch 'master' of github.com:elastic/beats
2096668
@narph
Merge branch 'master' of github.com:elastic/beats
da8ac1f
@narph
Merge branch 'master' of github.com:elastic/beats
c2d8930
@narph
Merge branch 'master' of github.com:elastic/beats
7bd9e73
@narph
Merge branch 'master' of github.com:elastic/beats
6e89a84
@narph
Merge branch 'master' of github.com:elastic/beats
bdf21e9
@narph
Merge branch 'master' of github.com:elastic/beats
7833687
@narph
map azure-eventhub fields
87c3c81
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label Sep 4, 2020
@narph narph self-assigned this Sep 4, 2020
@narph narph added [zube]: In Review needs_backport PR is waiting to be backported to other branches. Team:Platforms Label for the Integrations - Platforms team labels Sep 4, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-platforms (Team:Platforms)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label Sep 4, 2020
@elasticmachine
Copy link
Collaborator

elasticmachine commented Sep 4, 2020 •
edited by jenkins-beats-ci bot
Loading

💔 Tests Failed

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2021-10-06T15:13:06.273+0000

  • Duration: 130 min 8 sec

  • Commit: 0b086b3

Test stats 🧪

Test Results
Failed 6
Passed 6287
Skipped 661
Total 6954

Test errors 6

Expand to view the tests failures

Build&Test / x-pack/filebeat-unitTest / test_export_index_pattern – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
    Expand to view the error details

     Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

    Expand to view the stacktrace

     self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern>

    def test_export_index_pattern(self):
        """
        Test that the index-pattern can be exported with `export index-pattern`
        """
>       output = self.run_export_cmd("index-pattern")

../../libbeat/tests/system/beat/common_tests.py:61: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern>
cmd = 'index-pattern', extra = []

    def run_export_cmd(self, cmd, extra=[]):
        """
        Runs the given export command and returns the output as a string.
        Raises an exception if the command fails.
        :param cmd: the export command
        :param extra: Extra arguments (optional)
        :return: The output as a string.
        """
        self.render_config_template()
    
        args = ["export", cmd]
        if len(extra) != 0:
            args += extra
        exit_code = self.run_beat(extra_args=args, logging_args=[])
        output = self.get_log()
        if exit_code != 0:
>           raise Exception("export command returned with an error: {}".format(output))
E           Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

../../libbeat/tests/system/beat/common_tests.py:32: Exception

Build&Test / x-pack/filebeat-unitTest / test_export_index_pattern_migration – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
    Expand to view the error details

     Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

    Expand to view the stacktrace

     self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern_migration>

    def test_export_index_pattern_migration(self):
        """
        Test that the index-pattern can be exported with `export index-pattern` (migration enabled)
        """
>       output = self.run_export_cmd("index-pattern", extra=['-E', 'migration.6_to_7.enabled=true'])

../../libbeat/tests/system/beat/common_tests.py:74: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern_migration>
cmd = 'index-pattern', extra = ['-E', 'migration.6_to_7.enabled=true']

    def run_export_cmd(self, cmd, extra=[]):
        """
        Runs the given export command and returns the output as a string.
        Raises an exception if the command fails.
        :param cmd: the export command
        :param extra: Extra arguments (optional)
        :return: The output as a string.
        """
        self.render_config_template()
    
        args = ["export", cmd]
        if len(extra) != 0:
            args += extra
        exit_code = self.run_beat(extra_args=args, logging_args=[])
        output = self.get_log()
        if exit_code != 0:
>           raise Exception("export command returned with an error: {}".format(output))
E           Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

../../libbeat/tests/system/beat/common_tests.py:32: Exception

Build&Test / x-pack/filebeat-windows-windows-2019 / test_export_index_pattern – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
    Expand to view the error details

     Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

    Expand to view the stacktrace

     self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern>

    def test_export_index_pattern(self):
        """
        Test that the index-pattern can be exported with `export index-pattern`
        """
>       output = self.run_export_cmd("index-pattern")

..\..\libbeat\tests\system\beat\common_tests.py:61: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern>
cmd = 'index-pattern', extra = []

    def run_export_cmd(self, cmd, extra=[]):
        """
        Runs the given export command and returns the output as a string.
        Raises an exception if the command fails.
        :param cmd: the export command
        :param extra: Extra arguments (optional)
        :return: The output as a string.
        """
        self.render_config_template()
    
        args = ["export", cmd]
        if len(extra) != 0:
            args += extra
        exit_code = self.run_beat(extra_args=args, logging_args=[])
        output = self.get_log()
        if exit_code != 0:
>           raise Exception("export command returned with an error: {}".format(output))
E           Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

..\..\libbeat\tests\system\beat\common_tests.py:32: Exception

Build&Test / x-pack/filebeat-windows-windows-2019 / test_export_index_pattern_migration – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
    Expand to view the error details

     Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

    Expand to view the stacktrace

     self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern_migration>

    def test_export_index_pattern_migration(self):
        """
        Test that the index-pattern can be exported with `export index-pattern` (migration enabled)
        """
>       output = self.run_export_cmd("index-pattern", extra=['-E', 'migration.6_to_7.enabled=true'])

..\..\libbeat\tests\system\beat\common_tests.py:74: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern_migration>
cmd = 'index-pattern', extra = ['-E', 'migration.6_to_7.enabled=true']

    def run_export_cmd(self, cmd, extra=[]):
        """
        Runs the given export command and returns the output as a string.
        Raises an exception if the command fails.
        :param cmd: the export command
        :param extra: Extra arguments (optional)
        :return: The output as a string.
        """
        self.render_config_template()
    
        args = ["export", cmd]
        if len(extra) != 0:
            args += extra
        exit_code = self.run_beat(extra_args=args, logging_args=[])
        output = self.get_log()
        if exit_code != 0:
>           raise Exception("export command returned with an error: {}".format(output))
E           Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

..\..\libbeat\tests\system\beat\common_tests.py:32: Exception

Build&Test / x-pack/filebeat-pythonIntegTest / test_export_index_pattern – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
    Expand to view the error details

     Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

    Expand to view the stacktrace

     self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern>

    def test_export_index_pattern(self):
        """
        Test that the index-pattern can be exported with `export index-pattern`
        """
>       output = self.run_export_cmd("index-pattern")

../../libbeat/tests/system/beat/common_tests.py:61: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern>
cmd = 'index-pattern', extra = []

    def run_export_cmd(self, cmd, extra=[]):
        """
        Runs the given export command and returns the output as a string.
        Raises an exception if the command fails.
        :param cmd: the export command
        :param extra: Extra arguments (optional)
        :return: The output as a string.
        """
        self.render_config_template()
    
        args = ["export", cmd]
        if len(extra) != 0:
            args += extra
        exit_code = self.run_beat(extra_args=args, logging_args=[])
        output = self.get_log()
        if exit_code != 0:
>           raise Exception("export command returned with an error: {}".format(output))
E           Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

../../libbeat/tests/system/beat/common_tests.py:32: Exception

Build&Test / x-pack/filebeat-pythonIntegTest / test_export_index_pattern_migration – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
    Expand to view the error details

     Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

    Expand to view the stacktrace

     self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern_migration>

    def test_export_index_pattern_migration(self):
        """
        Test that the index-pattern can be exported with `export index-pattern` (migration enabled)
        """
>       output = self.run_export_cmd("index-pattern", extra=['-E', 'migration.6_to_7.enabled=true'])

../../libbeat/tests/system/beat/common_tests.py:74: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

self = <test_filebeat_xpack.FilebeatXPackTest testMethod=test_export_index_pattern_migration>
cmd = 'index-pattern', extra = ['-E', 'migration.6_to_7.enabled=true']

    def run_export_cmd(self, cmd, extra=[]):
        """
        Runs the given export command and returns the output as a string.
        Raises an exception if the command fails.
        :param cmd: the export command
        :param extra: Extra arguments (optional)
        :return: The output as a string.
        """
        self.render_config_template()
    
        args = ["export", cmd]
        if len(extra) != 0:
            args += extra
        exit_code = self.run_beat(extra_args=args, logging_args=[])
        output = self.get_log()
        if exit_code != 0:
>           raise Exception("export command returned with an error: {}".format(output))
E           Exception: export command returned with an error: Error generating Index Pattern: field <azure-eventhub.offset> is duplicated, remove it or set 'overwrite: true', {Name:offset Type:long Description:The position of an event within a partition. Format: Fields:[] MultiFields:[] Enabled:<nil> Analyzer: SearchAnalyzer: Norms:false Dynamic:{Value:<nil>} Index:<nil> DocValues:<nil> CopyTo: IgnoreAbove:0 AliasPath: MigrationAlias:false Dimension:<nil> DynamicTemplate:false Unit: MetricType: ObjectType: ObjectTypeMappingType: ScalingFactor:0 ObjectTypeParams:[] Analyzed:<nil> Count:0 Searchable:<nil> Aggregatable:<nil> Script: Pattern: InputFormat: OutputFormat: OutputPrecision:<nil> LabelTemplate: UrlTemplate:[] OpenLinkInCurrentTab:<nil> Overwrite:false DefaultField:<nil> Path:azure-eventhub.offset}, {"aggregatable":true,"analyzed":false,"count":0,"doc_values":true,"indexed":true,"name":"azure-eventhub.offset","scripted":false,"searchable":true,"type":"number"}.

../../libbeat/tests/system/beat/common_tests.py:32: Exception

Steps errors 11

Expand to view the steps failures

Show only the first 10 steps failures

x-pack/filebeat-unitTest - mage build unitTest
  • Took 4 min 36 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-unitTest - mage build unitTest
  • Took 3 min 2 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-pythonIntegTest - mage pythonIntegTest
  • Took 33 min 30 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/filebeat-pythonIntegTest - mage pythonIntegTest
  • Took 30 min 14 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/filebeat-pythonIntegTest - mage pythonIntegTest
  • Took 31 min 50 sec . View more details here
  • Description: mage pythonIntegTest
x-pack/filebeat-windows-windows-2019 - mage build unitTest
  • Took 7 min 47 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-windows-windows-2019 - mage build unitTest
  • Took 4 min 0 sec . View more details here
  • Description: mage build unitTest
x-pack/filebeat-windows-windows-2019 - mage build unitTest
  • Took 4 min 47 sec . View more details here
  • Description: mage build unitTest
gsutil -m -q cp -a public-read build/system-tests-*.tar.gz gs://beats-ci-temp/Beats/beats/PR-20981-2
  • Took 0 min 3 sec . View more details here
  • Description: @echo off gsutil -m -q cp -a public-read build/system-tests-*.tar.gz gs://beats-ci-temp/Beats/beats/PR-20981-2
Error signal
  • Took 0 min 0 sec . View more details here
  • Description: Error 'hudson.AbortException: script returned exit code 1'

🐛 Flaky test report

❕ There are test failures but not known flaky tests.

Expand to view the summary

Genuine test errors 6

💔 There are test failures but not known flaky tests, most likely a genuine test failure.

  • Name: Build&Test / x-pack/filebeat-unitTest / test_export_index_pattern – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
  • Name: Build&Test / x-pack/filebeat-unitTest / test_export_index_pattern_migration – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
  • Name: Build&Test / x-pack/filebeat-windows-windows-2019 / test_export_index_pattern – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
  • Name: Build&Test / x-pack/filebeat-windows-windows-2019 / test_export_index_pattern_migration – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_export_index_pattern – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest
  • Name: Build&Test / x-pack/filebeat-pythonIntegTest / test_export_index_pattern_migration – x-pack.filebeat.tests.system.test_filebeat_xpack.FilebeatXPackTest

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

jsoriano
jsoriano approved these changes Sep 4, 2020
View reviewed changes
Copy link
Member

@jsoriano jsoriano left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mapping fixes LGTM, but I think that long-term we should use azure.eventhub instead of azure-eventhub.

x-pack/filebeat/input/azureeventhub/_meta/fields.yml
description: >
Fields regarding the Azure event hub information.
fields:
- name: azure-eventhub
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should use underscore to combine words, but in any case I think these metrics should be stored under azure.eventhub instead of azure-eventhub. But changing this would be a breaking change (unless we duplicate fields), we would have to decide when to do it. Most likely in 8.0.

@jsoriano
Copy link
Member

jsoriano commented Sep 4, 2020

Failures in CI seem related.

@mergify
Copy link
Contributor

mergify bot commented Apr 7, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b azure-input-fields upstream/azure-input-fields
git merge upstream/master
git push upstream azure-input-fields

@mergify
Copy link
Contributor

mergify bot commented Sep 22, 2021

This pull request does not have a backport label. Could you fix it @narph? 🙏
To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

  • backport-v./d./d./d is the label to automatically backport to the 7./d branch. /d is the digit

NOTE: backport-skip has been added to this pull request.

@mergify mergify bot added the backport-skip Skip notification from the automated backport with mergify label Sep 22, 2021
@narph narph mentioned this pull request Oct 6, 2021
Closed
19 tasks
@mergify
Copy link
Contributor

mergify bot commented Oct 11, 2021

This pull request is now in conflicts. Could you fix it? 🙏
To fixup this pull request, you can check out it locally. See documentation: https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch upstream
git checkout -b azure-input-fields upstream/azure-input-fields
git merge upstream/master
git push upstream azure-input-fields

@narph
Copy link
Contributor Author

narph commented Nov 17, 2021

decided on no renaming

@narph narph closed this Nov 17, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@jsoriano jsoriano jsoriano approved these changes

Assignees

@narph narph

Labels
backport-skip Skip notification from the automated backport with mergify needs_backport PR is waiting to be backported to other branches. Team:Platforms Label for the Integrations - Platforms team [zube]: In Review
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@narph @elasticmachine @jsoriano