Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Filebeat network shares support #19736

Closed
1 of 2 tasks
kvch opened this issue Jul 8, 2020 · 5 comments
Closed
1 of 2 tasks

Filebeat network shares support #19736

kvch opened this issue Jul 8, 2020 · 5 comments
Labels
enhancement Filebeat Filebeat Stalled Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team

Comments

@kvch
Copy link
Contributor

kvch commented Jul 8, 2020

The log input of Filebeat does not support reading files from network shares and cloud providers. There are two main issues that have to be addressed before we are ready.

  • Identifying files Identity tracking of files in Filebeat inputs #13492
    Filebeat identifies files based on inode and device ids. However, these two numbers might change from time to time when the files are located on a network share. Thus, Filebeat cannot tell apart files correctly. Users might end up with duplicated data as Filebeat believes it has found a new file and rereads whole content.
  • Reading from files occasionally return null
    The offset returned from file readers are sometimes incorrect. Filebeat needs to check if the return bytes sequence is nil. If yes, it needs to revert back to the end of the file, as it has tried to read from an invalid area. A possible solution is described in this comment: Add initial support for configurable file identity tracking #18748 (comment)
@kvch kvch added enhancement Filebeat Filebeat [zube]: Meta Team:Services (Deprecated) Label for the former Integrations-Services team labels Jul 8, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@kvch
Copy link
Contributor Author

kvch commented Jul 8, 2020

CC @urso

@botelastic
Copy link

botelastic bot commented Jun 8, 2021

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@botelastic botelastic bot added the Stalled label Jun 8, 2021
@botelastic botelastic bot closed this as completed Jul 8, 2021
@zube zube bot added [zube]: Done and removed [zube]: Meta labels Jul 8, 2021
@zube zube bot removed the [zube]: Done label Oct 7, 2021
@kvch kvch added Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team and removed Team:Services (Deprecated) Label for the former Integrations-Services team labels Apr 6, 2022
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@kvch kvch reopened this Apr 6, 2022
@botelastic botelastic bot removed the Stalled label Apr 6, 2022
@botelastic
Copy link

botelastic bot commented Apr 6, 2023

Hi!
We just realized that we haven't looked into this issue in a while. We're sorry!

We're labeling this issue as Stale to make it hit our filters and make sure we get back to it as soon as possible. In the meantime, it'd be extremely helpful if you could take a look at it as well and confirm its relevance. A simple comment with a nice emoji will be enough :+1.
Thank you for your contribution!

@botelastic botelastic bot added the Stalled label Apr 6, 2023
@botelastic botelastic bot closed this as completed Oct 3, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Filebeat Filebeat Stalled Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

No branches or pull requests

2 participants