Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Filebeat hangs if it tries to open a FIFO with no writer #18682

Closed
benbuzbee opened this issue May 21, 2020 · 2 comments · Fixed by #20450
Closed

Filebeat hangs if it tries to open a FIFO with no writer #18682

benbuzbee opened this issue May 21, 2020 · 2 comments · Fixed by #20450
Assignees
@kvch
Labels
Team:Services (Deprecated) Label for the former Integrations-Services team

Comments

@benbuzbee
Copy link

benbuzbee commented May 21, 2020 •
edited
Loading

This line tries to open a file on the main input thread

beats/filebeat/input/log/harvester.go

Line 508 in b0f7ae7

f, err := file_helper.ReadOpen(h.state.Source)

If the file is a Unix FIFO, the open will hang until there is a writer. Since this is the main input thread, it means that no further files that may match the input pattern will be processed.

Suggestion 1:
On linux open with O_NONBLOCK
From http://man7.org/linux/man-pages/man7/fifo.7.html

A process can open a FIFO in nonblocking mode.  In this case, opening
       for read-only succeeds even if no one has opened on the write side
       yet

Suggestion 2:
Skip fifos

Suggestion 3:
Open on a background thread
@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 21, 2020
@benbuzbee benbuzbee changed the title Filebeat log input not re-scanning at scan_duration Filebeat hangs if it tries to open a FIFO with no writer May 21, 2020
@benbuzbee
Copy link
Author

A quick, way to repro is probably (untested)

  • Setup a glob pattern for /dir/*, scan frequency 1s
  • test that echo a > /dir/newfile gets picked up
  • mkfifo /dir/fifo
  • wait for filebeat to scan fifo
  • test that echo b > /dir/othernewfile does NOT get picked up

@andresrc andresrc added the Team:Services (Deprecated) Label for the former Integrations-Services team label May 22, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/integrations-services (Team:Services)

@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 22, 2020
@kvch kvch mentioned this issue Aug 5, 2020
Merged
4 tasks
kvch added a commit to kvch/beats that referenced this issue Aug 6, 2020
@kvch
Check if Filebeat harvester tries to open named pipe
62d932b 
Closes elastic#18682
kvch added a commit that referenced this issue Aug 6, 2020
@kvch
Check if Filebeat log harvester tries to open named pipe (#20450)
a89d81f 
## What does this PR do?

This PR adds a check before opening a file for harvester Filebeat. If the file is a named pipe, an error is returned and the file is not opened.

## Why is it important?

Previously if someone wanted to open a named pipe without a writer, Filebeat hangs.

## Related issues

Closes #18682
melchiormoulin pushed a commit to melchiormoulin/beats that referenced this issue Oct 14, 2020
@kvch @melchiormoulin
Check if Filebeat log harvester tries to open named pipe (elastic#20450)
9e96a54 
## What does this PR do?

This PR adds a check before opening a file for harvester Filebeat. If the file is a named pipe, an error is returned and the file is not opened.

## Why is it important?

Previously if someone wanted to open a named pipe without a writer, Filebeat hangs.

## Related issues

Closes elastic#18682
kvch added a commit to kvch/beats that referenced this issue Nov 4, 2020
@kvch
Check if Filebeat log harvester tries to open named pipe (elastic#20450)
b4f183c 
## What does this PR do?

This PR adds a check before opening a file for harvester Filebeat. If the file is a named pipe, an error is returned and the file is not opened.

## Why is it important?

Previously if someone wanted to open a named pipe without a writer, Filebeat hangs.

## Related issues

Closes elastic#18682

(cherry picked from commit a89d81f)
@kvch kvch mentioned this issue Nov 4, 2020
Merged
4 tasks
kvch added a commit to kvch/beats that referenced this issue Nov 4, 2020
@kvch
Check if Filebeat log harvester tries to open named pipe (elastic#20450)
b8ef2a8 
## What does this PR do?

This PR adds a check before opening a file for harvester Filebeat. If the file is a named pipe, an error is returned and the file is not opened.

## Why is it important?

Previously if someone wanted to open a named pipe without a writer, Filebeat hangs.

## Related issues

Closes elastic#18682

(cherry picked from commit a89d81f)
@kvch kvch mentioned this issue Nov 4, 2020
Merged
4 tasks
@zube zube bot removed the [zube]: Done label Nov 5, 2020
kvch added a commit that referenced this issue Nov 6, 2020
@kvch
Check if Filebeat log harvester tries to open named pipe (#20450) (#2…
4eb9989 
…2422)

## What does this PR do?

This PR adds a check before opening a file for harvester Filebeat. If the file is a named pipe, an error is returned and the file is not opened.

## Why is it important?

Previously if someone wanted to open a named pipe without a writer, Filebeat hangs.

## Related issues

Closes #18682

(cherry picked from commit a89d81f)
kvch added a commit that referenced this issue Nov 6, 2020
@kvch
Check if Filebeat log harvester tries to open named pipe (#20450) (#2…
8d5318e 
…2423)

## What does this PR do?

This PR adds a check before opening a file for harvester Filebeat. If the file is a named pipe, an error is returned and the file is not opened.

## Why is it important?

Previously if someone wanted to open a named pipe without a writer, Filebeat hangs.

## Related issues

Closes #18682

(cherry picked from commit a89d81f)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kvch kvch

Labels
Team:Services (Deprecated) Label for the former Integrations-Services team
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Check if Filebeat log harvester tries to open named pipe kvch/beats
4 participants
@benbuzbee @andresrc @kvch @elasticmachine