Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tail plugin still tails previously deleted files #3965

Closed
d0o0 opened this issue Apr 3, 2018 · 8 comments
Closed

Tail plugin still tails previously deleted files #3965

d0o0 opened this issue Apr 3, 2018 · 8 comments
Labels
area/tail bug unexpected problem or unintended behavior

Comments

@d0o0
Copy link

d0o0 commented Apr 3, 2018

hi,
my log files are under path /data/logs/ and named as %Month-%Day.log
and i use telegraf logparser to monitor on /data/logs/**.log
everyday i will remove the files of yesterday.

when i use lsof and i find sooo many files deleted but still monitored by telegraf.
and i chack tail.go, can not find any way to close the opened file.

so, what should i do ?

Bug report

Relevant telegraf.conf:

System info:

[Include Telegraf version, operating system name, and other relevant details]

Steps to reproduce:

  1. ...
  2. ...

Expected behavior:

Actual behavior:

Additional info:

[Include gist of relevant config, logs, etc.]

@danielnelson danielnelson added bug unexpected problem or unintended behavior area/tail labels Apr 3, 2018
@danielnelson
Copy link
Contributor

@d0o0 Can you edit in the info in the bug report template, I added the form to your original post.

@dugwood
Copy link

dugwood commented Apr 22, 2018

Just FYI: if you run a service telegraf reload, telegraf will let go all those deleted files.

Update: actually I just hit #3573 so it's not a good solution... unless you use from_beginning = true.

@davidgarciaMontreal
Copy link

davidgarciaMontreal commented Aug 7, 2020

I'm having the same problem. I stopped telegraf and restarted and the tail plugin started tailing those files that were previously deleted.
Telegraf 1.12.3 (git: HEAD 16784bc)

telegraf  23027 26939 telegraf  987r      REG              253,4       111    263224 /var/log/xxx/2020/08/05/.db_telegraf_1596629490_183272.json.RHYqPi (deleted)
telegraf  23027 26939 telegraf  988r      REG              253,4       335    263222 /var/log/xxx/2020/08/05/.db_telegraf_1596622296_270476.json.bVNB30 (deleted)
telegraf  23027 26939 telegraf  989r      REG              253,4       111    263225 /var/log/xxx/2020/08/05/.db_telegraf_1596633092_402469.json.YTwmIr (deleted)
telegraf  23027 26939 telegraf  990r      REG              253,4       111    263218 /var/log/xxx/2020/08/05/.db_telegraf_1596607838_351340.json.7b54vr (deleted)
telegraf  23027 26939 telegraf  991r      REG              253,4       111    263216 /var/log/xxx/2020/08/05/.db_telegraf_1596600633_544686.json.zghnK9 (deleted)
telegraf  23027 26939 telegraf  992r      REG              253,4       222    263213 /var/log/xxx/2020/08/05/.db_telegraf_1596589834_399533.json.mnRT5I (deleted)
telegraf  23027 26939 telegraf  993r      REG              253,4       224    263230 /var/log/xxx/2020/08/05/.db_telegraf_1596651099_157346.json.1gxa99 (deleted)
telegraf  23027 26939 telegraf  994r      REG              253,4       111    263220 /var/log/xxx/2020/08/05/.db_telegraf_1596615034_797587.json.rufPhJ (deleted)
telegraf  23027 26939 telegraf  995r      REG              253,4       224    263229 /var/log/xxx/2020/08/05/.db_telegraf_1596647495_874988.json.kPobg1 (deleted)
telegraf  23027 26939 telegraf  996r      REG              253,4       224    263234 /var/log/xxx/2020/08/05/.db_telegraf_1596665501_440017.json.FNedHJ (deleted)
telegraf  23027 26939 telegraf  997r      REG              253,4         0    263219 /var/log/xxx/2020/08/05/.db_telegraf_1596611438_390929.json.mOQWoA (deleted)
telegraf  23027 26939 telegraf  998r      REG              253,4       445    263228 /var/log/xxx/2020/08/05/.db_telegraf_1596643907_595240.json.rzXcnS (deleted)
telegraf  23027 26939 telegraf  999r      REG              253,4       335    263223 /var/log/xxx/2020/08/05/.db_telegraf_1596625898_792400.json.qg9vW9 (deleted)
telegraf  23027 26939 telegraf 1000r      REG              253,4       113    263231 /var/log/xxx/2020/08/05/.db_telegraf_1596654702_359294.json.sL891i (deleted)
telegraf  23027 26939 telegraf 1001r      REG              253,4       224    263233 /var/log/xxx/2020/08/05/.db_telegraf_1596661899_775658.json.ojubOA (deleted)
telegraf  23027 26939 telegraf 1002r      REG              253,4       223    263235 /var/log/xxx/2020/08/05/.db_telegraf_1596669105_553583.json.eqKfAS (deleted)
telegraf  23027 26939 telegraf 1003r      REG              253,4       223    263237 /var/log/xxx/2020/08/06/.db_telegraf_1596694304_897222.json.IG9mma (deleted)
telegraf  23027 26939 telegraf 1004r      REG              253,4       561    263247 /var/log/xxx/2020/08/06/.db_telegraf_1596730349_094865.json.Cy8BfB (deleted)
telegraf  23027 26939 telegraf 1005r      REG              253,4       113    263243 /var/log/xxx/2020/08/06/.db_telegraf_1596715938_282536.json.k6QYG1 (deleted)
telegraf  23027 26939 telegraf 1006r      REG              253,4       337    263254 /var/log/xxx/2020/08/06/.db_telegraf_1596755556_882000.json.HF88tB (deleted)
telegraf  23027 26939 telegraf 1007r      REG              253,4       450    263251 /var/log/xxx/2020/08/06/.db_telegraf_1596744760_589425.json.qiZpOa (deleted)
telegraf  23027 26939 telegraf 1008r      REG              253,4       113    263244 /var/log/xxx/2020/08/06/.db_telegraf_1596719534_102595.json.D5L7za (deleted)

@sjwang90
Copy link
Contributor

@davidgarciaMontreal Do you mind filling out the following bug template:

Bug report

Relevant telegraf.conf:

System info:

[Include Telegraf version, operating system name, and other relevant details]

Steps to reproduce:

  1. ...
  2. ...

Expected behavior:

Actual behavior:

Additional info:

[Include gist of relevant config, logs, etc.]

@sjwang90 sjwang90 changed the title [ fd of deleted files can not be removed] telegraf cannot remove fd of deleted files Tail plugin still tailing previously deleted files Oct 20, 2020
@sjwang90 sjwang90 changed the title Tail plugin still tailing previously deleted files Tail plugin still tails previously deleted files Oct 20, 2020
@davidgarciaMontreal
Copy link

Hello @sjwang90 ,
I have already created a ticket with the same information: #7959
Let me know if you need anything else from me.
BR David

@sjwang90
Copy link
Contributor

Thanks! Closing in preference for #7959

@Philipzhong1980
Copy link

we meet the same issue and when will fix this bug?

@sjwang90
Copy link
Contributor

sjwang90 commented Jan 4, 2021

Please follow the issue #7959 for status of the issue fix.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area/tail bug unexpected problem or unintended behavior
Projects
None yet
Development

No branches or pull requests

6 participants