"Fatal Error: 'Unable to clear dir'"

[nigelp]

Wow, way to go to make paying customers jump through a million hoops to file a bug report on your paid products Jason. Not awesome.

Anyway, I'm getting a weird one with Quick Cache Pro. It's fairly benign (in that it doesn't corrupt or delete anything), but it is annoying.

1. Action: Post or edit a post on the site
2. Press the Publish or Save As Pending Button
3. Expected behaviour - saved and return to page
   Accual (intermittent) behaviour - see attached image pops up.

The problem doesn't happen all the time, maybe 20% of the time.

Thanks.

Nigel P.

[raamdev]

Wow, way to go to make paying customers jump through a million hoops to file a bug report on your paid products Jason. Not awesome.

I'm not sure what "million hoops" you're referring to here. The Support link to open a trouble ticket for paying customers is clearly visible on our website inside the "More" menu. You're also able to file bug reports here on GitHub or even post questions in the community forum over at WordPress.org. All of those places are monitored by us.

Anyway, I'm getting a weird one with Quick Cache Pro. It's fairly benign (in that it doesn't corrupt or delete anything), but it is annoying.

I've been trying to reproduce this bug in my test installation but have been entirely unsuccessful. Could you please provide me with a few more details about your setup?

• WordPress version
• Quick Cache Pro version
• PHP version
• Web host (optional, of course, but helpful for ruling out web hosts with setups that are known to sometimes cause issues)

[nigelp]

Hi Raam,

What I meant by hoops is the need to sign up for a Github account, which is
simply a bridge too far for most people. It's only because I already had an
account that I actually continued. It's unnecessary.

Anyway to your questions:

WordPress - 3.9.2
Quick Cache Pro - Version 140725
PHP - 5.3.10
Host - Medialayer.com

Cheers.

Nigel

On Wed, Aug 27, 2014 at 7:38 PM, Raam Dev [email protected] wrote:

Wow, way to go to make paying customers jump through a million hoops to
file a bug report on your paid products Jason. Not awesome.

I'm not sure what "million hoops" you're referring to here. The Support
link to open a trouble ticket https://www.websharks-inc.com/support/
for paying customers is clearly visible on our website inside the "More"
menu. You're also able to file bug reports here on GitHub or even post
questions in the community forum over at WordPress.org
http://wordpress.org/support/plugin/quick-cache. All of those places
are monitored by us.

Anyway, I'm getting a weird one with Quick Cache Pro. It's fairly benign
(in that it doesn't corrupt or delete anything), but it is annoying.

I've been trying to reproduce this bug in my test installation but have
been entirely unsuccessful. Could you please provide me with a few more
details about your setup?

• WordPress version
• Quick Cache Pro version
• PHP version
• Web host (optional, of course, but helpful for ruling out web
  hosts with setups that are known to sometimes cause issues)

—
Reply to this email directly or view it on GitHub
#288 (comment)
.

[raamdev]

Hi Nigel,

I'm still unable to reproduce this problem, so I have no way of troubleshooting it. You may want to contact your web host to see if there's any automated cleanup script that runs on the server that might be clearing out files from the cache directory. It's unlikely, but worth checking anyway.

Another thing you might want to do is add define('WP_DEBUG_DISPLAY', false); to your wp-config.php file. That will prevent those messages appearing on the site.

If you still want to monitor those errors, I recommend setting define('WP_DEBUG_LOG', true); in your wp-config.php file. That will Enable Debug logging to the wp-content/debug.log file, which you can then periodically download and inspect for errors. That's a much cleaner way of handling this and it's the recommended configuration for production website. For further details, please see http://codex.wordpress.org/Debugging_in_WordPress

I'll look into a better way of handling a scenario where Quick Cache encounters this error, so that it can be handled more gracefully.

I see you're running v140725. I recommend upgrading to v140829, released yesterday, and watching for this issue again. If you continue to see this issue with the latest version, please let me know by updating this issue.

[nigelp]

Hi Raam,

Yes unfortunately I'm still seeing the issue even after updating. I'm also
noticing that one of my social network widgets in my sidebar is also now
playing up (it's stopped showing Twitter and YouTube counts), so I'm
wondering whether your new version has messed up our widget caching as well?

Cheers.

Nigel

On Sat, Aug 30, 2014 at 5:05 PM, Raam Dev [email protected] wrote:

Hi Nigel,

I'm still unable to reproduce this problem, so I have no way of
troubleshooting it. You may want to contact your web host to see if there's
any automated cleanup script that runs on the server that might be clearing
out files from the cache directory. It's unlikely, but worth checking
anyway.

Another thing you might want to do is add define('WP_DEBUG_DISPLAY',
false); to your wp-config.php file. That will prevent those messages
appearing on the site.

If you still want to monitor those errors, I recommend setting define('WP_DEBUG_LOG',
true); in your wp-config.php file. That will Enable Debug logging to the
wp-content/debug.log file, which you can then periodically download and
inspect for errors. That's a much cleaner way of handling this and it's the
recommended configuration for production website. For further details,
please see http://codex.wordpress.org/Debugging_in_WordPress

I'll look into a better way of handling a scenario where Quick Cache
encounters this error, so that it can be handled more gracefully.

I see you're running v140725. I recommend upgrading to v140829, released
yesterday, and watching for this issue again. If you continue to see this
issue with the latest version, please let me know by updating this issue.

—
Reply to this email directly or view it on GitHub
#288 (comment)
.

[jaswrks]

I too, have been unable to reproduce this issue so far.

I've seen it reported by a couple of site owners (i.e. it seems to occur rarely). When it's reported it has been extremely difficult to pinpoint. The exception that QC throws here, is to alert a site owner about its inability to clear the cache as expected. The warning that preceeds the exception thrown by QC, which reads "directory not empty", suggests that QC is missing a particular file inside of this directory whenever it scans the file system. What file or sub-directory that is, remains a mystery to me.

@nigelp You could help us to narrow this down by taking a look inside of the directory when this error occurs (if it occurs again). When you inspect this directory, what is left inside of the directory that PHP is complaining about? That would really help us find the underlying cause of this.

In the error message you posted earlier, it was the /index.q directory that was not empty, and thus PHP was unable to delete the directory. A directory must be empty before the file system will allow it to be deleted. This is what resulted in the error message to begin with. What we need to determine is why that directory was not empty.

We would expect it to be empty here. In the sub-routine where this occurs (in the code), QC has just finished emptying this directory. That's why it is quite rare and somewhat surprising to see this message. Making it difficult to reproduce.

[raamdev]

Yes unfortunately I'm still seeing the issue even after updating. I'm also
noticing that one of my social network widgets in my sidebar is also now
playing up (it's stopped showing Twitter and YouTube counts), so I'm
wondering whether your new version has messed up our widget caching as well?

If your widgets use JavaScript to update, Quick Cache wouldn't be caching those. If, however, they don't use JavaScript to update themselves, then they might be getting cached with the rest of the page. Since you seem to indicate they updated fine prior to the last update, I would suspect they use JavaScript and therefore remain unaffected by Quick Cache.

• Have you tried clearing your cache and testing to see if they update as expected?
• What plugin(s) you're using for the social network widgets?

[nigelp]

Hiya,

Well it just happened again. So I'm attaching -

a) The error message again.
b) The list of files still remaining in the index.q folder. I did a screen
grab of each page of files, hence the 4 files. Just in case there's some
sort of corrupted file in there.

Hope that helps. :)

Nigel

On Tue, Sep 2, 2014 at 3:48 PM, JasWSInc [email protected] wrote:

[image: cacheerror]
https://cloud.githubusercontent.com/assets/97901/4058544/64a9f07a-2dd7-11e4-8bd6-f5940e5d4d45.jpg

I too, have been unable to reproduce this issue so far.

I've seen it reported by a couple of site owners (i.e. it seems to occur
rarely). When it's reported it has been extremely difficult to pinpoint.
The exception that QC throws here, is to alert a site owner about its
inability to clear the cache as expected. The warning that preceeds the
exception thrown by QC, which reads "directory not empty", suggests that QC
is missing a particular file inside of this directory whenever it scans the
file system. What file or sub-directory that is, remains a mystery to me.

@nigelp https://github.com/nigelp You could help us to narrow this down
by taking a look inside of the directory when this error occurs (if it
occurs again). When you inspect this directory, what is left inside of the
directory that PHP is complaining about? That would really help us find the
underlying cause of this.

In the error message you posted earlier, it was the /index.q directory
that was not empty, and thus PHP was unable to delete the directory. A
directory must be empty before the file system will allow it to be deleted.
This is what resulted in the error message to begin with. What we need to
determine is why that directory was not empty.

We would expect it to be empty here. In the sub-routine where this occurs
(in the code), QC has just finished emptying this directory. That's why it
is quite rare and somewhat surprising to see this message. Making it
difficult to reproduce.

—
Reply to this email directly or view it on GitHub
#288 (comment)
.

[nigelp]

Hi Raam,

The widget problem seems to have gone away currently.

Nigel

On Tue, Sep 2, 2014 at 7:16 PM, Raam Dev [email protected] wrote:

Yes unfortunately I'm still seeing the issue even after updating. I'm also
noticing that one of my social network widgets in my sidebar is also now
playing up (it's stopped showing Twitter and YouTube counts), so I'm
wondering whether your new version has messed up our widget caching as
well?

If your widgets use JavaScript to update, Quick Cache wouldn't be caching
those. If, however, they don't use JavaScript to update themselves, then
they might be getting cached with the rest of the page. Since you seem to
indicate they updated fine prior to the last update, I would suspect they
use JavaScript and therefore remain unaffected by Quick Cache.

• Have you tried clearing your cache and testing to see if they update
  as expected?
• What plugin(s) you're using for the social network widgets?

—
Reply to this email directly or view it on GitHub
#288 (comment)
.

[raamdev]

Well it just happened again. So I'm attaching -

a) The error message again.
b) The list of files still remaining in the index.q folder. I did a screen
grab of each page of files, hence the 4 files. Just in case there's some
sort of corrupted file in there.

I didn't receive any of these attachments. Note that GitHub doesn't accept attachments via email. You need to login to the web interface and drag-n-drop images/files.

[nigelp]

Sorry, here you go.

[raamdev]

@nigelp Thank you for the screenshots. I assume you connected via FTP and took those screenshots immediately after receiving that error message, correct?

That's a lot more files than I expected to see. I just ran a test with query-string cache file (index.q/999c8064d0815d88b25c2f2889968e1c.html) and Quick Cache purged the file as expected when I published the post.

@jaswsinc I'm wondering if maybe @nigelp's site is so busy that by the time Quick Cache has finished cleaning out the index.q/ directory and attempts to delete it, new visitor requests have caused Quick Cache to generate new files in this directory, so that when Quick Cache attempts to delete it an error about the directory not being empty results. What do you think?

If that's what's happening here, we could probably do an atomic operation where we rename the index.q/ directory before cleaning it out and deleting it, so that if new requests are made in the interim, they don't affect that operation.

[jaswrks]

@raamdev I think that you are absolutely right! Even if we can't be 100% sure that it's the issue here in this particular report, what you just pointed out is still an important issue in my view. Something we could definitely improve.

I also think some refactoring to consolidate some of the routines we have now (which are spread out a little more than they need to be) would be helpful too; i.e. we could try to consolidate some of our file system operations as much as possible, so that we can focus on tuning things like this in.

Given our current workflow, where we create feature branches for each issue, I think it would be really helpful to identify and/or create some issues that are focused on refactoring portions of the codebase. Making it easy for one of us to come in and work on that sort of issue too. In short, issues that are not bug reports, not feature requests, but ideas focused on ways to improve the structure of the codebase to make new things possible, and easier to maintain.

[nigelp]

Hi Raam,

Yes they were taken immediately afterwards using Filezilla.

One thing I should mention is I use both Cloudflare and Google PageSpeed on
the site as well as QC. Not sure if that's relevant, but thought you ought
to know.

Nigel

On Tue, Sep 9, 2014 at 1:50 AM, Raam Dev [email protected] wrote:

@nigelp https://github.com/nigelp Thank you for the screenshots. I
assume you connected via FTP and took those screenshots immediately after
receiving that error message, correct?

That's a lot more files than I expected to see. I just ran a test with
query-string cache file (index.q/999c8064d0815d88b25c2f2889968e1c.html)
and Quick Cache purged the file as expected when I published the post.

@jaswsinc https://github.com/jaswsinc I'm wondering if maybe @nigelp
https://github.com/nigelp's site is so busy that by the time Quick
Cache has finished cleaning out the index.q/ directory and attempts to
delete it, new visitor requests have caused Quick Cache to generate new
files in this directory, so that when Quick Cache attempts to delete it an
error about the directory not being empty results. What do you think?

If that's what's happening here, we could probably do an atomic operation
where we rename the index.q/ directory before cleaning it out and
deleting it, so that if new requests are made in the interim, they don't
affect that operation.

—
Reply to this email directly or view it on GitHub
#288 (comment)
.

[raamdev]

FYI, I'm marking this as a bug so that we can track it as such. There are several related GitHub issues open referencing this bug, all of which will need to be implemented before we can confirm the fix and close this issue (see #316, #317, #318, #319).

Bug description

At this time, we believe this bug occurs on busy sites where Quick Cache is purging a directory and then attempting to delete it, but is unable to delete the directory because new cache files are being created faster than Quick Cache can purge them, thereby resulting in an error messaging reporting that the directory is not empty.

We have a plan for a fix and this bug will be a priority in the next development cycle.

[jaswrks]

@raamdev I can work on this if you like.

[raamdev]

I'm reopening this issue, as there's obviously still some work that needs to be done here.

[eurobank]

I also have this problem, a lot of tmp leftovers. And i don't have a big site, 1800 blog posts and 100 visits per day.

[GermontZ]

hello ! Maybe the content of my error log will help you...

[13-Dec-2014 13:12:08 UTC] PHP Fatal error: Uncaught exception 'RuntimeException' with message 'SplFileInfo::getMTime() [splfileinfo.getmtime]: stat failed for /home/mysite/public_html/continut/a/cache/http/www-mysite-ro-548c3b28bf324494887728-tmp/fck/editor/dialog/fck-about-html.v/desktop.html' in /home/mysite/public_html/continut/plugins/quick-cache-pro/includes/share.php:1671
Stack trace:
#0 /home/mysite/public_html/continut/plugins/quick-cache-pro/includes/share.php(1671): SplFileInfo->getMTime()
#1 /home/mysite/public_html/continut/plugins/quick-cache-pro/includes/share.php(1462): quick_cache\share->delete_files_from_host_cache_dir('/^.+/i', true)
#2 /home/mysite/public_html/continut/plugins/quick-cache-pro/quick-cache-pro.inc.php(1291): quick_cache\share->purge_files_from_host_cache_dir('/^.+/i')
#3 [internal function]: quick_cache\plugin->purge_cache()
#4 /home/mysite/public_html/wp-includes/plugin.php(580): call_user_func_array(Array, Array)
#5 /home/mysite/public_html/wp-cron.php(103): do_a in /home/mysite/public_html/continut/plugins/quick-cache-pro/includes/share.php on line 1671

and some info about the server installation:

Apache Version 2.4.7
PHP Version 5.4.25
MySQL Version 5.5.40-cll
Architecture x86_64
Operating System linux
Dedicated IP Address secret ;)
Perl Version 5.10.1
Kernel Version 2.6.32-458.23.2.lve1.2.52.el6.x86_64
cPanel Pro 1.0 (RC1)

[raamdev]

Next release changelog:

• Bug Fix: Addressed another issue related to "Fatal Error: 'Unable to clear dir'" and tmp directories that don't get cleared by Quick Cache. This fix discards iteration references before renaming the tmp directories. Props @jaswsinc. See #288.

[raamdev]

The latest fix will go out with the next release.

If you're interested in testing a beta release of Quick Cache before the next version comes out, please sign-up to be a beta tester here.

[raamdev]

@mchlbrry @eurobank @Alergic Quick Cache v141229 (Release Candidate) has been released that includes fixes for the issues you were having. If you could test this Release Candidate to confirm the issue has been fixed for you, that would be great! Please see http://www.websharks-inc.com/post/quick-cache-v141229-release-candidate/

[eurobank]

Installed it and run an aggresive spider software against my website.

After 2+ hours i see NO leftovers anymore.

[GermontZ]

I installed this RC, rebuilt all cache, and cleared manually. The cache folder contained just 'https' folder, but empty. No tmp and no other folders.

Error log was not modified anymore today, not even after this RC update.

[raamdev]

@eurobank writes...

After 2+ hours i see NO leftovers anymore.

@Alergic writes...

No tmp and no other folders.

Woohoo! That's great to hear! Thank you very much for confirming the fix worked as expected. :)

cc @jaswsinc

[jaswrks]

I installed this RC, rebuilt all cache, and cleared manually. The cache folder contained just 'https' folder, but empty. No tmp and no other folders.

@Alergic Great to hear!

[mchlbrry]

Ive had QC running for a few days and unfortunately im still getting the persistent tmp directories.

Here's what I did after I updating the plugin a couple of times:

• Backed up QC settings
• 'Restored' QC in the WP admin
• Wiped wp-content/cache off the server
• Enabled QC and imported settings

Is there something in my server config that could be causing the issue? Thanks for taking time to address this, i'm sure you've got other bigger issues you'd rather be tackling.

[raamdev]

@mchlbrry Could please you try enabling WordPress debugging by adding the following lines to your wp-config.php file then, after you see new tmp directories not getting cleared, check the wp-content/debug.log file for any errors that might be related to this issue?

define('WP_DEBUG', true); // enables debugging
define('WP_DEBUG_LOG', true); // logs messages to wp-content/debug.log
define('WP_DEBUG_DISPLAY', false); // hides debug messages from showing in the browser

See also: https://codex.wordpress.org/Debugging_in_WordPress

[mchlbrry]

@raamdev, here are the errors I can see:

[08-Jan-2015 21:17:03 UTC] PHP Fatal error:  Uncaught exception 'RuntimeException' with message 'SplFileInfo::getMTime(): stat failed for /var/www/html/wordpress/wp-content/cache/quick-cache/cache/http/www-domain-com-54aef3cf69cc8928126844-tmp/album-of-the-year-2013.html' in /var/www/html/wordpress/wp-content/plugins/quick-cache-pro/includes/share.php:1675
Stack trace:
#0 /var/www/html/wordpress/wp-content/plugins/quick-cache-pro/includes/share.php(1675): SplFileInfo->getMTime()
#1 /var/www/html/wordpress/wp-content/plugins/quick-cache-pro/includes/share.php(1462): quick_cache\share->delete_files_from_host_cache_dir('/^.+/i', true)
#2 /var/www/html/wordpress/wp-content/plugins/quick-cache-pro/quick-cache-pro.inc.php(1291): quick_cache\share->purge_files_from_host_cache_dir('/^.+/i')
#3 [internal function]: quick_cache\plugin->purge_cache()
#4 /var/www/html/wordpress/wp-includes/plugin.php(580): call_user_func_array(Array, Array)
#5 /var/www/html/wordpress/wp-cron.php(103): do_action_ref_array('_cron_quick_cac...', Array)
#6 {main}
  thrown in /var/www/html/wordpress/wp-content/plugins/quick-cache-pro/includes/share.php on line 1675

Edit: removed site URL and WP directory path

[raamdev]

@mchlbrry Thank you for the errors. I've opened a new Issue (#397 Temp Files in Cache Directory Fail to Clear) to continue tracking this bug, as this issue is getting a bit long and it's getting hard to follow relevant information. I've copied your error report over there.

Please follow Issue #397 for further updates.

[raamdev]

Quick Cache v150121 (Release Candidate) was released yesterday that includes further fixes for this issue. If you'd like to help test the Release Candidate, that would be greatly appreciated! Please see http://www.websharks-inc.com/post/quick-cache-v150121-release-candidate/

---

Please report future any updates to this issue ("Fatal Error: 'Unable to clear dir'") on Issue #397.