Intermittent start failure due to "Text file busy"

[wilkinsona]

In Spring Boot CI builds (which run on Linux), we occasionally see Embedded Mongo 2.0.3 fail to start due to a text file being busy:

Caused by: java.io.IOException: Cannot run program "/tmp/extract-e66d8762-5929-4f3f-994a-00c185df3844extractmongod": error=26, Text file busy
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1048)
	at de.flapdoodle.embed.process.runtime.ProcessControl.start(ProcessControl.java:205)
	at de.flapdoodle.embed.process.runtime.AbstractProcess.<init>(AbstractProcess.java:99)
	at de.flapdoodle.embed.mongo.AbstractMongoProcess.<init>(AbstractMongoProcess.java:53)
	at de.flapdoodle.embed.mongo.MongodProcess.<init>(MongodProcess.java:50)
	at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:44)
	at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:34)
	at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:108)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeCustomInitMethod(AbstractAutowireCapableBeanFactory.java:1827)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1770)
	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1698)
	... 36 more
Caused by: java.io.IOException: error=26, Text file busy
	at java.lang.UNIXProcess.forkAndExec(Native Method)
	at java.lang.UNIXProcess.<init>(UNIXProcess.java:247)
	at java.lang.ProcessImpl.start(ProcessImpl.java:134)
	at java.lang.ProcessBuilder.start(ProcessBuilder.java:1029)
	... 50 more

Here's one example of the failure from which I took the above stack trace.

I wonder if there's a stream or writer that's not being closed or flushed when the file's being written so that it can subsequently be busy when an attempt is made to execute it?

[Loki-Afro]

@wilkinsona as far as I know "Text file busy" is triggered when another process does access the file, but I could be wrong here.

is there a way to do an lsof on that file when the issue occurs?

[wilkinsona]

I can certainly try to get some lsof output when the problem next occurs. I'll update this issue if I am able to do so. That said, I would be surprised if there's another process accessing the file.

Having done a bit more research, I wonder if this could be a symptom of this JDK bug? If so, perhaps it would be appropriate to retry?

[Loki-Afro]

interesting JDK bug, you could try out the script mentioned there in your environment since you are the first who is facing this issue it could be very environment specific ...

[wilkinsona]

As far as I can tell, the problem in the JDK bug may be specific to Linux but nothing more than that. I've just reproduced it in two different Linux VMs but have been unable to reproduce it on macOS.

[Loki-Afro]

and only open jdk seems to be affected, I can't reproduce it on my mac either and not on linux (4.16.3) with an oracle jdk ...