Deleting TDB2 location on Windows

[luigi-asprino]

Version

4.7.0

What happened?

Hi

I'm using Windows 10 to address this issue (SPARQL-Anything/sparql.anything#266 (comment)). Specifically, we are persisting data on a TDB2 and we want to delete the folder once the computation has been completed. However, deleting the folder fails because there are still files used by the program even when the TDB is explicitly closed.

I've created a unit test to reproduce the exception.

@Test
	public void playWithTDBLoc() throws IOException {
		String testFolder = "tmp/testTDB";
		Dataset d1 = TDB2Factory.connectDataset(testFolder);
		Txn.executeWrite(d1, ()->{
			RDFDataMgr.read(d1, "https://www.w3.org/1999/02/22-rdf-syntax-ns#");
		});

		Txn.executeRead(d1, ()->{
			RDFDataMgr.write(System.out, d1, Lang.TRIG);
		});
		d1.close();
		FileUtils.deleteDirectory(new File(testFolder));
	}

Am I missing something?
Is there anything I can do to explicitly release the resources?

Note This happens only on Windows.

Relevant output and stacktrace

org.apache.commons.io.IOExceptionList: 1 exceptions: [java.io.IOException: Unable to delete file: tmp\testTDB\Data-0001]

	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:345)
	at org.apache.commons.io.FileUtils.deleteDirectory(FileUtils.java:1206)
	at io.github.sparqlanything.it.IssuesTest.playWithTDBLoc(IssuesTest.java:643)
	at java.base/jdk.internal.reflect.DirectMethodHandleAccessor.invoke(DirectMethodHandleAccessor.java:103)
	at java.base/java.lang.reflect.Method.invoke(Method.java:580)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
	at com.intellij.junit4.JUnit4IdeaTestRunner.startRunnerWithArgs(JUnit4IdeaTestRunner.java:69)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater$1.execute(IdeaTestRunner.java:38)
	at com.intellij.rt.execution.junit.TestsRepeater.repeat(TestsRepeater.java:11)
	at com.intellij.rt.junit.IdeaTestRunner$Repeater.startRunnerWithArgs(IdeaTestRunner.java:35)
	at com.intellij.rt.junit.JUnitStarter.prepareStreamsAndStart(JUnitStarter.java:232)
	at com.intellij.rt.junit.JUnitStarter.main(JUnitStarter.java:55)
Caused by: java.io.IOException: Unable to delete file: tmp\testTDB\Data-0001
	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1425)
	at org.apache.commons.io.FileUtils.cleanDirectory(FileUtils.java:338)
	... 27 more
Caused by: java.nio.file.FileSystemException: tmp\testTDB\Data-0001\GOSP.bpt: Impossibile accedere al file. Il file è utilizzato da un altro processo
	at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
	at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
	at java.base/sun.nio.fs.WindowsFileSystemProvider.implDelete(WindowsFileSystemProvider.java:273)
	at java.base/sun.nio.fs.AbstractFileSystemProvider.deleteIfExists(AbstractFileSystemProvider.java:109)
	at java.base/java.nio.file.Files.deleteIfExists(Files.java:1191)
	at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:142)
	at org.apache.commons.io.file.DeletingPathVisitor.visitFile(DeletingPathVisitor.java:36)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2786)
	at java.base/java.nio.file.Files.walkFileTree(Files.java:2857)
	at org.apache.commons.io.file.PathUtils.visitFileTree(PathUtils.java:687)
	at org.apache.commons.io.file.PathUtils.deleteDirectory(PathUtils.java:328)
	at org.apache.commons.io.file.PathUtils.delete(PathUtils.java:303)
	at org.apache.commons.io.file.PathUtils.delete(PathUtils.java:280)
	at org.apache.commons.io.FileUtils.forceDelete(FileUtils.java:1423)
	... 28 more

Are you interested in making a pull request?

None

[rvesse]

I think this is the longstanding JDK bug - https://bugs.openjdk.org/browse/JDK-4715154 - that affects memory mapped files on Windows

Terminating the JVM is the only way to allow these files to be deleted.

You might want to use Files.createTempDirectory() rather than hardcoding your temporary data location, that way the OS will put the test dataset into its temporary files area and those will naturally get cleaned up by the OS over time anyway. Then you don't need to worry about explicitly deleting the files in your tests.

[luigi-asprino]

Hi,
thanks for the hints. We need to explicitly delete the TDB and recreate a new one in a given location. I will try to come up with a workaround.

[luigi-asprino]

This is definitely not a Jena bug.

[luigi-asprino]

Btw a simple workaround would be clearing the TDB instead of deleting it.