SFTP Append doesn't work [?]

[IgnatBeresnev]

I've been trying to append text to the end of the file with OpenMode.APPEND, but no luck so far. It seems to ignore this flag and just uses the offset. If I set the offsets (both file & off) to 0, then it'll rewrite the file. How do I go about appending text to the end of the file WITHOUT offsets and ONLY with OpenMode.APPEND, if it's possible?

Here's how I'm trying to do it

String fileName = "test_write_continuously.txt";
String filePath = HOME_FOLDER + "/" + fileName;

String initialText = "This is initial text of the file. ";
byte[] bytes = initialText.getBytes();

RemoteFile writeInitial = sftpClient.open(filePath, new HashSet<>(Arrays.asList(OpenMode.WRITE, OpenMode.CREAT)));
writeInitial.write(0, bytes, 0, bytes.length);
writeInitial.close();

String appendText = "This is the continuation text that should be appended after initial. ";
byte[] appendTextBytes = appendText.getBytes();

RemoteFile writeAppend = sftpClient.open(filePath, new HashSet<>(Arrays.asList(OpenMode.WRITE, OpenMode.APPEND)));
writeAppend.write(0, appendTextBytes, 0, appendTextBytes.length);
writeAppend.close();

RemoteFile readContent = sftpClient.open(filePath, new HashSet<>(Arrays.asList(OpenMode.READ)));
byte[] readBytes = new byte[bytes.length + appendTextBytes.length];
readContent.read(0, readBytes, 0, readBytes.length);
String readString = new String(readBytes);
readContent.close();

assertEquals(initialText + appendText, readString);

What I want to achieve: continuously write to the end of the file, pretty much real-time logging on a remote server through ssh, and I need the data to be consistent and in order. You think it's possible to achieve? So far I've been having file rewrites

[IgnatBeresnev]

Maybe the problem is that we shouldn't put file offset if APPEND is specified?

return requester.request(newRequest(PacketType.WRITE)
        .putUInt64(fileOffset)
        .putUInt32(len)
        .putRawBytes(data, off, len)
);
``

[hierynomus]

According to the specifications, the fileOffset field is required and cannot be omitted. When a file is opened with SSH_FXF_APPEND, the sftp server should ignore the offset flag and append to the file.

Which SFTP server are you using?

[hierynomus]

I've added an integration test in 0f67fa2 which shows that it should work with your exact scenario. This is as far as I can see really a bug in your SFTP server.

[IgnatBeresnev]

You seem to be right. The problem is reproduced only on the server that uses OpenSSH 5.3. However, another server with OpenSSH 7.4 is passing tests just fine.

Thanks. Didn't even think that could be a problem

[hierynomus]

Thanks for confirming! Good to know that OpenSSH had a bug ;).