FileDescriptorCast: Allow casting AF_INET/AF_INET6 socket file descriptors to Socket API classes

[xueqiwang0v0]

Describe the bug
More tutorials on how to cast fd to socket. I got ClassCastException when casting. Are there any other restrictions?

To Reproduce
Steps to reproduce the behavior:
platform: Mac Arm MacOS 13.5.1
env: Java11
lib:

        <dependency>
            <groupId>com.kohlschutter.junixsocket</groupId>
            <artifactId>junixsocket-core</artifactId>
            <version>2.8.3</version>
            <type>pom</type>
        </dependency>
        <dependency>
            <groupId>com.kohlschutter.junixsocket</groupId>
            <artifactId>junixsocket-server</artifactId>
            <version>2.8.3</version>
        </dependency>
        <dependency>
            <groupId>com.kohlschutter.junixsocket</groupId>
            <artifactId>junixsocket-darwin</artifactId>
            <version>2.8.3</version>
        </dependency>

I‘m trying to transfer a tcp connection between two processes. To do this, I create a tcp socket in process1 and send the file descriptor of the tcp socket to process2. Then I try to rebuild the tcp socket in process2. I usedFileDescriptorCast.using(fd).as(Socket.class) and it throws ClassCastException.

I notice that there are only few global types are available for this fd, am I missing something? I follow the instruction here: https://kohlschutter.github.io/junixsocket/filedescriptors.html
The available types I get: [class java.lang.Number, class java.io.OutputStream, interface java.lang.Comparable, class java.io.FileInputStream, interface java.nio.channels.Channel, interface java.nio.channels.InterruptibleChannel, interface java.io.Serializable, interface java.io.Closeable, interface java.io.Flushable, interface java.nio.channels.ScatteringByteChannel, class java.lang.Integer, class java.io.FileDescriptor, interface java.lang.AutoCloseable, class java.lang.ProcessBuilder$Redirect, interface java.nio.channels.GatheringByteChannel, interface java.nio.channels.ReadableByteChannel, class java.nio.channels.spi.AbstractInterruptibleChannel, class java.lang.Object, class java.nio.channels.FileChannel, interface java.nio.channels.ByteChannel, class java.io.InputStream, interface java.nio.channels.WritableByteChannel, class java.io.FileOutputStream, interface java.nio.channels.SeekableByteChannel]

Expected behavior
FileDescriptorCast successfully cast the fd to socket.

Output/Screenshots
When I called the FileDescriptorCast.using(fd).as(Socket.class);, I got

java.lang.ClassCastException: Cannot access file descriptor as class java.net.Socket
	at org.newsclub.net.unix.FileDescriptorCast.as(FileDescriptorCast.java:445)

Any help would be appreciated!

[kohlschuetter]

Thanks for reporting, @xueqiwang0v0!

I think it's due to the fact that it's likely an AF_INET or AF_INET6 socket (you mention TCP), which junixsocket currently doesn't provide specific support for.

In that sense, junixsocket "works as expected", but obviously it's not working for you.

I see three possible solutions:

1. provide hooks into Java SDK internals, essentially constructing native SocketImpl instances given a FileDescriptor. This is a moving target, really difficult to maintain. If there was a public API to use, junixsocket would immediately adopt it.
2. providing a custom implementation for AF_INET/AF_INET6 (unclear if there are benefits over the vanilla Java implementation; maybe specific Linux-only features, etc.?).
3. have a AFGenericSocket for all "other" sockets that are not explicitly supported, but then we would also have to provide an AFGenericSocketAddress (which most likely wouldn't be cross-platform compatible), add support for generic socket options, etc. The problem here is that it's not only a significant amount of upfront work but quite hard to ensure everything works for any kind of socket, so you would end up with something that's not super useful per se but would occasionally fail to do the right thing for exotic socket types.

I can see the utility for ensuring compatibility with apps expecting a Socket instance, so option 3 may happen eventually.

A workaround that works right now would be to "cast" the FileDescriptor to ByteChannel or FileInputStream/FileOutputStream instead, so you can at least communicate bidirectionally.

Any other ideas?

[xueqiwang0v0]

Thanks for your reply, @kohlschuetter !

Sorry for my misusage about junixsocket library. It offers great help to my project! So many thanks again!

I'm currently working on option 1. I achieved rebuilding Socket using reflection, while ServerSocket rebuild is still unsolved. For option 3, will there be a rough plan?

[kohlschuetter]

There's no plan yet for option 3, but it may eventually arrive.

I will keep this ticket open for now to be reminded of it. Contributions are of course welcome!

[kohlschuetter]

@xueqiwang0v0
I've made some progress towards supporting your use case, but I haven't fully tested it yet.
Please try the latest 2.9.0-SNAPSHOT.

Make sure the following section is in your POM:

            <repositories>
                <repository>
                    <id>snapshots-repo</id>
                    <url>https://oss.sonatype.org/content/repositories/snapshots</url>
                    <releases>
                        <enabled>false</enabled>
                    </releases>
                    <snapshots>
                        <enabled>true</enabled>
                    </snapshots>
                </repository>
            </repositories>

Then run mvn clean compile --update-snapshots (or similar) from your project.

If you use Gradle, please specify the following section in your build.gradle file:

repositories {
    maven {
        url "https://oss.sonatype.org/content/repositories/snapshots"
        mavenContent {
            snapshotsOnly()
        }
    }
}

Then run gradle clean build --refresh-dependencies (or similar) from your project.

[xueqiwang0v0]

@kohlschuetter Thanks for your help!
I tested the new version 2.9.0-SNAPSHOT using 2 cases:

1. Case 1: transfer a tcp connection.
   I sent the tcp socket fd to another process and then rebuilt the connection using FileDescriptorCast.
   The new version works well. The rebuilt tcp connection received messages and replied as expected.

2. Case 2: transfer a listening tcp server.
   I sent the serverSocket fd to another process and then rebuilt it using FileDescriptorCast. Then I tried to establish a new tcp connection and communicate.
   There's something going wrong. The rebuilt server did not block waiting for the new connection. As a result, it throwed null pointer exception when I tried to get the inputStream and outputStream of the new connection.
   My Core Rebuild Steps:

// rebuild the server socket
ServerSocket serverSocket = FileDescriptorCast.using(serverFD).as(ServerSocket.Class);

// wait for client connection
// The method should block until a connection is made.
Socket connSocket = serverSocket.accept();
log.info("New client connected.");

// communicate
BufferedReader input = new BuferredReader(new InputStreamReader(connSocket.getInputStream()));
PrintWriter newClientOutput = new PrintWriter(connSocket.getOutputStream(), true);

[kohlschuetter]

@xueqiwang0v0
Cool, that's good news! (regarding case 1).

Regarding case 2, I think you found another bug that we should fix.
Is there a chance that your socket was configured non-blocking in native code?

I think that calling configureBlocking(true) is currently not working as expected since there is some Java Socket code that ignores that call if the Java base class "thinks" it's blocking already.

Please try adding

serverSocket.getChannel().configureBlocking(false);
serverSocket.getChannel().configureBlocking(true);

right after casting the socket, and before calling accept.

[xueqiwang0v0]

@kohlschuetter It works! Thanks a lot!

[kohlschuetter]

@xueqiwang0v0
That's great news!

Please re-test with the latest snapshot (20240129.162003-4); I've made some adjustments.
Before testing, please make sure to remove the two configureBlocking calls from your code.

[kohlschuetter]

@gamlerhart FYI, this could be relevant for you too, given your previous experiments.
https://gamlor.info/posts-output/2019-10-15-java-file-descriptor-rant/en/

[xueqiwang0v0]

@kohlschuetter The new version works well. I removed configureBlocking and the accept method still block waiting for new connection.

[gamlerhart]

@kohlschuetter Thanks for the notification. I've added a update note to the old blog post, for future visitor.

[kohlschuetter]

Fixed in 2.9.0, closing.