FTP/SFTP support for loading data (JSON, CSV, XML)

[aethant]

Not sure whether this is a bug report, or a feature request or a request for clarification - is it possible to do an apoc.load call, using FTP/SFTP as a url? If so, how would you pass in the username and password?

I'm not really fluent in Java to delve into the code myself, but I'd imagine the underlying mechanics would involve opening the stream and reading it, just like any other file. Perhaps username and password could be passed in with the options object, just like the separator?

If it is currently possible, can someone point me in the right direction of how? If not, can I please officially request support for this? It'd save like 3 steps on a project I'm currently working on.

[abhishekunotech]

@aethant : If you do want to delve into the Java of this, Current implementation of apoc.load eventually traces back to creation of a HTTPClient Instance in Java using the URL you pass to apoc.load()...,

You can use an FTPClient instance of Java, in its place, if you have the resources to implement that and re-compile APOC locally.

[sarmbruster]

I think it should be easy to change https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/3.1/src/main/java/apoc/export/util/FileUtils.java#L18 to also check for ftp. By default the JVM does have URLStreamHandlers for http, https and ftp aboard. For any other protocols it is possible to supply additional URLStreamHandlers to the JVM, see https://docs.oracle.com/javase/8/docs/api/java/net/URL.html#URL-java.lang.String-java.lang.String-int-java.lang.String-

I think it's actually a good idea to change apoc code to be agnostic of url schemes and leave it up to the JVM if it can be loaded or not.

[abhishekunotech]

@sarmbruster, it will still require changes at https://github.com/neo4j-contrib/neo4j-apoc-procedures/blob/3.1/src/main/java/apoc/util/Util.java#L223, to create Clients specific to protocol.

It might require a bit more than a simple patch, to rework the requests to be url-scheme agnostic.

But, I see where you would want to go with it.

[jexp]

FTP at least should be easy to check via ftp:// URLs
Not sure about URL-Handlers for sftp