You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
when using suro to upload file to s3/hdfs,i found something not reasonable.
if the suro sink can not connect the hdfs, localFile file will be saved as local instead;
when the hdfs was connected again, suro upload the saved files again...
but the uploadPath was constructed using current time .
i think it'll be better to use the file's last modified time to build the upload Path. so that the file i created will be update to the right date directory...
Right now, RemotePrefixFormatter is not having any parameter. We can change the interface RemotePrefixFormatter.get() as the RemotePrefixFormatter.get(File file) and then we can create another PrefixFormatter which will return the date of file's last modified.
when using suro to upload file to s3/hdfs,i found something not reasonable.
if the
suro
sink can not connect the hdfs, localFile file will be saved as local instead;when the hdfs was connected again, suro upload the saved files again...
but the uploadPath was constructed using current time .
i think it'll be better to use the file's last modified time to build the upload Path. so that the file i created will be update to the right date directory...
https://github.com/Netflix/suro/blob/master/suro-s3/src/main/java/com/netflix/suro/sink/remotefile/RemoteFileSink.java#L214
The text was updated successfully, but these errors were encountered: