Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add doPrivilege blocks for socket connect ops in repository-hdfs #22793

Merged
merged 8 commits into from
Jan 27, 2017

Conversation

Tim-Brooks
Copy link
Contributor

This is related to #22116. The repository-hdfs plugin opens socket
connections. As SocketPermission is transitioned out of core, hdfs
will require connect permission. This pull request wraps operations
that require this permission in doPrivileged blocks.

@Tim-Brooks
Copy link
Contributor Author

Tim-Brooks commented Jan 25, 2017

In hdfs we are calling doPrivileged and explicitly defining the permissions. I was not sure if there was a reason we were doing this differently, so I left it as is and just added new SocketPermission("*", "connect").

@rjernst
Copy link
Member

rjernst commented Jan 25, 2017

In hdfs we are calling doPrivileged and explicitly defining the permissions

We do this to limit what craziness hdfs can do (and know exactly what it requires).

Copy link
Member

@rjernst rjernst left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

Copy link
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left one comment looks great so far

try {
SpecialPermission.check();
// FSDataInputStream can open connection on read()
return AccessController.doPrivileged((PrivilegedExceptionAction<Integer>) is::read);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we maybe try to trigger this differently? I mean can we for instance try to call #available() or can we maybe read the first byte on open and wrap in a BufferedInputStream and then do this:

InputStream stream = is.isMarkSupported() ? is : new BufferedInputStream(is);
// do the following in doPrivileged?
stream.mark(1);
stream.skip(1);
stream.reset();
return stream;

@Tim-Brooks
Copy link
Contributor Author

@s1monw In my testing available() was not doing the trick. So I went with the buffered, mark, skip, reset approach.

Copy link
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left one more comment we are close I think

}
}
};
InputStream stream = is.markSupported() ? is : new BufferedInputStream(is);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we run into an exception here we have to close the stream.

we usually do this:

boolean success = false;
try {
 // do something with the stream
  success = true;
  return stream;
} finally {
  if (success == false) {
    IOUtils.closeWhileHandlingException(stream);
  }
}

@Tim-Brooks
Copy link
Contributor Author

Note to other reviews:

Simon and I talked about this for a while and decided it was safest to return a InputStream implementation that wraps all of the methods.

Copy link
Contributor

@s1monw s1monw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@Tim-Brooks Tim-Brooks merged commit eb4562d into elastic:master Jan 27, 2017
@Tim-Brooks Tim-Brooks deleted the fix_hdfs branch January 27, 2017 21:01
jbaiera pushed a commit to jbaiera/elasticsearch that referenced this pull request Sep 19, 2017
…-hdfs (elastic#22793)"

Only pulled the relevant changes - such as the Priveleged input stream implementation for HDFS.
@clintongormley clintongormley added :Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs and removed :Plugin Repository HDFS labels Feb 14, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Distributed Coordination/Snapshot/Restore Anything directly related to the `_snapshot/*` APIs >enhancement v6.0.0-alpha1
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants