AccessControlException (java.io.FilePermission) when trying to access Cloud Storage

[Orbyt]

@lesv

Continuing from the conversation here.

I have tried every method suggested here but routinely get a java.security.AccessControlException which differs depending on which authentication used.

If I don't use .setCredentials (and therefore it trys to authenticate automatically or use GOOGLE_APPLICATION_CREDENTIALS env variable) I get an AccessControlException for a file called active_config. If I instead do call .setCredentials and use a file downloaded by creating a service account key (as specified in option 1 here: https://developers.google.com/identity/protocols/application-default-credentials) then I get the same exception but for that specific file.

@lesv you made this comment in the other issue:

In your case, passing the credentials created by gcloud is incorrect - you need to pass credentials obtained from https://console.cloud.google.com -- why don't you create a new issue, and mention me in it and I'll help you out.

I assume you are referring to the same method I linked to above, correct? If so, as stated, i've tried that.

[lesv]

First off, what environment are you developing for? GAE Std, Flex, GKE, GCP, something else?
Is this local development?

Take a look at the results of gcloud config list, is account set? project?

Are you using Maven, Gradle, or something else?

If Maven, which plugin are you using?

[Orbyt]

@lesv Java App Engine Standard Environment. I am trying to develop and test out the application locally.

Output of gcloud config list is the following:

Your active configuration is: [default]

[core]
account = [email protected]
disable_usage_reporting = True
project = myprojectname-123

Using Maven. There are quite a few plugins listed in my pom.xml, which ones are you asking about?

[lesv]

If you are using the old Maven Plugin, you'll be having the problem that they were discussing in #1500 - we can't access the file system, so none of the access mechanisms work. You might wish to try to the new plugin, but beware, it's BETA, and has trouble doing async IO requests - this should be fixed early next Q.

Some of the older client libraries (not from this REPO) continue to work with App Engine. (This is probably a better solution than using the new plugin for now). Apologies for the confusion -- I'll look into why we have samples that don't work locally.

[Orbyt]

@lesv I assume this conversation I had earlier is relevant: GoogleCloudPlatform/app-maven-plugin#151

The appengine-maven-plugin I have defined in my pom.xml is as follows:

<plugin>
    <groupId>com.google.cloud.tools</groupId>
    <artifactId>appengine-maven-plugin</artifactId>
    <version>${appengine-maven-plugin-version}</version>
</plugin>

I assume this is the "new" plugin. This issue persists.

[Orbyt]

Also, note that their seems to be a issue when using the following method:

Storage storage = StorageOptions.newBuilder()
            .setCredentials(ServiceAccountCredentials.
                fromStream(getServletContext().getResourceAsStream("credentials.json")))
            .build()
            .getService();

throws an NPE:

java.lang.NullPointerException
[INFO] GCLOUD: 	at com.google.api.client.repackaged.com.google.common.base.Preconditions.checkNotNull(Preconditions.java:213)
[INFO] GCLOUD: 	at com.google.api.client.util.Preconditions.checkNotNull(Preconditions.java:127)
[INFO] GCLOUD: 	at com.google.auth.oauth2.ServiceAccountCredentials.fromStream(ServiceAccountCredentials.java:252)
[INFO] GCLOUD: 	at com.google.auth.oauth2.ServiceAccountCredentials.fromStream(ServiceAccountCredentials.java:237)

I can read the file normally using GSON, so the file definitely exists. The issue seems to be internal.

[shinfan]

@Orbyt Can you provide me the full stacktrace of the AccessControlException?

[Orbyt]

@shinfan

Code:

public class DemoServlet extends HttpServlet {
  @Override
  public void doGet(HttpServletRequest req, HttpServletResponse resp)
      throws IOException {

    Storage storage = StorageOptions.getDefaultInstance().getService();
  }
} 

Stacktrace:

[INFO] GCLOUD: java.security.AccessControlException: access denied ("java.io.FilePermission" "/Users/example/.config/gcloud/active_config" "read")
[INFO] GCLOUD: 	at java.security.AccessControlContext.checkPermission(AccessControlContext.java:472)
[INFO] GCLOUD: 	at java.security.AccessController.checkPermission(AccessController.java:884)
[INFO] GCLOUD: 	at java.lang.SecurityManager.checkPermission(SecurityManager.java:549)
[INFO] GCLOUD: 	at com.google.appengine.tools.development.devappserver2.DevAppServer2Factory$CustomSecurityManager.checkPermission(DevAppServer2Factory.java:232)
[INFO] GCLOUD: 	at java.lang.SecurityManager.checkRead(SecurityManager.java:888)
[INFO] GCLOUD: 	at java.io.FileInputStream.<init>(FileInputStream.java:127)
[INFO] GCLOUD: 	at com.google.common.io.Files$FileByteSource.openStream(Files.java:126)
[INFO] GCLOUD: 	at com.google.common.io.Files$FileByteSource.openStream(Files.java:116)
[INFO] GCLOUD: 	at com.google.common.io.ByteSource$AsCharSource.openStream(ByteSource.java:448)
[INFO] GCLOUD: 	at com.google.common.io.CharSource.openBufferedStream(CharSource.java:92)
[INFO] GCLOUD: 	at com.google.common.io.CharSource.readFirstLine(CharSource.java:234)
[INFO] GCLOUD: 	at com.google.common.io.Files.readFirstLine(Files.java:523)
[INFO] GCLOUD: 	at com.google.cloud.ServiceOptions.getActiveGoogleCloudConfig(ServiceOptions.java:374)
[INFO] GCLOUD: 	at com.google.cloud.ServiceOptions.getGoogleCloudProjectId(ServiceOptions.java:391)
[INFO] GCLOUD: 	at com.google.cloud.ServiceOptions.getDefaultProjectId(ServiceOptions.java:363)
[INFO] GCLOUD: 	at com.google.cloud.ServiceOptions.getDefaultProject(ServiceOptions.java:336)
[INFO] GCLOUD: 	at com.google.cloud.ServiceOptions.<init>(ServiceOptions.java:284)
[INFO] GCLOUD: 	at com.google.cloud.HttpServiceOptions.<init>(HttpServiceOptions.java:179)
[INFO] GCLOUD: 	at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:69)
[INFO] GCLOUD: 	at com.google.cloud.storage.StorageOptions.<init>(StorageOptions.java:27)
[INFO] GCLOUD: 	at com.google.cloud.storage.StorageOptions$Builder.build(StorageOptions.java:64)
[INFO] GCLOUD: 	at com.google.cloud.storage.StorageOptions.getDefaultInstance(StorageOptions.java:99)
[INFO] GCLOUD: 	at myapp.DemoServlet.doGet(DemoServlet.java:19)

[shinfan]

Thanks, this looks the same issue as #1500. The cause of this issue is:

[INFO] GCLOUD: 	at com.google.cloud.ServiceOptions.getGoogleCloudProjectId(ServiceOptions.java:391)

where getGoogleCloudProjectId is called due to the runtime failure in com.google.cloud.ServiceOptions.getAppEngineProjectId

The failure is caused by the reflection method here where the GAE API classes are not loaded in the classpath.

To confirm, can you try to package your application via maven: mvn clean package and make sure appengine-api-1.0-sdk-1.9.50.jar is in your jar/war package?

[Orbyt]

@shinfan Is this why using

.setCredentials(ServiceAccountCredentials.fromStream(
    getServletContext().getResourceAsStream("credentials.json"))
)

Also fails, except with an NPE, as shown above?

I executed mvn clean package. A target/ directory was generated, with the following items:

• A /generated-sources directory
• A /maven-archiver directory
• A /maven-statusdirectory
• A /snapshot-app-engine directory
• A /snapshot-app-engine-1.0 directory
• A snapshot-app-engine-1.0.war

Inside /snapshot-app-engine-1.0/WEB-INF/lib/ I see a appengine-api-1.0-sdk-1.9.50.jar.

[shinfan]

The NPE issue of .setCredentialsshould be a separate issue. Normally you don't need to use that method.

@lesv Is it okay to ask the customer to send a copy of their sample application so I can try to reproduce the issue here? I could not repro with my own app (my own app works fine after adding the appengine sdk dependency)

[Orbyt]

@shinfan @lesv I would have no issue posting a repo to github as the project is empty and only contains a single DemoServlet class. However, because of that i'm not sure how useful posting it would be.

@shinfan You mentioned you are no longer able to reproduce this issue. Are you saying your are able to call StorageOptions.getDefaultInstance().getService(); without any issues? If so, could this be an issue within the gcloud SDK and it's setup on my machine? Perhaps some type of authentication issue?

[shinfan]

Yes, my sample application was able to call StorageOptions.getDefaultInstance().getService(); without any issues. Maybe there are some issues of your local environment but I am not 100% sure, that is why I think it would be helpful for me to debug with your app. If your app runs fine on my local machine, we can probably confirm it is an environment setup issue.

[Orbyt]

@shinfan Give me a few minutes and I will upload a repository.

[Orbyt]

@shinfan I've uploaded a private repository and sent you a collaboration invite. If others need or would like an invite for any reason, please let me know.

[Orbyt]

@shinfan Additionally, perhaps uploading your sample application would help me debug.

[Orbyt]

Well, automatic authentication seems to suddenly work and an AccessControlExpception is no longer thrown. I am able to access GCS and list my projects bucket names.

I am not sure why it has suddenly started working. Yesterday I was curious to see if I would have the same issues listed above with the Node.js implementation of App Engine. I cloned the getting started example, and added the google-cloud dependency from npm. I was immediately able to access Cloud Storage without any issues and without manually defining credentials.

Later that evening, I switched back to my Java App Engine project, and it began working as expected. Perhaps someone here has an idea of why this process fixed this issue.

[jakeam]

I'm also having a similar problem, I get the below error. I have also tried using a service key and setting the environment variable to point to the key with no luck.

{ "error": { "errors": [ { "domain": "global", "reason": "backendError", "message": "java.security.AccessControlException: access denied (\"java.io.FilePermission\" \"/home/jakeam/.config/gcloud/active_config\" \"read\")" } ], "code": 503, "message": "java.security.AccessControlException: access denied (\"java.io.FilePermission\" \"/home/jakeam/.config/gcloud/active_config\" \"read\")" } }

[shinfan]

@jabubake This issue should be fixed by adding the correct dependency into your application pom:
https://github.com/GoogleCloudPlatform/google-cloud-java/blob/master/APPENGINE.md

[jakeam]

@shinfan That dependency is included.

[jakeam]

I'm still experiencing this issue, is there any update on this?

[lesv]

Have you tried updating to 0.10.0-beta ?? (Not sure it will help, but at least that will eliminate that case) (I'm going to reread the notes.)

[lesv]

@Orbyt Apologies - I looked at some sample code and saw that it included the client library so I thought it was actually using the API. But I looked at the code and it's actually using older blobstore API's. In looking at our documentation, the recommendation (r/w to gcs) is to use the GCS for GAE client.

@jerjou Please update the GAE Bookshelf to use these API's.

[jerjou]

• Re: GAE Bookshelf - I believe the bookshelf is already using google-cloud-java - the package name matches.
• Re: java.io.FilePermission error - You might be able to get around this problem by setting the GOOGLE_CLOUD_PROJECT environment variable to your project id. Note you should only have to do this when testing locally - it should be inferred once your app is deployed onto App Engine.

[garrettjonesgoogle]

@jakeam , are you still having trouble with this?

[garrettjonesgoogle]

No response, so I'm going to close this out. Feel free to re-open if there are still issues.