Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Can't set CORS to <Origin>*</Origin> when initiating resumable uploads? #1336

Closed
omerzach opened this issue Oct 25, 2016 · 7 comments
Closed
Assignees
Labels
api: storage Issues related to the Cloud Storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@omerzach
Copy link

https://cloud.google.com/storage/docs/cross-origin#How_CORS_Works:

When using the resumable upload protocol, the Origin from the first (start upload) request is always used to decide the Access-Control-Allow-Origin header in the response, even if you use a different Origin for subsequent requests. Therefore, you should either use the same Origin for the first and subsequent requests, or if the first request has a different Origin than subsequent requests, use the XML API with the CORS configuration set to *.

I'm initiating resumable uploads from my server, sending the generated upload URL to my clients (web/mobile apps), and then having them actually POST files to Google Cloud Storage. It seems I need CORS configured to <Origin>*</Origin> to support this, but I don't see a way to do so in the SDK. Is there a solution I'm missing?

@lesv
Copy link
Contributor

lesv commented Oct 25, 2016

Take a look at the Cors, Cors.Builder, and Cors.Origin classes in the javadocs

I couldn't find an example, but the tests show creating and setting them.

@lesv lesv added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. docs api: storage Issues related to the Cloud Storage API. labels Oct 25, 2016
@omerzach
Copy link
Author

Yep I see how to create the Cors policy I need, but not how to associate it with the API request the SDK is making for me?

@omerzach
Copy link
Author

Looks like I have the exact same issue as this: danialfarid/ng-file-upload#1192

In their PHP solution they add "Origin" => env('APP_ADDRESS') to their initial call that starts the resumable upload. I'd like to similarly add either * or my specific client address but don't see a way to do this with the SDK.

@lesv
Copy link
Contributor

lesv commented Oct 25, 2016

So, CORS is typically something you set once. (At least as I understand it)

You set CORS on a bucket, and not AFAIK on an object. (Are you thinking something else)?

Bucket.Builder has a cors(Iterable<Cors> cors) method
BucketInfo.Builder also has a cors method.

So, in looking, it appears that you can update a Bucket using a BucketInfo

     Cors cors = Cors.newBuilder()
        .setMaxAgeSeconds(100)
        .setOrigins(origins)
        .setResponseHeaders(headers)
        .setMethods(methods)
        .build();

    String bucketName = "my_unique_bucket";
    BucketInfo bucketInfo = BucketInfo.builder(bucketName).cors(cors).build();
    Bucket bucket = storage.update(bucketInfo);

Note - the cors() method will become setCors() in the next release.

@mziccard
Copy link
Contributor

Hi @omerzach! Are you using google-cloud-java to create the resumable upload URL? If yes, can you share with us the code you are using?

google-cloud-storage uses the JSON api under the hood. AFAIK when you create a resumable upload URL using the JSON api you need to preserve the same Origin that you used for the opening request in all subsequent upload requests.

A possible workaround could be to override the Origin header in the request that opens the resumable URL, as it's done in danialfarid/ng-file-upload#1192. Unfortunately, at the moment google-cloud-storage does not expose such a option. In fact, we do not even expose a method to create a resumable URL in our Storage interface.

@omerzach
Copy link
Author

Yeah, looks like that's exactly the issue.

So I guess the only solution is to directly call the endpoint to start the resumable upload with a custom origin instead of using the SDK?

For what it's worth, I'm calling from Scala, not Java, but:

  val options: StorageOptions = StorageOptions.defaultInstance
  val storageRpc = new DefaultStorageRpc(options)

  val storageObject = {
    (new StorageObject)
      .setBucket(bucket)
      .setName(key)
      .setContentType(contentType)
      .setSize(BigInt(contentLength).bigInteger)
  }

  storageRpc.open(storageObject, Map.empty.asJava)

@mziccard
Copy link
Contributor

So I guess the only solution is to directly call the endpoint to start the resumable upload with a custom origin instead of using the SDK?

Yeah that would be your best option for now. Provided the http client lets you override origin (wouldn't give that for granted). Have a look at the code in DefaultStorageRpc.open for an idea of how to open the resumable upload URL.

@mziccard mziccard closed this as completed Nov 1, 2016
github-actions bot pushed a commit that referenced this issue Sep 15, 2022
…16.0 (#1336)

[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [com.google.cloud:google-cloud-bigquery](https://togithub.com/googleapis/java-bigquery) | `2.15.0` -> `2.16.0` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-bigquery/2.16.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-bigquery/2.16.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-bigquery/2.16.0/compatibility-slim/2.15.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-bigquery/2.16.0/confidence-slim/2.15.0)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>googleapis/java-bigquery</summary>

### [`v2.16.0`](https://togithub.com/googleapis/java-bigquery/blob/HEAD/CHANGELOG.md#&#8203;2160-httpsgithubcomgoogleapisjava-bigquerycomparev2150v2160-2022-09-12)

[Compare Source](https://togithub.com/googleapis/java-bigquery/compare/v2.15.0...v2.16.0)

##### Features

-   Add preserveAsciiControlCharacters to CsvOptions ([#&#8203;2143](https://togithub.com/googleapis/java-bigquery/issues/2143)) ([856893f](https://togithub.com/googleapis/java-bigquery/commit/856893f4d8f1b419365d8f179ce9f9e571dec718))
-   Add reference file schema option for federated formats ([#&#8203;2269](https://togithub.com/googleapis/java-bigquery/issues/2269)) ([8c488e6](https://togithub.com/googleapis/java-bigquery/commit/8c488e64259bd67716342f48f96d2932c5e57c3e))

##### Bug Fixes

-   Socket-timeout at bigquery.it.ITNightlyBigQueryTest: testForTableNotFound ([#&#8203;2260](https://togithub.com/googleapis/java-bigquery/issues/2260)) ([a9b5fb2](https://togithub.com/googleapis/java-bigquery/commit/a9b5fb2c1078788ddb1ac3169c9ce597af228ac0))

##### Dependencies

-   Update dependency com.google.apis:google-api-services-bigquery to v2-rev20220827-2.0.0 ([#&#8203;2261](https://togithub.com/googleapis/java-bigquery/issues/2261)) ([3c67d21](https://togithub.com/googleapis/java-bigquery/commit/3c67d21c10f66b3c5313a1733f4e81db42c1b7c3))
-   Update dependency com.google.cloud:google-cloud-datacatalog-bom to v1.9.3 ([#&#8203;2259](https://togithub.com/googleapis/java-bigquery/issues/2259)) ([5e30a04](https://togithub.com/googleapis/java-bigquery/commit/5e30a04e5b14b03e60e587787180b27f605d6abd))
-   Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.2 ([#&#8203;2267](https://togithub.com/googleapis/java-bigquery/issues/2267)) ([8472fe5](https://togithub.com/googleapis/java-bigquery/commit/8472fe580a8197aaa3957dd3231fed0a9511fbb5))

</details>

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-asset).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzMi4xOTQuNSIsInVwZGF0ZWRJblZlciI6IjMyLjE5NC41In0=-->
github-actions bot pushed a commit that referenced this issue Sep 16, 2022
🤖 I have created a release *beep* *boop*
---


## [3.6.0](googleapis/java-asset@v3.5.0...v3.6.0) (2022-09-15)


### Features

* Add client library support for AssetService v1 BatchGetEffectiveIamPolicies API ([3919a1d](googleapis/java-asset@3919a1d))
* Add client library support for AssetService v1 BatchGetEffectiveIamPolicies API ([#1300](googleapis/java-asset#1300)) ([3919a1d](googleapis/java-asset@3919a1d))
* Release of query system ([3919a1d](googleapis/java-asset@3919a1d))


### Dependencies

* Update dependency com.google.api.grpc:proto-google-cloud-orgpolicy-v1 to v2.3.2 ([#1302](googleapis/java-asset#1302)) ([d01d900](googleapis/java-asset@d01d900))
* Update dependency com.google.api.grpc:proto-google-cloud-orgpolicy-v1 to v2.3.3 ([#1332](googleapis/java-asset#1332)) ([c1511c2](googleapis/java-asset@c1511c2))
* Update dependency com.google.api.grpc:proto-google-cloud-os-config-v1 to v2.5.2 ([#1309](googleapis/java-asset#1309)) ([cf96ee9](googleapis/java-asset@cf96ee9))
* Update dependency com.google.api.grpc:proto-google-cloud-pubsub-v1 to v1.102.11 ([#1297](googleapis/java-asset#1297)) ([d56eedd](googleapis/java-asset@d56eedd))
* Update dependency com.google.api.grpc:proto-google-cloud-pubsub-v1 to v1.102.12 ([#1316](googleapis/java-asset#1316)) ([a3713fd](googleapis/java-asset@a3713fd))
* Update dependency com.google.api.grpc:proto-google-cloud-pubsub-v1 to v1.102.13 ([#1321](googleapis/java-asset#1321)) ([883b7b8](googleapis/java-asset@883b7b8))
* Update dependency com.google.api.grpc:proto-google-cloud-pubsub-v1 to v1.102.14 ([#1334](googleapis/java-asset#1334)) ([56cb4b4](googleapis/java-asset@56cb4b4))
* Update dependency com.google.api.grpc:proto-google-identity-accesscontextmanager-v1 to v1.4.1 ([#1307](googleapis/java-asset#1307)) ([b90baf7](googleapis/java-asset@b90baf7))
* Update dependency com.google.cloud:google-cloud-bigquery to v2.14.4 ([#1301](googleapis/java-asset#1301)) ([64f7ea5](googleapis/java-asset@64f7ea5))
* Update dependency com.google.cloud:google-cloud-bigquery to v2.14.6 ([#1315](googleapis/java-asset#1315)) ([fa179e2](googleapis/java-asset@fa179e2))
* Update dependency com.google.cloud:google-cloud-bigquery to v2.14.7 ([#1320](googleapis/java-asset#1320)) ([06d1a16](googleapis/java-asset@06d1a16))
* Update dependency com.google.cloud:google-cloud-bigquery to v2.15.0 ([#1326](googleapis/java-asset#1326)) ([df36595](googleapis/java-asset@df36595))
* Update dependency com.google.cloud:google-cloud-bigquery to v2.16.0 ([#1336](googleapis/java-asset#1336)) ([870779d](googleapis/java-asset@870779d))
* Update dependency com.google.cloud:google-cloud-core to v2.8.10 ([#1323](googleapis/java-asset#1323)) ([09e03b8](googleapis/java-asset@09e03b8))
* Update dependency com.google.cloud:google-cloud-core to v2.8.11 ([#1331](googleapis/java-asset#1331)) ([a112cec](googleapis/java-asset@a112cec))
* Update dependency com.google.cloud:google-cloud-core to v2.8.9 ([#1314](googleapis/java-asset#1314)) ([8edc2b8](googleapis/java-asset@8edc2b8))
* Update dependency com.google.cloud:google-cloud-pubsub to v1.120.11 ([#1298](googleapis/java-asset#1298)) ([172b34b](googleapis/java-asset@172b34b))
* Update dependency com.google.cloud:google-cloud-pubsub to v1.120.12 ([#1317](googleapis/java-asset#1317)) ([1ea636f](googleapis/java-asset@1ea636f))
* Update dependency com.google.cloud:google-cloud-pubsub to v1.120.13 ([#1322](googleapis/java-asset#1322)) ([b7522b9](googleapis/java-asset@b7522b9))
* Update dependency com.google.cloud:google-cloud-pubsub to v1.120.14 ([#1335](googleapis/java-asset#1335)) ([e9142b4](googleapis/java-asset@e9142b4))
* Update dependency com.google.cloud:google-cloud-resourcemanager to v1.5.3 ([#1318](googleapis/java-asset#1318)) ([aa3a1bb](googleapis/java-asset@aa3a1bb))
* Update dependency com.google.cloud:google-cloud-resourcemanager to v1.5.4 ([#1328](googleapis/java-asset#1328)) ([fca0ce5](googleapis/java-asset@fca0ce5))
* Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.2 ([#1330](googleapis/java-asset#1330)) ([ccb704c](googleapis/java-asset@ccb704c))
* Update dependency com.google.cloud:google-cloud-shared-dependencies to v3.0.3 ([#1340](googleapis/java-asset#1340)) ([0d87a9d](googleapis/java-asset@0d87a9d))
* Update dependency com.google.cloud:google-cloud-storage to v2.11.3 ([#1299](googleapis/java-asset#1299)) ([d59e6c6](googleapis/java-asset@d59e6c6))

---
This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants