Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Storage bucket.getFiles() not including sub-directories when specifying a prefix #514

Closed
drewbryant opened this issue Apr 28, 2015 · 7 comments
Assignees
Labels
api: storage Issues related to the Cloud Storage API.

Comments

@drewbryant
Copy link

GCS supports also returning "sub-directories" (i.e., common object prefixes) when listing a bucket and specifying a prefix (e.g., "/") within the query.

The sub-directories are returned as CommonPrefixes within the response.

Could these be included as well in the bucket.getFiles() response? Or maybe add a flag to specify whether subdirectories should also be returned.

Currently, the only work-around I've come up with is listing all objects within the bucket (expensive) and then filtering the response.

@stephenplusplus
Copy link
Contributor

Is this in the JSON API? https://cloud.google.com/storage/docs/json_api/v1/buckets/list doesn't seem to show it returning anything other than the objects that match the original prefix.

@drewbryant
Copy link
Author

Looks like it's in the XML API, but not the JSON API currently:
https://cloud.google.com/storage/docs/reference-headers#delimiter

@ryanseys
Copy link
Contributor

It might be the same thing as prefixes available through the JSON API (I'm not familiar with the XML API -- didn't even know one existed before today), in which case it's accessible through the apiResponse object provided in the 4th parameter in the getFiles() call: See here for relevant code. We're sorry it's not directly accessible through the first array of files.

@ryanseys
Copy link
Contributor

Yes, I believe that is the case. You can access the CommonPrefixes value like so:

var query = {
  delimiter: '/'
};
bucket.getFiles(query, function(err, files, nextQuery, apiResponse) {
  // files is array of files
  // apiResponse.prefixes is the same as CommonPrefixes
});

@drewbryant
Copy link
Author

Great to see that the common prefixes are already coming back in the fourth parameter. I think it's actually better that you keep the prefixes separated from the actual files/objects. Thanks Ryan/Stephen!

@ryanseys
Copy link
Contributor

ryanseys commented May 1, 2015

No problem, glad we could help!

@HadyZ
Copy link

HadyZ commented Apr 15, 2019

Hello I tried to specify the delimiter but nothing change I cannot access a folder ref to read files from , I am using admin sdk

any help

chingor13 pushed a commit that referenced this issue Aug 22, 2022
chingor13 pushed a commit that referenced this issue Aug 22, 2022
chingor13 pushed a commit that referenced this issue Aug 22, 2022
sofisl pushed a commit that referenced this issue Oct 11, 2022
… work correctly. (#514)

This PR was generated using Autosynth. 🌈

Synth log will be available here:
https://source.cloud.google.com/results/invocations/ddb5e2f8-cc18-4a70-ac58-0145aa7d8d54/targets

- [ ] To automatically regenerate this PR, check this box.

PiperOrigin-RevId: 335986756
Source-Link: googleapis/googleapis@b58004f
sofisl pushed a commit that referenced this issue Oct 13, 2022
… work correctly. (#514)

This PR was generated using Autosynth. 🌈

Synth log will be available here:
https://source.cloud.google.com/results/invocations/ddb5e2f8-cc18-4a70-ac58-0145aa7d8d54/targets

- [ ] To automatically regenerate this PR, check this box.

PiperOrigin-RevId: 335986756
Source-Link: googleapis/googleapis@b58004f
sofisl pushed a commit that referenced this issue Nov 10, 2022
[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [sinon](https://sinonjs.org/) ([source](https://togithub.com/sinonjs/sinon)) | [`^10.0.0` -> `^11.0.0`](https://renovatebot.com/diffs/npm/sinon/10.0.0/11.1.0) | [![age](https://badges.renovateapi.com/packages/npm/sinon/11.1.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/sinon/11.1.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/sinon/11.1.0/compatibility-slim/10.0.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/sinon/11.1.0/confidence-slim/10.0.0)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>sinonjs/sinon</summary>

### [`v11.1.0`](https://togithub.com/sinonjs/sinon/blob/master/CHANGELOG.md#&#8203;1110--2021-05-25)

[Compare Source](https://togithub.com/sinonjs/sinon/compare/v11.0.0...31be9a5d5a4762ef01cb195f29024616dfee9ce8)

\==================

-   Add sinon.promise() implementation ([#&#8203;2369](https://togithub.com/sinonjs/sinon/issues/2369))
-   Set wrappedMethod on getters/setters ([#&#8203;2378](https://togithub.com/sinonjs/sinon/issues/2378))
-   \[Docs] Update fake-server usage & descriptions ([#&#8203;2365](https://togithub.com/sinonjs/sinon/issues/2365))
-   Fake docs improvement ([#&#8203;2360](https://togithub.com/sinonjs/sinon/issues/2360))
-   Update nise to 5.1.0 (fixed [#&#8203;2318](https://togithub.com/sinonjs/sinon/issues/2318))

### [`v11.0.0`](https://togithub.com/sinonjs/sinon/blob/master/CHANGELOG.md#&#8203;1100--2021-05-24)

[Compare Source](https://togithub.com/sinonjs/sinon/compare/v10.0.1...v11.0.0)

\==================

-   Explicitly use samsam 6.0.2 with fix for [#&#8203;2345](https://togithub.com/sinonjs/sinon/issues/2345)
-   Update most packages ([#&#8203;2371](https://togithub.com/sinonjs/sinon/issues/2371))
-   Update compatibility docs ([#&#8203;2366](https://togithub.com/sinonjs/sinon/issues/2366))
-   Update packages (includes breaking fake-timers change, see [#&#8203;2352](https://togithub.com/sinonjs/sinon/issues/2352))
-   Warn of potential memory leaks ([#&#8203;2357](https://togithub.com/sinonjs/sinon/issues/2357))
-   Fix clock test errors

### [`v10.0.1`](https://togithub.com/sinonjs/sinon/blob/master/CHANGELOG.md#&#8203;1001--2021-04-08)

[Compare Source](https://togithub.com/sinonjs/sinon/compare/v10.0.0...v10.0.1)

\==================

-   Upgrade sinon components (bumps y18n to 4.0.1)
-   Bump y18n from 4.0.0 to 4.0.1

</details>

---

### Configuration

📅 **Schedule**: "after 9am and before 3pm" (UTC).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻️ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box.

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/nodejs-text-to-speech).
sofisl pushed a commit that referenced this issue Nov 10, 2022
[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [ts-loader](https://togithub.com/TypeStrong/ts-loader) | [`^8.0.0` -> `^9.0.0`](https://renovatebot.com/diffs/npm/ts-loader/8.1.0/9.0.0) | [![age](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/compatibility-slim/8.1.0)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/npm/ts-loader/9.0.0/confidence-slim/8.1.0)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>TypeStrong/ts-loader</summary>

### [`v9.0.0`](https://togithub.com/TypeStrong/ts-loader/blob/master/CHANGELOG.md#v900)

[Compare Source](https://togithub.com/TypeStrong/ts-loader/compare/v8.1.0...v9.0.0)

Breaking changes:

-   minimum webpack version: 5
-   minimum node version: 12

Changes:

-   [webpack 5 migration](https://togithub.com/TypeStrong/ts-loader/pull/1251) - thanks [@&#8203;johnnyreilly](https://togithub.com/johnnyreilly), [@&#8203;jonwallsten](https://togithub.com/jonwallsten), [@&#8203;sokra](https://togithub.com/sokra), [@&#8203;appzuka](https://togithub.com/appzuka), [@&#8203;alexander-akait](https://togithub.com/alexander-akait)

</details>

---

### Configuration

:date: **Schedule**: "after 9am and before 3pm" (UTC).

:vertical_traffic_light: **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

:recycle: **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox.

:no_bell: **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box.

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/nodejs-tasks).
sofisl pushed a commit that referenced this issue Nov 11, 2022
Co-authored-by: release-please[bot] <55107282+release-please[bot]@users.noreply.github.com>
sofisl pushed a commit that referenced this issue Nov 11, 2022
sofisl pushed a commit that referenced this issue Nov 11, 2022
- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 408439482

Source-Link: googleapis/googleapis@b9f6184

Source-Link: googleapis/googleapis-gen@eb888bc
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZWI4ODhiYzIxNGVmYzdiZjQzYmY0NjM0YjQ3MDI1NDU2NWE2NTlhNSJ9
sofisl pushed a commit that referenced this issue Nov 17, 2022
This PR was generated using Autosynth. 🌈

Synth log will be available here:
https://source.cloud.google.com/results/invocations/b8e77027-49e5-4749-89f7-3839afb36021/targets

- [ ] To automatically regenerate this PR, check this box.

Source-Link: googleapis/synthtool@89c849b
Source-Link: googleapis/synthtool@a783321
Source-Link: googleapis/synthtool@b7413d3
Source-Link: googleapis/synthtool@5f6ef0e
sofisl pushed a commit that referenced this issue Jan 10, 2023
This PR was generated using Autosynth. 🌈

Synth log will be available here:
https://source.cloud.google.com/results/invocations/940354f9-15cd-4361-bbf4-dc9af1426979/targets

- [ ] To automatically regenerate this PR, check this box.

Source-Link: googleapis/synthtool@99c93fe
sofisl pushed a commit that referenced this issue Sep 13, 2023
Fixes googleapis/synthtool#1103
Source-Link: googleapis/synthtool@c3e41da
Post-Processor: gcr.io/repo-automation-bots/owlbot-nodejs:latest@sha256:e37a815333a6f3e14d8532efe90cba8aa0d34210f8c0fdbdd9e6a34dcbe51e96
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the Cloud Storage API.
Projects
None yet
Development

No branches or pull requests

4 participants