Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement per-operation options for storage standalone functions #9211

Closed
coryan opened this issue Jun 8, 2022 · 2 comments
Closed

Implement per-operation options for storage standalone functions #9211

coryan opened this issue Jun 8, 2022 · 2 comments
Labels
cpp: backlog While desirable, we do not have time to work on this for the foreseeable future. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@coryan
Copy link
Contributor

coryan commented Jun 8, 2022

The storage library has a few standalone functions that refactor common workflows making multiple calls to the storage::Client class. For these classes we need a different implementation of the per-operation options. Consider, for example, ComposeMany(). Basically it does something like this:

template <typename Options>
StatusOr<ObjectMetadata> ComposeMany(
      storage::Client client, std::vector<std::string> object_names, Options&&... options) {
  auto c1 = client.Compose(/* some of the objects from `object_names */, std::forward<Options>(options));
  auto c2 = client.Compose(/* some of the objects from `object_names */, std::forward<Options>(options));
....
  auto cN = client.Compose(/* some of the objects from `object_names */, std::forward<Options>(options));

  return client.Compose(c1 .... cN, std::forward<Options>(options));
}

The problem is that any google::cloud::Options hidden in the parameter pack may get moved-copied in the first call creating c1. The remaining calls get the wrong overrides. We need to do something like:

template <typename Options>
StatusOr<ObjectMetadata> ComposeMany(
      storage::Client client, std::vector<std::string> object_names, Options&&... options) {
  auto const grouped = storage::internal::GroupOptions(std::forward<Options>(options)...);
  auto c1 = client.Compose(/* some of the objects from `object_names */, std::forward<Options>(options)..., grouped);
  auto c2 = client.Compose(/* some of the objects from `object_names */, std::forward<Options>(options)..., grouped);
....
  auto cN = client.Compose(/* some of the objects from `object_names */, std::forward<Options>(options)..., grouped);

  return client.Compose(c1 .... cN, std::forward<Options>(options)..., grouped);
}

I think the relevant functions are:

  • DeleteByPrefix()
  • ComposeMany()
  • ParallelUploadFile()

These functions have many helpers in the internal namespace, but I think we can ignore those?

@coryan coryan added the type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. label Jun 8, 2022
@coryan
Copy link
Contributor Author

coryan commented Oct 12, 2022

Still want this.

@coryan coryan self-assigned this Mar 22, 2023
@coryan
Copy link
Contributor Author

coryan commented Jul 19, 2023

Evidently we don't have time to work on this. To the backlog.

@coryan coryan closed this as completed Jul 19, 2023
@coryan coryan removed their assignment Jul 19, 2023
@coryan coryan added the cpp: backlog While desirable, we do not have time to work on this for the foreseeable future. label Jul 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cpp: backlog While desirable, we do not have time to work on this for the foreseeable future. type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

1 participant