Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for GCS datastore #9888

Closed
3 tasks done
bjornleffler opened this issue May 23, 2023 · 9 comments
Closed
3 tasks done

Add support for GCS datastore #9888

bjornleffler opened this issue May 23, 2023 · 9 comments
Labels
kind/maintenance Work required to avoid breaking changes or harm to project's status quo kind/stale need/author-input Needs input from the original author

Comments

@bjornleffler
Copy link

Checklist

  • My issue is specific & actionable.
  • I am not suggesting a protocol enhancement.
  • I have searched on the issue tracker for my issue.

Description

I've built a new go-datastore for Google Cloud Storage (GCS), available at:
https://github.com/bjornleffler/go-ds-gcs

I would like to:

  1. Add a new config profile to kubo to simplify configuring kubo with the GCS datastore.
  2. Merge github.com/bjornleffler/go-ds-gcs as github.com/ipfs/go-ds-gcs

This new feature is code complete, tested and documented.

@bjornleffler bjornleffler added the kind/feature A new feature label May 23, 2023
@bjornleffler bjornleffler changed the title Support Add support for GCS datastore May 23, 2023
@lidel
Copy link
Member

lidel commented Jun 5, 2023

Triage notes:

  • every built-in datastore support introduces maintenance cost (see Badger2 integration #9230)
  • on top of that, the policy is to not include datastores specific to commercial services, for the same reason we don't have S3 one, and it lives in separate repository

Instead of introducing code to this repo, out suggestion here is to do keep this as plugin (https://github.com/bjornleffler/go-ds-gcs), and add "External datastore plugins" section to README or /docs/plugins|datastores 🙏

This way people will be able to discover third-party backends without introducing maintenance cost here.

@lidel lidel closed this as not planned Won't fix, can't repro, duplicate, stale Jun 5, 2023
@bjornleffler
Copy link
Author

I totally get the maintenance issue, maintaining code is hard. I can figure out some way to do the configuration separately form the main kubo binary. Probably something like this:
https://github.com/ipfs/go-ds-s3#configuration

How do I get my code merge my datastore code as a separate repository, similar to the S3 datastore repository?

@BigLep BigLep reopened this Jun 5, 2023
@BigLep
Copy link
Contributor

BigLep commented Jun 5, 2023

@lidel : I reopened so we speak to the question about repository location.

@lidel
Copy link
Member

lidel commented Jun 12, 2023

  • We are happy to take new (maintained) datastores under https://github.com/ipfs-shipyard/ umbrella
    • (the ``go-ds-s3is inipfs/` for legacy reasons, because we don't want to break existing users)
    • @bjornleffler if you want to donate your repo to ipfs-shipyard/ and still be its maintainer, add me as Admin, and I'll move it (or just keep it as is, if yo udont want to break your existign users)

@lidel lidel added need/author-input Needs input from the original author kind/maintenance Work required to avoid breaking changes or harm to project's status quo and removed kind/feature A new feature labels Jun 12, 2023
@github-actions
Copy link

Oops, seems like we needed more information for this issue, please comment with more details or this issue will be closed in 7 days.

@bjornleffler
Copy link
Author

Yes, that sounds great. Lets move the repository into the new location. Existing users can migrate.

Apologies for the late reply, I missed the previous notification.

@lidel
Copy link
Member

lidel commented Jun 19, 2023

@bjornleffler I seem to still lack permissions, maybe Transfer repo to me? I will re-add you after I'm done with moving it it the right place.

2023-06-19_12-14

@bjornleffler
Copy link
Author

Transfer requested. Please let me know if you need anything else.

@lidel
Copy link
Member

lidel commented Jun 19, 2023

That did the trick, thank you! Done: https://github.com/ipfs-shipyard/go-ds-gcs/

@bjornleffler you should have necessary permissions to continue being the maintainer of that repo.
If anything is missing, feel free to reach out (email, or Slack/Matrix in #ipfs-implementes room)

@lidel lidel closed this as not planned Won't fix, can't repro, duplicate, stale Jun 19, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind/maintenance Work required to avoid breaking changes or harm to project's status quo kind/stale need/author-input Needs input from the original author
Projects
None yet
Development

No branches or pull requests

3 participants