-
Notifications
You must be signed in to change notification settings - Fork 31
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
* add scaladoc support. * move test data. * Revert "move test data." This reverts commit 3928720. * fix test data path. * cross scala version. * another try. * final fix. * prevent forcing unsupported versions.
- Loading branch information
Showing
35 changed files
with
200 additions
and
428 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -9,8 +9,32 @@ name: Website | |
| - published | ||
| push: | ||
| branches: | ||
| - zio2 | ||
| - series/2.x | ||
| pull_request: {} | ||
| jobs: | ||
| build: | ||
| name: Build and Test | ||
| runs-on: ubuntu-latest | ||
| if: ${{ github.event_name == 'pull_request' }} | ||
| steps: | ||
| - name: Git Checkout | ||
| uses: actions/[email protected] | ||
| with: | ||
| fetch-depth: '0' | ||
| - name: Setup Scala | ||
| uses: actions/[email protected] | ||
| with: | ||
| distribution: temurin | ||
| java-version: 17 | ||
| check-latest: true | ||
| - name: Check if the README file is up to date | ||
| run: sbt docs/checkReadme | ||
| - name: Check if the site workflow is up to date | ||
| run: sbt docs/checkGithubWorkflow | ||
| - name: Check artifacts build process | ||
| run: sbt +publishLocal | ||
| - name: Check website build process | ||
| run: sbt docs/clean; sbt docs/buildWebsite | ||
| publish-docs: | ||
| name: Publish Docs | ||
| runs-on: ubuntu-latest | ||
|
|
@@ -38,6 +62,7 @@ jobs: | |
| generate-readme: | ||
| name: Generate README | ||
| runs-on: ubuntu-latest | ||
| if: ${{ (github.event_name == 'push') || ((github.event_name == 'release') && (github.event_name == 'published')) }} | ||
| steps: | ||
| - name: Git Checkout | ||
| uses: actions/[email protected] | ||
|
|
@@ -59,16 +84,15 @@ jobs: | |
| git add README.md | ||
| git commit -m "Update README.md" || echo "No changes to commit" | ||
| - name: Create Pull Request | ||
| uses: peter-evans/create-pull-request@v4 | ||
| uses: peter-evans/create-pull-request@v4.2.3 | ||
| with: | ||
| body: | | ||
| body: |- | ||
| Autogenerated changes after running the `sbt docs/generateReadme` command of the [zio-sbt-website](https://zio.dev/zio-sbt) plugin. | ||
| I will automatically update the README.md file whenever there is new change for README.md, e.g. | ||
| - After each release, I will update the version in the installation section. | ||
| - After any changes to the "docs/index.md" file, I will update the README.md file accordingly. | ||
| branch: zio-sbt-website/update-readme | ||
| commit-message: Update README.md | ||
| branch-suffix: short-commit-hash | ||
| delete-branch: true | ||
| title: Update README.md | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,25 +1,166 @@ | ||
| # ZIO-S3 | ||
| An S3 client for ZIO | ||
| [//]: # (This file was autogenerated using `zio-sbt-website` plugin via `sbt generateReadme` command.) | ||
| [//]: # (So please do not edit it manually. Instead, change "docs/index.md" file or sbt setting keys) | ||
| [//]: # (e.g. "readmeDocumentation" and "readmeSupport".) | ||
|
|
||
| [![Project stage][Stage]][Stage-Page] | ||
| [![CircleCI][Badge-Circle]][Link-Circle] | ||
| [![Release][Badge-SonatypeReleases]][Link-SonatypeReleases] | ||
| [![Snapshots][Badge-SonatypeSnapshots]][Link-SonatypeSnapshots] | ||
| # ZIO S3 | ||
|
|
||
| Setup | ||
| ----- | ||
| [ZIO S3](https://github.com/zio/zio-s3) is a thin wrapper over S3 async client for ZIO. | ||
|
|
||
| The library is a thin wrapper over s3 async client. | ||
| Please find more details in the [website](https://zio.dev/zio-s3/) | ||
| [](https://github.com/zio/zio/wiki/Project-Stages)  [](https://oss.sonatype.org/content/repositories/releases/dev/zio/zio-s3_2.13/) [](https://oss.sonatype.org/content/repositories/snapshots/dev/zio/zio-s3_2.13/) [](https://javadoc.io/doc/dev.zio/zio-s3-docs_2.13) [](https://github.com/zio/zio-s3) | ||
|
|
||
| [Badge-Circle]: https://circleci.com/gh/zio/zio-s3.svg?style=svg "circleci" | ||
| [Link-Circle]: https://circleci.com/gh/zio/zio-s3 "circleci" | ||
| ## Introduction | ||
|
|
||
| [Link-SonatypeReleases]: https://oss.sonatype.org/content/repositories/releases/dev/zio/zio-s3_2.12/ "Sonatype Releases" | ||
| [Badge-SonatypeReleases]: https://img.shields.io/nexus/r/https/oss.sonatype.org/dev.zio/zio-s3_2.12.svg "Sonatype Releases" | ||
| ZIO-S3 is a thin wrapper over the s3 async java client. It exposes the main operations of the s3 java client. | ||
|
|
||
| [Badge-SonatypeSnapshots]: https://img.shields.io/nexus/s/https/oss.sonatype.org/dev.zio/zio-s3_2.12.svg "Sonatype Snapshots" | ||
| [Link-SonatypeSnapshots]: https://oss.sonatype.org/content/repositories/snapshots/dev/zio/zio-s3_2.12/ "Sonatype Snapshots" | ||
| ```scala | ||
| import software.amazon.awssdk.auth.credentials.AwsBasicCredentials | ||
| import zio.Chunk | ||
| import zio.s3._ | ||
| import zio.stream.{ZSink, ZStream} | ||
| import software.amazon.awssdk.services.s3.model.S3Exception | ||
|
|
||
| [Stage]: https://img.shields.io/badge/Project%20Stage-Production%20Ready-brightgreen.svg | ||
| [Stage-Page]: https://github.com/zio/zio/wiki/Project-Stages | ||
| // list all buckets available | ||
| listBuckets.provideLayer( | ||
| live("us-east-1", AwsBasicCredentials.create("accessKeyId", "secretAccessKey")) | ||
| ) | ||
|
|
||
| // list all objects of all buckets | ||
| val l2: ZStream[S3, S3Exception, String] = (for { | ||
| bucket <- ZStream.fromIterableZIO(listBuckets) | ||
| obj <- listAllObjects(bucket.name) | ||
| } yield obj.bucketName + "/" + obj.key).provideLayer( | ||
| live("us-east-1", AwsBasicCredentials.create("accessKeyId", "secretAccessKey")) | ||
| ) | ||
| ``` | ||
|
|
||
| All available s3 combinators and operations are available in the package object `zio.s3`, you only need to `import zio.s3._` | ||
|
|
||
| ## Installation | ||
|
|
||
| In order to use this library, we need to add the following line in our `build.sbt` file: | ||
|
|
||
| ```scala | ||
| libraryDependencies += "dev.zio" %% "zio-s3" % "0.4.2.1" | ||
| ``` | ||
|
|
||
| ## Example 1 | ||
|
|
||
| Let's try an example of creating a bucket and adding an object into it. To run this example, we need to run an instance of _Minio_ which is object storage compatible with S3: | ||
|
|
||
| ```bash | ||
| docker run -p 9000:9000 -e MINIO_ACCESS_KEY=MyKey -e MINIO_SECRET_KEY=MySecret minio/minio server --compat /data | ||
| ``` | ||
|
|
||
| In this example we create a bucket and then add a JSON object to it and then retrieve that: | ||
|
|
||
| ```scala | ||
| import software.amazon.awssdk.auth.credentials.AwsBasicCredentials | ||
| import software.amazon.awssdk.regions.Region | ||
| import zio._ | ||
| import zio.s3._ | ||
| import zio.stream.{ZStream, ZPipeline} | ||
| import zio.{Chunk, ExitCode, URIO} | ||
|
|
||
| import java.net.URI | ||
|
|
||
| object ZIOS3Example extends ZIOAppDefault { | ||
|
|
||
| val myApp = for { | ||
| _ <- createBucket("docs") | ||
| json = Chunk.fromArray("""{ "id" : 1 , "name" : "A1" }""".getBytes) | ||
| _ <- putObject( | ||
| bucketName = "docs", | ||
| key = "doc1", | ||
| contentLength = json.length, | ||
| content = ZStream.fromChunk(json), | ||
| options = UploadOptions.fromContentType("application/json") | ||
| ) | ||
| _ <- getObject("docs", "doc1") | ||
| .via(ZPipeline.utf8Decode) | ||
| .foreach(Console.printLine(_)) | ||
| } yield () | ||
|
|
||
| def run = | ||
| myApp | ||
| .provide( | ||
| live( | ||
| Region.CA_CENTRAL_1, | ||
| AwsBasicCredentials.create("MyKey", "MySecret"), | ||
| Some(URI.create("http://localhost:9000")) | ||
| ) | ||
| ) | ||
| } | ||
| ``` | ||
|
|
||
| ## Example 2 | ||
|
|
||
| ```scala | ||
| import software.amazon.awssdk.services.s3.model.S3Exception | ||
| import zio._ | ||
| import zio.stream.{ ZSink, ZStream } | ||
| import zio.s3._ | ||
|
|
||
| // upload | ||
| val json: Chunk[Byte] = Chunk.fromArray("""{ "id" : 1 , "name" : "A1" }""".getBytes) | ||
| val up: ZIO[S3, S3Exception, Unit] = putObject( | ||
| "bucket-1", | ||
| "user.json", | ||
| json.length, | ||
| ZStream.fromChunk(json), | ||
| UploadOptions.fromContentType("application/json") | ||
| ) | ||
|
|
||
| // multipartUpload | ||
| import java.io.FileInputStream | ||
| import java.nio.file.Paths | ||
|
|
||
| val is = ZStream.fromInputStream(new FileInputStream(Paths.get("/my/path/to/myfile.zip").toFile)) | ||
| val proc2: ZIO[S3, S3Exception, Unit] = | ||
| multipartUpload( | ||
| "bucket-1", | ||
| "upload/myfile.zip", | ||
| is, | ||
| MultipartUploadOptions.fromUploadOptions(UploadOptions.fromContentType("application/zip")) | ||
| )(4) | ||
|
|
||
| // download | ||
| import java.io.OutputStream | ||
|
|
||
| val os: OutputStream = ??? | ||
| val proc3: ZIO[S3, Exception, Long] = getObject("bucket-1", "upload/myfile.zip").run(ZSink.fromOutputStream(os)) | ||
| ``` | ||
|
|
||
| ## Support any commands? | ||
|
|
||
| If you need a method which is not wrapped by the library, you can have access to underlying S3 client in a safe manner by using | ||
|
|
||
| ```scala | ||
| import java.util.concurrent.CompletableFuture | ||
| import zio.s3._ | ||
| import software.amazon.awssdk.services.s3.S3AsyncClient | ||
|
|
||
| def execute[T](f: S3AsyncClient => CompletableFuture[T]) | ||
| ``` | ||
|
|
||
| ## Documentation | ||
|
|
||
| Learn more on the [ZIO S3 homepage](https://zio.dev/zio-s3/)! | ||
|
|
||
| ## Contributing | ||
|
|
||
| For the general guidelines, see ZIO [contributor's guide](https://zio.dev/about/contributing). | ||
|
|
||
| ## Code of Conduct | ||
|
|
||
| See the [Code of Conduct](https://zio.dev/about/code-of-conduct) | ||
|
|
||
| ## Support | ||
|
|
||
| Come chat with us on [![Badge-Discord]][Link-Discord]. | ||
|
|
||
| [Badge-Discord]: https://img.shields.io/discord/629491597070827530?logo=discord "chat on discord" | ||
| [Link-Discord]: https://discord.gg/2ccFBr4 "Discord" | ||
|
|
||
| ## License | ||
|
|
||
| [License](LICENSE) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.