Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for decoding event streams #566

Merged
merged 7 commits into from
Jul 20, 2023
Merged

Add support for decoding event streams #566

merged 7 commits into from
Jul 20, 2023

Conversation

adam-fowler
Copy link
Member

@adam-fowler adam-fowler commented Jul 19, 2023

See S3.SelectObjectContent, Lambda.InvokeWithResponseStream etc. You can see documentation on decoding these here https://docs.aws.amazon.com/AmazonS3/latest/API/RESTSelectObjectAppendix.html.

  • Added an AWSEventStream type that reads from a base ByteBuffer async sequence.
  • These buffers are passed to an EventStreamDecoder which decodes the headers and then passes payload onto relevant decoder.

Related PRs: soto-project/soto-codegenerator#74, soto-project/soto#686

@codecov
Copy link

codecov bot commented Jul 19, 2023

Codecov Report

Merging #566 (bbf4079) into 7.x.x (15744ff) will decrease coverage by 0.16%.
The diff coverage is 81.31%.

@@            Coverage Diff             @@
##            7.x.x     #566      +/-   ##
==========================================
- Coverage   78.87%   78.72%   -0.16%     
==========================================
  Files          71       73       +2     
  Lines        6292     6487     +195     
==========================================
+ Hits         4963     5107     +144     
- Misses       1329     1380      +51     
Impacted Files Coverage Δ
Sources/SotoCore/Doc/AWSShape.swift 50.72% <ø> (ø)
Sources/SotoCore/Encoder/ResponseContainer.swift 35.80% <0.00%> (-1.38%) ⬇️
Sources/SotoCore/Encoder/EventStreamDecoder.swift 77.85% <77.85%> (ø)
Sources/SotoXML/Expat.swift 94.87% <80.00%> (+0.13%) ⬆️
Sources/SotoCore/Concurrency/EventStream.swift 95.91% <95.91%> (ø)
Sources/SotoTestUtils/TestUtils.swift 84.61% <100.00%> (ø)

... and 6 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@adam-fowler adam-fowler requested a review from 0xTim July 19, 2023 15:17
Copy link
Member

@0xTim 0xTim left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nits about access control and doc comments but take or leave them

public struct EventDecodingContainer {
let payload: ByteBuffer

public func decodePayload() -> ByteBuffer {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor nit - this has no DocC comments

Comment on lines 22 to 24
public init() {}

public func decode<T: Decodable>(_ type: T.Type, from buffer: inout ByteBuffer) throws -> T {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these need to be public if the type isn't?

@adam-fowler adam-fowler merged commit adb4c26 into 7.x.x Jul 20, 2023
@adam-fowler adam-fowler deleted the event-stream branch July 20, 2023 08:23
adam-fowler added a commit that referenced this pull request Jul 24, 2023
* Add EventStream type

* Split event stream code into two files

* If response is chunked assume it is raw

* Fix test errors

* Fix a couple strict concurrency issues I missed

* Add tests for EventStreams

* Changes from PR review
adam-fowler added a commit that referenced this pull request Jul 29, 2023
* Add EventStream type

* Split event stream code into two files

* If response is chunked assume it is raw

* Fix test errors

* Fix a couple strict concurrency issues I missed

* Add tests for EventStreams

* Changes from PR review
adam-fowler added a commit that referenced this pull request Aug 12, 2023
* Add EventStream type

* Split event stream code into two files

* If response is chunked assume it is raw

* Fix test errors

* Fix a couple strict concurrency issues I missed

* Add tests for EventStreams

* Changes from PR review
adam-fowler added a commit that referenced this pull request Dec 24, 2023
* Add EventStream type

* Split event stream code into two files

* If response is chunked assume it is raw

* Fix test errors

* Fix a couple strict concurrency issues I missed

* Add tests for EventStreams

* Changes from PR review
adam-fowler added a commit that referenced this pull request Apr 5, 2024
* Add EventStream type

* Split event stream code into two files

* If response is chunked assume it is raw

* Fix test errors

* Fix a couple strict concurrency issues I missed

* Add tests for EventStreams

* Changes from PR review
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants