Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Adding grpc compression support #1000

Merged
merged 54 commits into from
May 10, 2022

Conversation

rajanya-google
Copy link
Member

@rajanya-google rajanya-google commented Feb 8, 2022

Adding gRPC compression support.

@rajanya-google rajanya-google requested a review from a team as a code owner February 8, 2022 17:13
@product-auto-label product-auto-label bot added the api: pubsub Issues related to the googleapis/java-pubsub API. label Feb 8, 2022
@rajanya-google rajanya-google requested review from pradn and removed request for pradn February 9, 2022 16:51
@rajanya-google rajanya-google requested a review from a team as a code owner February 9, 2022 22:01
@snippet-bot
Copy link

snippet-bot bot commented Feb 9, 2022

No region tags are edited in this PR.

This comment is generated by snippet-bot.
If you find problems with this result, please file an issue at:
https://github.com/googleapis/repo-automation-bots/issues.
To update this comment, add snippet-bot:force-run label or use the checkbox below:

  • Refresh this comment

@rajanya-google rajanya-google changed the title feat: Adding grpc compression feat: Adding grpc compression support Feb 9, 2022
Copy link
Contributor

@beltran beltran left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this PR was submitted, would the feature work, or are changes required in the server? If this is the case we probably would want to hold it off.

@rajanya-google rajanya-google requested a review from mmicatka March 2, 2022 17:29
Copy link
Contributor

@anguillanneuf anguillanneuf left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajanya-google Please don't add samples in the same PR.

I need to work with you to plan the samples in the internal tools. This feature also needs to be released first before the sample tests will pass. When writing samples, please open a branch from this repo instead of a fork.

I also suggest we provide a short code snippet in the docstring as part of the library documentation.

@rajanya-google
Copy link
Member Author

rajanya-google commented Mar 21, 2022

@rajanya-google Please don't add samples in the same PR.

I have removed sample for compression.

I need to work with you to plan the samples in the internal tools. This feature also needs to be released first before the sample tests will pass. When writing samples, please open a branch from this repo instead of a fork.

Sure.

I also suggest we provide a short code snippet in the docstring as part of the library documentation.

I don't quite follow this. Could you please point me to the location?

@anguillanneuf anguillanneuf self-requested a review March 24, 2022 17:49
@anguillanneuf anguillanneuf dismissed their stale review March 24, 2022 17:50

Thank you for removing samples are removed from this PR.

@anguillanneuf
Copy link
Contributor

anguillanneuf commented Mar 24, 2022

I also suggest we provide a short code snippet in the docstring as part of the library documentation.

I don't quite follow this. Could you please point me to the location?

Here's an example: https://cloud.google.com/java/docs/reference/google-cloud-pubsub/latest/com.google.cloud.pubsub.v1.PublisherInterface#methods

rajanya-google and others added 8 commits May 6, 2022 15:49
…onfig to v1.4.0 (googleapis#1105)

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [com.google.cloud:google-cloud-shared-config](https://togithub.com/googleapis/java-shared-config) | `1.3.3` -> `1.4.0` | [![age](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.4.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.4.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.4.0/compatibility-slim/1.3.3)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/com.google.cloud:google-cloud-shared-config/1.4.0/confidence-slim/1.3.3)](https://docs.renovatebot.com/merge-confidence/) |

---

### Release Notes

<details>
<summary>googleapis/java-shared-config</summary>

### [`v1.4.0`](https://togithub.com/googleapis/java-shared-config/blob/HEAD/CHANGELOG.md#&#8203;140-httpsgithubcomgoogleapisjava-shared-configcomparev133v140-2022-04-28)

[Compare Source](https://togithub.com/googleapis/java-shared-config/compare/v1.3.3...v1.4.0)

##### Features

-   **java:** remove native image module ([#&#8203;471](https://togithub.com/googleapis/java-shared-config/issues/471)) ([7fcba01](https://togithub.com/googleapis/java-shared-config/commit/7fcba016b3138d7beaa4e962853f9bc80f59438c))

##### [1.3.3](https://togithub.com/googleapis/java-shared-config/compare/v1.3.2...v1.3.3) (2022-04-19)

##### Bug Fixes

-   **java:** remove protobuf feature from native profile ([#&#8203;461](https://togithub.com/googleapis/java-shared-config/issues/461)) ([ffd07cb](https://togithub.com/googleapis/java-shared-config/commit/ffd07cb18ee7d45d4daee1d9ea6f6d321fdca874))

##### Dependencies

-   update dependency com.google.cloud:native-image-support to v0.12.11 ([#&#8203;459](https://togithub.com/googleapis/java-shared-config/issues/459)) ([d20008d](https://togithub.com/googleapis/java-shared-config/commit/d20008df15209708fdf9d06828b567778190f4d0))
-   update dependency com.google.cloud:native-image-support to v0.13.1 ([#&#8203;465](https://togithub.com/googleapis/java-shared-config/issues/465)) ([b202064](https://togithub.com/googleapis/java-shared-config/commit/b2020648816feb4740ad70acedfed470d7da5bcf))

##### [1.3.2](https://togithub.com/googleapis/java-shared-config/compare/v1.3.1...v1.3.2) (2022-03-28)

##### Dependencies

-   revert google-java-format to 1.7 ([#&#8203;453](https://togithub.com/googleapis/java-shared-config/issues/453)) ([cbc777f](https://togithub.com/googleapis/java-shared-config/commit/cbc777f3e9ab75edb6fa2e0268a7446ae4111725))

##### [1.3.1](https://togithub.com/googleapis/java-shared-config/compare/v1.3.0...v1.3.1) (2022-03-25)

##### Dependencies

-   update dependency com.google.cloud:native-image-support to v0.12.10 ([#&#8203;443](https://togithub.com/googleapis/java-shared-config/issues/443)) ([5b39d5e](https://togithub.com/googleapis/java-shared-config/commit/5b39d5ee15121f052226ff873b6ab101e9c71de5))
-   update dependency com.google.googlejavaformat:google-java-format to v1.15.0 ([#&#8203;426](https://togithub.com/googleapis/java-shared-config/issues/426)) ([4c3c4b6](https://togithub.com/googleapis/java-shared-config/commit/4c3c4b66129632181e6bc363a0ecccf4f5aac914))
-   update dependency org.graalvm.buildtools:junit-platform-native to v0.9.11 ([#&#8203;448](https://togithub.com/googleapis/java-shared-config/issues/448)) ([f7f518e](https://togithub.com/googleapis/java-shared-config/commit/f7f518e87d1d9feb9ac54d7c099f97d8751ee3da))
-   update dependency org.graalvm.buildtools:native-maven-plugin to v0.9.11 ([#&#8203;449](https://togithub.com/googleapis/java-shared-config/issues/449)) ([3e1c0b5](https://togithub.com/googleapis/java-shared-config/commit/3e1c0b5a1d2f4a0db88c06a0d683ed90cbc745e2))

</details>

---

### Configuration

📅 **Schedule**: At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-pubsub).
- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 444328399

Source-Link: googleapis/googleapis@c7ca416

Source-Link: googleapis/googleapis-gen@d617054
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiZDYxNzA1NDUzYTYyYjNlY2RhNzhhYTMwYzE5Mjg0MGViYzVhOGE5MCJ9

feat: AuditConfig for IAM v1

PiperOrigin-RevId: 439356405

Source-Link: googleapis/googleapis@afa2ba1

Source-Link: googleapis/googleapis-gen@3e40c17
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiM2U0MGMxN2UxNTEwYzk1ZmFiNThmYzIxNDNjY2I2MWNjZWNhNTk4OSJ9
…a to 2.16.0 (googleapis#1107)

- [ ] Regenerate this pull request now.

PiperOrigin-RevId: 446250659

Source-Link: googleapis/googleapis@dc4ef31

Source-Link: googleapis/googleapis-gen@5fdda4d
Copy-Tag: eyJwIjoiLmdpdGh1Yi8uT3dsQm90LnlhbWwiLCJoIjoiNWZkZGE0ZGRmYmFiODc5OThlNzdlNGE0NTNlMGZmODc5ODZkMmRiOCJ9
…-info-reports-plugin to v3.3.0 (googleapis#1104)

[![WhiteSource Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| [org.apache.maven.plugins:maven-project-info-reports-plugin](https://maven.apache.org/plugins/) ([source](https://togithub.com/apache/maven-project-info-reports-plugin)) | `3.2.2` -> `3.3.0` | [![age](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.3.0/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.3.0/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.3.0/compatibility-slim/3.2.2)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/maven/org.apache.maven.plugins:maven-project-info-reports-plugin/3.3.0/confidence-slim/3.2.2)](https://docs.renovatebot.com/merge-confidence/) |

---

### Configuration

📅 **Schedule**: At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [WhiteSource Renovate](https://renovate.whitesourcesoftware.com). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-pubsub).
@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels May 9, 2022
@rajanya-google rajanya-google changed the base branch from main to hannahrogers-google-patch-2 May 9, 2022 15:04
@rajanya-google rajanya-google changed the base branch from hannahrogers-google-patch-2 to main May 9, 2022 15:04
@rajanya-google rajanya-google changed the base branch from main to 1.111.0-sp May 9, 2022 15:06
@rajanya-google rajanya-google changed the base branch from 1.111.0-sp to main May 9, 2022 15:06
@rajanya-google rajanya-google changed the base branch from main to 1.114.x May 9, 2022 15:10
@rajanya-google rajanya-google changed the base branch from 1.114.x to main May 9, 2022 15:10
@rajanya-google rajanya-google added the owlbot:run Add this label to trigger the Owlbot post processor. label May 9, 2022
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 9, 2022
@product-auto-label product-auto-label bot added size: m Pull request size is medium. and removed size: l Pull request size is large. labels May 9, 2022
@rajanya-google rajanya-google added the owlbot:run Add this label to trigger the Owlbot post processor. label May 9, 2022
@gcf-owl-bot gcf-owl-bot bot removed the owlbot:run Add this label to trigger the Owlbot post processor. label May 9, 2022
Copy link
Contributor

@mmicatka mmicatka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@anguillanneuf anguillanneuf merged commit 4ad1a3f into googleapis:main May 10, 2022
MessageAndConsumer toAck2 = pollQueueMessageAndConsumer(receiveQueue);
toAck2.consumer().ack();

assertNotEquals(toAck1.message().getData(), toAck2.message().getData());
Copy link
Contributor

@anguillanneuf anguillanneuf May 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rajanya-google Can you help me understand the purpose of this test? I see two messages of different sizes getting published using a publisher with no batching configured, so each message is essentially published in its own batch of batch size 1, each message has a size greater than the compression threshold of 10 bytes, they will get compressed, then we read these messages and confirm that their data fields aren't equal (what's the point of that)?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the googleapis/java-pubsub API. size: m Pull request size is medium.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants