Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Pubsub java client has pub/ack latency greater than a second #1367

Closed
jrheizelman opened this issue Nov 2, 2016 · 6 comments
Closed

Pubsub java client has pub/ack latency greater than a second #1367

jrheizelman opened this issue Nov 2, 2016 · 6 comments
Assignees

Comments

@jrheizelman
Copy link

In comparative tests, the gcloud java client's publish latency is over 10x worse than a grpc client in java and the gcloud Python client. Even in relatively low-throughput tests, the client consistently takes 1-2 seconds for a publish call, when other tests are getting results on the order of tens of milliseconds. at the same message size/QPS.

@mziccard
Copy link
Contributor

mziccard commented Nov 2, 2016

Even in relatively low-throughput tests, the client consistently takes 1-2 seconds for a publish call

This is very strange. Can you share the code you are using for testing? I just run a quick test and I am getting publish times in the range 50-60milliseconds, which is in line with other user reports.

@jrheizelman
Copy link
Author

Sure, we've been using it in a load testing framework here:

https://github.com/GoogleCloudPlatform/pubsub/blob/master/load-test-framework/src/main/java/com/google/pubsub/clients/gcloud/CPSPublisherTask.java

The call is:
Stopwatch stopwatch = Stopwatch.createStarted();
pubSub.publish(topic, messages);
stopwatch.stop();

We've been getting those numbers pretty consistently, even when there's only 1 thread sending the publish requests in at a time. In the same framework, a grpc client (if you want to see it, it's present in the grpc branch of the same repo) is getting 100-150ms pub/ack latency on the same test, same topic as the gcloud one.

@mziccard
Copy link
Contributor

mziccard commented Nov 2, 2016

@jrheizelman can you clarify what you mean with "pub/ack latency"? Are you measuring the total time it takes for a message to be published and then received?

@jrheizelman
Copy link
Author

No, the time it takes for the publish call to complete (i.e. be acked by the server). In this case, the time elapsed on the stopwatch in the call in my above post, so the amount of time the publish call blocks for.

Similarly, when we tried using the async publish call to see where the slowdown was taking place, the future was returned almost immediately, as you'd expect, but then the get() call took a second+

@jrheizelman jrheizelman changed the title Pubsub java client has terrible pub/ack latency Pubsub java client has pub/ack latency greater than a second Nov 3, 2016
@mziccard
Copy link
Contributor

mziccard commented Nov 3, 2016

@jrheizelman can you share with us the command you are using to start the test? (so we can compare data on a common config)

@jrheizelman
Copy link
Author

Closing this issue because from communication with the pubsub teams, this is a known issue that's being worked on.

github-actions bot pushed a commit to suztomo/google-cloud-java that referenced this issue Jun 29, 2022
* chore: update java template

* chore: fix tests

* chore: fix tests

* chore: update project
Source-Link: googleapis/synthtool@1155a97
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:b0b1c1c89570e229b1026372a2b8989ba31495007055b8d30178b7648503eefa
github-actions bot pushed a commit that referenced this issue Jul 1, 2022
* chore: update java template

* chore: fix tests

* chore: fix tests

* chore: update project
Source-Link: googleapis/synthtool@1155a97
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:b0b1c1c89570e229b1026372a2b8989ba31495007055b8d30178b7648503eefa
github-actions bot pushed a commit that referenced this issue Sep 30, 2022
[![Mend Renovate](https://app.renovatebot.com/images/banner.svg)](https://renovatebot.com)

This PR contains the following updates:

| Package | Change | Age | Adoption | Passing | Confidence |
|---|---|---|---|---|---|
| gcp-releasetool | `==1.8.7` -> `==1.8.8` | [![age](https://badges.renovateapi.com/packages/pypi/gcp-releasetool/1.8.8/age-slim)](https://docs.renovatebot.com/merge-confidence/) | [![adoption](https://badges.renovateapi.com/packages/pypi/gcp-releasetool/1.8.8/adoption-slim)](https://docs.renovatebot.com/merge-confidence/) | [![passing](https://badges.renovateapi.com/packages/pypi/gcp-releasetool/1.8.8/compatibility-slim/1.8.7)](https://docs.renovatebot.com/merge-confidence/) | [![confidence](https://badges.renovateapi.com/packages/pypi/gcp-releasetool/1.8.8/confidence-slim/1.8.7)](https://docs.renovatebot.com/merge-confidence/) |

---

### Configuration

📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined).

🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied.

♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox.

🔕 **Ignore**: Close this PR and you won't be reminded about this update again.

---

 - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, click this checkbox.

---

This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://app.renovatebot.com/dashboard#github/googleapis/java-asset).
<!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzMi4yMDguMCIsInVwZGF0ZWRJblZlciI6IjMyLjIwOC4wIn0=-->
suztomo pushed a commit that referenced this issue Feb 1, 2023
* chore: update java template

* chore: fix tests

* chore: fix tests

* chore: update project
Source-Link: googleapis/synthtool@1155a97
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:b0b1c1c89570e229b1026372a2b8989ba31495007055b8d30178b7648503eefa
suztomo pushed a commit that referenced this issue Feb 1, 2023
* chore: update java template

* chore: fix tests

* chore: fix tests

* chore: update project
Source-Link: googleapis/synthtool@1155a97
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:b0b1c1c89570e229b1026372a2b8989ba31495007055b8d30178b7648503eefa
suztomo pushed a commit that referenced this issue Feb 1, 2023
* chore: update java template

* chore: fix tests

* chore: fix tests

* chore: update project
Source-Link: googleapis/synthtool@1155a97
Post-Processor: gcr.io/cloud-devrel-public-resources/owlbot-java:latest@sha256:b0b1c1c89570e229b1026372a2b8989ba31495007055b8d30178b7648503eefa

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants