Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken flow control in rare cases #80

Closed
mmurygin opened this issue Feb 28, 2018 · 1 comment
Closed

Broken flow control in rare cases #80

mmurygin opened this issue Feb 28, 2018 · 1 comment
Assignees
Labels
api: pubsub Issues related to the googleapis/nodejs-pubsub API. priority: p2 Moderately-important priority. Fix may not be included in next release. 🚨 This issue needs some love. type: question Request for information or clarification. Not an issue.

Comments

@mmurygin
Copy link

In production I have a memory consuming task. So if I get two messages at a time my app runs out of memory. I found out that flow control settings do not always works.

Environment details

  • OS: Ubuntu 16.04
  • Node.js version: 8.9.4
  • npm version: 5.5.1
  • @google-cloud/pubsub version: 0.16.4

I created a repository with code to reproduce this issue.

Steps to reproduce

  1. Clone repository https://github.com/muryginm/google-cloud-pubsub-issues
  2. npm install
  3. GOOGLE_PROJECT=your-project-id GOOGLE_APPLICATION_CREDENTIALS=path-to-credentials npm run flow-control
  4. Repeat step 3 a couple of times. At least one of 5 tests will fail.

In source code I created topic, subscription and track for messages that are currently in process. Am I doing something wrong?

Thanks!

@mmurygin mmurygin changed the title Broken flow control in rare cases Broken flow control in rare cases [bug] Feb 28, 2018
@mmurygin mmurygin changed the title Broken flow control in rare cases [bug] Broken flow control in rare cases Feb 28, 2018
@callmehiphop
Copy link
Contributor

@muryginm based on some recent dialogue shared with me from the PubSub team, there is no guarantee that maxMessages will be hit exactly. That's because the PubSub API does not accept such an option and sometimes will give us batches of messages that can be higher than the desired limit.

As far as the increased memory usage, I've been chasing down similar reports (#13) and hope to have a fix out soon.

I'm going to close this issue because I think the main issue being described is duplicative of another issue, but if you think this in incorrect, let me know and I'll be happy to re-open it.

@callmehiphop callmehiphop added priority: p2 Moderately-important priority. Fix may not be included in next release. type: question Request for information or clarification. Not an issue. labels Feb 28, 2018
@google-cloud-label-sync google-cloud-label-sync bot added the api: pubsub Issues related to the googleapis/nodejs-pubsub API. label Jan 31, 2020
@yoshi-automation yoshi-automation added the 🚨 This issue needs some love. label Apr 6, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: pubsub Issues related to the googleapis/nodejs-pubsub API. priority: p2 Moderately-important priority. Fix may not be included in next release. 🚨 This issue needs some love. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

3 participants