deprecate pubsub PullRequest returnImmediately flag

[lukasGemela]

Is your feature request related to a problem? Please describe.
whilst investigating problems with delayed pubsub messages we found out that our problems were caused by returnImmediately flag set to true. According to documentation this flag is actually deprecated server-side and they heavily discourage from using it now:
https://cloud.google.com/pubsub/docs/reference/rpc/google.pubsub.v1#pullrequest

Describe the solution you'd like
Either remove it or deprecate it using annotation.

Describe alternatives you've considered
No alternatives.

Additional context
We could also do the PR but I open this report to run decision whether keep the flag in deprecated state or remove it completely.

[dzou]

Thanks for the report. We had discussions of deprecating this in the past, though had trouble coordinating this change with the Pub/Sub team. Will follow-up and ask what is our long-term plan for this. @meltsufin @elefeint

[meltsufin]

Related to: spring-attic/spring-cloud-gcp#2247.
If I recall correctly, we decided to keep it last time we looked at it because there was no directly replacement for it.

[lukasGemela]

I think my biggest issue is that setting returnImmediately to true has some serious side effect for us (message delivery delayed for hours), the flag itself is deprecated and there is no mentioning in documentation or javaDoc. Spring template documentation should at least discourage people from using this flag

[meltsufin]

I think my biggest issue is that setting returnImmediately to true has some serious side effect for us (message delivery delayed for hours)

This sounds like a bug potentially. Can you elaborate how the use of this flag delays message delivery?

[lukasGemela]

I think my biggest issue is that setting returnImmediately to true has some serious side effect for us (message delivery delayed for hours)

This sounds like a bug potentially. Can you elaborate how the use of this flag delays message delivery?

sure, please read through this ticket
https://issuetracker.google.com/issues/180098468

Also google doc states

Warning: setting this field to true is discouraged because it adversely impacts the performance of subscriptions.pull operations. We recommend that users do not set this field.

[meltsufin]

@lukasGemela Thanks for the reference to the ticket. We'll work internally to figure out how to proceed and update this issue soon.

[meltsufin]

@lukasGemela The word from the Pub/Sub team is that the way returnImmediately=true works at the moment is that it waits up to 1 second before returning the pull call, and it's only kind of deprecated because there are still users who depend on its behavior and there is no direct replacement yet.
It's also not recommended because it will generally decrease throughput and latency.

An alternative is to use the streaming subscriber. Have you explored that?

[meltsufin]

I also got more details on why returnImmediately=true may cause very long delays in message delivery. Since it only waits for up to 1 second, it could happen that it is just not enough time to retrieve any messages internally from the Pub/Sub servers. This can happen repeatedly, causing very long delays. Hence, it's not recommended.

[lukasGemela]

@lukasGemela The word from the Pub/Sub team is that the way returnImmediately=true works at the moment is that it waits up to 1 second before returning the pull call, and it's only kind of deprecated because there are still users who depend on its behavior and there is no direct replacement yet.
It's also not recommended because it will generally decrease throughput and latency.

An alternative is to use the streaming subscriber. Have you explored that?

yes, I did, but that does not really fit our use case. But we just set returnImmediately flag to false and it works fine now, as stated under that google issue. I would at least add something into clients javadoc/documentation that setting this flag to true is not recommended and why 🤷‍♂️

[meltsufin]

Yes, we'll do that. Thanks for bringing up this issue so that we can document it better.