flow_control: Refactor setWatermark

[tyxia]

Context
In upcoming flow control part 3 PR where downstream pushes back sidestream, sidestream needs to subscribe to downstream watermark events (via addDownstreamWatermarkCallbacks), so that back pressure can be invoked when when the downstream buffers are overrun

Problem Statement
The subscription（i.e.，addDownstreamWatermarkCallbacks)happens on pool ready which is initiated by client (e.g., ext_proc)'s startStream. However, the setWatermark happens after it (one line below). This leads to problem that addDownstreamWatermarkCallbacks in sidestream failed because watermark is not set

Tracing example of the repro :

Solution

Move setWatermark earlier right after stream has been initiated/started successfully . The watermark callbacks are passed by streamOptions

Risk Level: LOW? runtime gard.
Testing: ext_proc + gRPC integration test
Docs Changes: N/A
Release Notes: N/A
Platform Specific Features: N/A

[repokitteh-read-only]

As a reminder, PRs marked as draft will not be automatically assigned reviewers,
or be handled by maintainer-oncall triage.

Please mark your PR as ready when you want it to be reviewed!

🐱

Caused by: #36738 was opened by tyxia.

see: more, trace.

[tyxia]

/assign @alyssawilk @yanavlasov