Streaming lists documentation #39993

wojtek-t · 2023-03-14T08:50:32Z

netlify · 2023-03-14T09:02:41Z

✅ Pull request preview available for checking

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`f02bfac`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-main-staging/deploys/64199bf8f2ff270008903625
😎 Deploy Preview	https://deploy-preview-39993--kubernetes-io-main-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site settings.

sftim · 2023-03-14T09:18:11Z

/sig api-machinery

wojtek-t · 2023-03-20T09:16:26Z

/assign @lavalamp

@lavalamp - can you please review from the technical POV?

content/en/docs/reference/using-api/api-concepts.md

lavalamp · 2023-03-20T20:09:58Z

/lgtm

looks great, just grammar nits :)

k8s-ci-robot · 2023-03-20T20:10:05Z

LGTM label has been added.

Git tree hash: 015155382d9e65681ab441470d253f45f2b6c3b0

wojtek-t · 2023-03-21T09:16:00Z

@lavalamp - thanks! comments applied, PTAL

wojtek-t · 2023-03-21T09:16:31Z

@kubernetes/sig-docs-en-owners - PTAL

k8s-ci-robot · 2023-03-22T09:39:28Z

LGTM label has been added.

Git tree hash: 6dde35a44617a60d7096c4d03aad019ba5a03bec

tallclair

Not needed for this PR, but the api-concepts page is getting quite long. I think it might make sense to pull all the watch content out to a separate page.

tallclair · 2023-03-24T16:40:22Z

content/en/docs/reference/using-api/api-concepts.md

+Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter
+in a **watch** request. If set, the API server starts the watch stream with synthetic init
+events to build the whole state of all existing objects followed by a `BOOKMARK` event


Add an anchor link to #watch-bookmarks here.

tallclair · 2023-03-24T16:40:50Z

content/en/docs/reference/using-api/api-concepts.md

+(if requested via `allowWatchBookmarks=true` option). The bookmark event includes the resource for that
+watch bookmark. After sending the bookmark event, the API server continues as for any other **watch**


The bookmark event includes the resource for that watch bookmark

What does this mean?

tallclair · 2023-03-24T16:43:23Z

content/en/docs/reference/using-api/api-concepts.md

+(if requested via `allowWatchBookmarks=true` option). The bookmark event includes the resource for that
+watch bookmark. After sending the bookmark event, the API server continues as for any other **watch**


Is the apiserver guaranteed to send all initial events before any other watch events?

tallclair · 2023-03-24T16:44:35Z

content/en/docs/reference/using-api/api-concepts.md

+
+Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter
+in a **watch** request. If set, the API server starts the watch stream with synthetic init


synthetic init events

Are these always type: ADDED? Maybe add a note about that here.

yes - added

tallclair · 2023-03-24T16:48:20Z

content/en/docs/reference/using-api/api-concepts.md

+Content-Type: application/json
+
+{
+  "type": "ADDED",


(aside) Did you consider adding a new event type for the synthetic init events? I could imagine a case where a controller might want to use WatchList to initialize it's cache, but also takes a special action when resources are added that shouldn't be performed on the initialization events.

Yes - it's not a good idea.
Because technically watch with RV=0 was supported from k8s 1.0 version and that also had ADDED events. Changing that would introduce more confusion or incompatibility than it's worth it.

And the bookmark give you exactly what you need in case you want to handle them differently (which btw isn't clear - we changed reflector to it (I mean optionally now) and having ADDED is good enough).

netlify · 2023-03-27T08:53:26Z

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

Name	Link
🔨 Latest commit	`2bf6e39`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-vnext-staging/deploys/64219ccfea664b0008be5b4e

wojtek-t

PTAL

@sftim - can you please approve if you're ok with it?

wojtek-t · 2023-03-27T08:51:14Z

content/en/docs/reference/using-api/api-concepts.md

+
+Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter
+in a **watch** request. If set, the API server starts the watch stream with synthetic init


yes - added

wojtek-t · 2023-03-27T08:51:19Z

content/en/docs/reference/using-api/api-concepts.md

+Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter
+in a **watch** request. If set, the API server starts the watch stream with synthetic init
+events to build the whole state of all existing objects followed by a `BOOKMARK` event


wojtek-t · 2023-03-27T08:51:24Z

content/en/docs/reference/using-api/api-concepts.md

+(if requested via `allowWatchBookmarks=true` option). The bookmark event includes the resource for that
+watch bookmark. After sending the bookmark event, the API server continues as for any other **watch**


wojtek-t · 2023-03-27T08:51:28Z

content/en/docs/reference/using-api/api-concepts.md

+(if requested via `allowWatchBookmarks=true` option). The bookmark event includes the resource for that
+watch bookmark. After sending the bookmark event, the API server continues as for any other **watch**


wojtek-t · 2023-03-27T08:52:57Z

content/en/docs/reference/using-api/api-concepts.md

+Content-Type: application/json
+
+{
+  "type": "ADDED",


Yes - it's not a good idea.
Because technically watch with RV=0 was supported from k8s 1.0 version and that also had ADDED events. Changing that would introduce more confusion or incompatibility than it's worth it.

And the bookmark give you exactly what you need in case you want to handle them differently (which btw isn't clear - we changed reflector to it (I mean optionally now) and having ADDED is good enough).

tengqm · 2023-03-27T09:19:30Z

content/en/docs/reference/using-api/api-concepts.md

+On large clusters, retrieving the collection of some resource types may result in
+a significant increase of resource usage (primarily RAM) on the control plane.
+In order to alleviate its impact and simplify the user experience of the **list** + **watch**
+pattern, Kubernetes - including version {{< skew currentVersion >}} - provides alpha support


Is it possible to make this statement ever-green? For example,

Suggested change

pattern, Kubernetes - including version {{< skew currentVersion >}} - provides alpha support

pattern, Kubernetes v1.27 introduces as alpha feature the support

content/en/docs/reference/using-api/api-concepts.md

wojtek-t · 2023-03-27T09:42:38Z

@tengqm - thanks PTAL

tengqm · 2023-03-27T10:59:56Z

/lgtm

k8s-ci-robot · 2023-03-27T11:00:05Z

LGTM label has been added.

Git tree hash: 6a0605a876c835898310f4c0009a46e330e1350a

sftim · 2023-03-27T13:21:11Z

content/en/docs/reference/using-api/api-concepts.md

@@ -226,6 +226,64 @@ As a client, you can request `BOOKMARK` events by setting the
 assume bookmarks are returned at any specific interval, nor can clients assume that
 the API server will send any `BOOKMARK` event even when requested.

+## Streaming lists {#watch-bookmarks}


This will shadow the ## Watch bookmarks heading earlier.
I have a suspicion this was my suggestion; anyway, I suggest:

Suggested change

## Streaming lists {#watch-bookmarks}

### Streaming lists {#watch-bookmark-initial-events}

The new text becomes part of “Efficient detection of changes”.

If we need to reframe that 2nd-level heading, perhaps to be “Data streams”, “Streamed collection data” or something else more generic, then we can do so.

Is there a case where a client does a watch with sendInitialEvents other than to end up running the watch? (Is there a reasonable case where client might want to abort the watch as soon as it sees the first BOOKMARK event?)

That wasn't your suggestion - it was @tallclair 's one.

But I messed it up - it should be for "watch bookmarks" section. Should be fixed now.

Is there a reasonable case where client might want to abort the watch as soon as it sees the first BOOKMARK event?

There might be a reasonable usecase where we actually want to stream the data without storing everything in memory. So yes - there are usecases (although we don't want to endorse them yet - probably in the future).

sftim

Given that this won't be part of efficient detection of changes, we should help readers learn about watches. They may land directly on this section without seeing the context earlier in the page.

/lgtm
this is already alpha quality

It'd be nice to tweak further

sftim · 2023-03-27T13:47:35Z

content/en/docs/reference/using-api/api-concepts.md

+pattern, Kubernetes v1.27 introduces as an alpha feature the support
+for requesting the initial state (previously requested via the **list** request) as part of
+the **watch** request.


Suggested change

pattern, Kubernetes v1.27 introduces as an alpha feature the support

for requesting the initial state (previously requested via the **list** request) as part of

the **watch** request.

pattern, Kubernetes {{< skew currentVersion >}} allows you to fetch the contents of a collection

as part of a **watch** request. See [Efficient detection of changes](#efficient-detection-of-changes)

to learn more about watches in Kubernetes.

As a client, you can either continue the **watch** to subscribe to further event updates, or you can

use this mechanism purely as a more efficient alternative to **list** a collection.

pattern, Kubernetes {{< skew currentVersion >}}

Please figure it out - I got the opposite comment from @tengqm - I'm with this project for 8y+, but it can be extremely annoying if different reviewers are giving contradicting comments.

Also - as I mentioend in the other comment I don't want to endorse this as a more efficient alternative to LIST yet - we're not ready for that. So let's not try to recommend things we're not ready for.

On the recommend vs. not, feel free to frame this as experimental rather than as more efficient. It's good if a reader can still see under what conditions they might try it out.

We don't have a clear style guideline for when to mention the current version or not. Our overall guidance is to aim for timeless documentation.
Sorry about the mixed messages. I know I've got a particular way of writing for maintainability, and because this is text not code, there's no single right way to do it.

Here's a test: imagine that what you write is still present in Kubernetes 1.42 and the feature is still alpha. Does it make sense and feel current for a 1.42 release? If it clearly does, it's probably fine.

sftim · 2023-03-27T13:50:41Z

content/en/docs/reference/using-api/api-concepts.md

+Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
+is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter


Suggested change

Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)

is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter

Provided your cluster is running at least Kubernetes v1.27 and you have enabled the

the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/), you can

perform a streaming list by specifying `sendInitialEvents=true` as query string parameter

The 1.27 version is mentioned 2 lines above (in addition to the header of the section). I don't think it makes sense to constantly repeat it - for me as a reader it would be more annoying than useful.

I'm proposing that this (and the feature state shortcode, which lots of readers don't fully notice) is where we make the requirements clear.

That way, we're documenting in one place both:

the version requirement

the need to explicitly enable a feature gate

My experience suggests putting these details in 2 places is likely to lead to more GitHub issues telling us the docs are wrong (even when the docs are strictly correct and have been misunderstood).

k8s-ci-robot · 2023-03-27T13:52:38Z

LGTM label has been added.

Git tree hash: 406e5ebd1042de1a64d850222582429fa62a9b93

sftim

/lgtm

Comfortably good enough for alpha

tengqm · 2023-03-27T15:37:27Z

/approve

k8s-ci-robot · 2023-03-27T15:37:38Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: tengqm

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~content/en/OWNERS~~ [tengqm]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 14, 2023

k8s-ci-robot requested review from lavalamp and liggitt March 14, 2023 08:50

k8s-ci-robot added language/en Issues or PRs related to English language cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. sig/docs Categorizes an issue or PR as relevant to SIG Docs. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 14, 2023

k8s-ci-robot added the sig/api-machinery Categorizes an issue or PR as relevant to SIG API Machinery. label Mar 14, 2023

wojtek-t mentioned this pull request Mar 15, 2023

Allow informers for getting a stream of data instead of chunking kubernetes/enhancements#3157

Open

25 tasks

wojtek-t force-pushed the send_initial_events branch from f1aab87 to 0d4cd27 Compare March 20, 2023 09:14

k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Mar 20, 2023

wojtek-t force-pushed the send_initial_events branch from 0d4cd27 to 2cd0599 Compare March 20, 2023 09:15

wojtek-t changed the title ~~[WIP] Streaming lists documentation~~ Streaming lists documentation Mar 20, 2023

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Mar 20, 2023

k8s-ci-robot assigned lavalamp Mar 20, 2023

lavalamp reviewed Mar 20, 2023

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

lavalamp reviewed Mar 20, 2023

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

lavalamp reviewed Mar 20, 2023

View reviewed changes

content/en/docs/reference/using-api/api-concepts.md Outdated Show resolved Hide resolved

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 20, 2023

wojtek-t force-pushed the send_initial_events branch from 2cd0599 to cf4a450 Compare March 21, 2023 09:15

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 21, 2023

k8s-ci-robot requested a review from lavalamp March 21, 2023 09:15

k8s-ci-robot assigned tengqm Mar 22, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 22, 2023

tallclair reviewed Mar 24, 2023

View reviewed changes

wojtek-t force-pushed the send_initial_events branch from 748d837 to 94b1049 Compare March 27, 2023 08:53

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 27, 2023

k8s-ci-robot requested a review from tengqm March 27, 2023 08:53

wojtek-t commented Mar 27, 2023

View reviewed changes

tengqm reviewed Mar 27, 2023

View reviewed changes

wojtek-t force-pushed the send_initial_events branch from 94b1049 to 023a0ee Compare March 27, 2023 09:42

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 27, 2023

sftim reviewed Mar 27, 2023

View reviewed changes

Streaming lists documentation

2bf6e39

wojtek-t force-pushed the send_initial_events branch from 023a0ee to 2bf6e39 Compare March 27, 2023 13:40

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 27, 2023

k8s-ci-robot requested a review from tengqm March 27, 2023 13:40

sftim reviewed Mar 27, 2023

View reviewed changes

k8s-ci-robot assigned sftim Mar 27, 2023

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Mar 27, 2023

sftim reviewed Mar 27, 2023

View reviewed changes

k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Mar 27, 2023

k8s-ci-robot merged commit f23cc95 into kubernetes:dev-1.27 Mar 27, 2023

k8s-ci-robot added this to the 1.27 milestone Mar 27, 2023

		(if requested via `allowWatchBookmarks=true` option). The bookmark event includes the resource for that
		watch bookmark. After sending the bookmark event, the API server continues as for any other watch

	pattern, Kubernetes - including version {{< skew currentVersion >}} - provides alpha support
	pattern, Kubernetes v1.27 introduces as alpha feature the support

	## Streaming lists {#watch-bookmarks}
	### Streaming lists {#watch-bookmark-initial-events}

-pattern, Kubernetes v1.27 introduces as an alpha feature the support
-for requesting the initial state (previously requested via the **list** request) as part of
-the **watch** request.
+pattern, Kubernetes {{< skew currentVersion >}} allows you to fetch the contents of a collection
+as part of a **watch** request. See [Efficient detection of changes](#efficient-detection-of-changes)
+to learn more about watches in Kubernetes.
+As a client, you can either continue the **watch** to subscribe to further event updates, or you can
+use this mechanism purely as a more efficient alternative to **list** a collection.

		Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
		is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter

-Provided that the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/)
-is enabled, this can be achieved by specifying `sendInitialEvents=true` as query string parameter
+Provided your cluster is running at least Kubernetes v1.27 and you have enabled the
+the `WatchList` [feature gate](/docs/reference/command-line-tools-reference/feature-gates/), you can
+perform a streaming list by specifying `sendInitialEvents=true` as query string parameter

Streaming lists documentation #39993

Streaming lists documentation #39993

Conversation

wojtek-t commented Mar 14, 2023

netlify bot commented Mar 14, 2023 • edited Loading

✅ Pull request preview available for checking

sftim commented Mar 14, 2023

wojtek-t commented Mar 20, 2023

lavalamp commented Mar 20, 2023

k8s-ci-robot commented Mar 20, 2023

wojtek-t commented Mar 21, 2023

wojtek-t commented Mar 21, 2023

k8s-ci-robot commented Mar 22, 2023

tallclair left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify bot commented Mar 27, 2023 • edited Loading

👷 Deploy Preview for kubernetes-io-vnext-staging processing.

wojtek-t left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

wojtek-t commented Mar 27, 2023

tengqm commented Mar 27, 2023

k8s-ci-robot commented Mar 27, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Mar 27, 2023

sftim left a comment

Choose a reason for hiding this comment

tengqm commented Mar 27, 2023

k8s-ci-robot commented Mar 27, 2023

netlify bot commented Mar 14, 2023 •

edited

Loading

netlify bot commented Mar 27, 2023 •

edited

Loading