Expose client tools auto update for find endpoint #46785

vapopov · 2024-09-19T20:28:15Z

In this PR added exposing client auto update fields (is enabled, and required client tools version for update)

Related: https://github.com/gravitational/teleport/pull/39805/files#diff-8d3c0408dd826f27e9f13cd15f09f4d7e89202862f0b13ce131a2e3072c2a40dR49-R53

rosstimothy

Can you add test coverage?

rosstimothy · 2024-09-19T20:44:30Z

lib/web/apiserver.go

-	}, nil
+	}
+
+	autoUpdateConfig, err := h.GetAccessPoint().GetAutoUpdateConfig(r.Context())


nit: the auth preference above is retrieved using h.cfg.AccessPoint, let's pick one and be consistent

rosstimothy · 2024-09-19T20:47:25Z

api/client/webclient/webclient.go

+	// ToolsVersion defines the version of {tsh, tctl} for client auto update.
+	ToolsVersion string `json:"tools_version"`
+	// ToolsAutoupdate enables client auto update feature.
+	ToolsAutoupdate bool `json:"tools_autoupdate"`


Should the auto update information all be contained within another struct instead of adding each individual field to the response?

grouping fields to another struct would be better, will change this one

rosstimothy · 2024-09-19T20:52:54Z

lib/web/apiserver.go

+	}
+
+	autoUpdateConfig, err := h.GetAccessPoint().GetAutoUpdateConfig(r.Context())
+	if err != nil && !trace.IsNotFound(err) {


I think it may also be possible to get a NotImplemented error back if the Proxy is newer than Auth. Even though this should be reading from the cache most of the time, a cluster admin can disable the cache, in which case this would make an RPC request to the old auth instance that doesn't implement the autoupdate service.

rosstimothy · 2024-09-19T20:54:22Z

lib/web/apiserver.go

+		return nil, trace.Wrap(err)
+	} else if err == nil {
+		response.ToolsAutoupdate = autoUpdateConfig.GetSpec().GetToolsAutoupdate()
+	}
+
+	autoUpdateVersion, err := h.GetAccessPoint().GetAutoUpdateVersion(r.Context())
+	if err != nil && !trace.IsNotFound(err) {
+		return nil, trace.Wrap(err)


Should failing to retrieve auto update information prevent the response entirely?

I added logging instead, but we might generate a lot of log messages with this error message in frequent requests to this endpoint

Log error instead returning error Add tests auto update settings in find endpoint Add check for not implemented error

rosstimothy · 2024-09-30T17:47:22Z

api/client/webclient/webclient.go

+	// ToolsVersion defines the version of {tsh, tctl} for client auto update.
+	ToolsVersion string `json:"tools_version"`
+	// ToolsAutoupdate enables client auto update feature.
+	ToolsAutoupdate bool `json:"tools_autoupdate"`


The json tag for the AutoUpdate field in the PingResponse is auto_update - we should be consistent with the tag here.

Suggested change

ToolsAutoupdate bool `json:"tools_autoupdate"`

ToolsAutoUpdate bool `json:"tools_auto_update"`

We should decide if we want to treat auto-update as a single hyphenated word (toolsAutoupdate/tools_autoupdate) or an abbreviated way of writing "automatic update" (toolAutoUpdate/tools_auto_update), and then be consistent about this.

Seems like @rosstimothy is suggesting two words, so maybe we move all of the JSON/YAML to the two-word form, and reserve autoupdate for the package, resource, and subcommand naming (autoupdate_config, autoupdate package, tctl autoupdate client) exclusively?

I was just trying to suggest to use consistent naming with https://github.com/gravitational/teleport/pull/46785/files#diff-c8c1c8dcbc610e7c6d2b2af815e65b0016525745ea97cc99791290e3df7e35e2R301.

rosstimothy · 2024-09-30T17:49:50Z

lib/web/apiserver.go

+	}
+
+	autoUpdateConfig, err := h.cfg.AccessPoint.GetAutoUpdateConfig(r.Context())
+	// TODO(vapopov) DELETE IN v18.0.0


Suggestion: make the deletion note clearer that its only the not found and not implemented checks that should be removed in v18

rosstimothy · 2024-09-30T17:49:54Z

lib/web/apiserver.go

+	}
+
+	autoUpdateVersion, err := h.cfg.AccessPoint.GetAutoUpdateVersion(r.Context())
+	// TODO(vapopov) DELETE IN v18.0.0


Suggestion: make the deletion note clearer that its only the not found and not implemented checks that should be removed in v18

rosstimothy · 2024-09-30T17:51:39Z

lib/web/apiserver_ping_test.go

+	assert.True(t, pr.AutoUpdate.ToolsAutoupdate, "tools_autoupdate must be enabled")
+	assert.Equal(t, teleport.Version, pr.AutoUpdate.ToolsVersion, "tools_version must be equal to current version")


Suggestion: decouple the json tags from the message

Suggested change

assert.True(t, pr.AutoUpdate.ToolsAutoupdate, "tools_autoupdate must be enabled")

assert.Equal(t, teleport.Version, pr.AutoUpdate.ToolsVersion, "tools_version must be equal to current version")

assert.True(t, pr.AutoUpdate.ToolsAutoupdate, "tools auto update must be enabled")

assert.Equal(t, teleport.Version, pr.AutoUpdate.ToolsVersion, "tools version must be equal to current version")

rosstimothy · 2024-09-30T17:52:30Z

lib/web/apiserver_ping_test.go

+	_, err = env.server.Auth().UpsertAutoUpdateVersion(ctx, version)
+	require.NoError(t, err)
+
+	req, err := http.NewRequest(http.MethodGet, proxy.newClient(t).Endpoint("webapi", "find"), nil)


Suggestion: add a test case that covers the default values when none of the auto update configurations are populated or there is an error retrieving the configurations?

public-teleport-github-review-bot · 2024-10-04T16:24:13Z

@vapopov See the table below for backport results.

Branch	Result
branch/v14	Failed
branch/v15	Failed
branch/v16	Failed

* Expose client tools auto update for find endpoint * Group auto update settings in find response Log error instead returning error Add tests auto update settings in find endpoint Add check for not implemented error * Add more test cases

* Expose client tools auto update for find endpoint (#46785) * Expose client tools auto update for find endpoint * Group auto update settings in find response Log error instead returning error Add tests auto update settings in find endpoint Add check for not implemented error * Add more test cases * Client AutoUpdate proto structure changes (#47532) * Update client autoupdate proto structure * Replace with reserved * Fix unit tests * Add more info in proto * Rename proto to be aligned RFD namings * Replace enum type for ToolsMode to string * Add packaging utility for client tools auto updates (#47060) * Add packaging utility for client tools auto updates * Add error handling for close functions * Move archive to existing utils package * Move archive helpers to integration/helper CR changes * CR changes * CR changes * CR changes Replace creating directory with extract path as argument * CR changes * Validate full size before un-archive Extract files to extractDir with ignore dir structure * Change compressing with relative paths Add test for cleanup and fix skip logic * CR changes * CR changes * Fix linter * Client tools auto update (#47466) * Add client tools auto update * Replace fork for posix platform for re-exec Move integration tests to client tools specific dir Use context cancellation with SIGTERM, SIGINT Remove cancelable tee reader with context replacement Renaming * Fix syscall path execution Fix archive cleanup if hash is not valid Limit the archive write bytes * Cover the case with single package for darwin platform after v17 * Move updater logic to tools package * Move context out from the library Base URL renaming * Add more context in comments * Changes in find endpoint * Replace test http server with `httptest` Replace hash for bytes matching Proper temp file close for archive download * Add more context to comments * Move feature flag to main package to be reused * Constant rename * Replace build tag with lib/modules to identify enterprise build * Replace fips tag with modules flag * Client auto updates integration for {tctl,tsh} (#47815) * Client auto updates integration for tctl/tsh * Add version validation Fix recursive version check for darwin platform Fix cleanup for multi-package support * Fix identifying tools removal from home directory * Replace ToolsMode with ToolsAutoUpdate * Reuse insecure flag for tests * Fix CheckRemote with login * Fix windows administrative access requirement Update must be able to be canceled, re-execute with latest version or last updated Show progress bar before request is made * Fix update cancellation for login action Address review comments * Add signal handler with stack context cancellation * Use copy instead of hard link for windows Fix progress bar if we can't receive size of package * Replace with list in order to support manual cancel * Download archive package to temp directory * Decrease timeout for client tools proxy call * Add audit logs for auto update resources (#48218) * Connect: Make sure tsh auto-updates are turned off (#49180) * Add dir for code shared between Node.js processes * Connect: Make sure tsh auto-updates are turned off * Pass TELEPORT_TOOLS_VERSION=off to tsh vnet-daemon * Disable client tools auto update disabled if there are no home dir (#49159) Move updater to general tools package * Move client auto update helper to lib package (#49247) --------- Co-authored-by: Rafał Cieślak <[email protected]>

* Expose client tools auto update for find endpoint (#46785) * Expose client tools auto update for find endpoint * Group auto update settings in find response Log error instead returning error Add tests auto update settings in find endpoint Add check for not implemented error * Add more test cases * Client AutoUpdate proto structure changes (#47532) * Update client autoupdate proto structure * Replace with reserved * Fix unit tests * Add more info in proto * Rename proto to be aligned RFD namings * Replace enum type for ToolsMode to string * Add packaging utility for client tools auto updates (#47060) * Add packaging utility for client tools auto updates * Add error handling for close functions * Move archive to existing utils package * Move archive helpers to integration/helper CR changes * CR changes * CR changes * CR changes Replace creating directory with extract path as argument * CR changes * Validate full size before un-archive Extract files to extractDir with ignore dir structure * Change compressing with relative paths Add test for cleanup and fix skip logic * CR changes * CR changes * Fix linter * Client tools auto update (#47466) * Add client tools auto update * Replace fork for posix platform for re-exec Move integration tests to client tools specific dir Use context cancellation with SIGTERM, SIGINT Remove cancelable tee reader with context replacement Renaming * Fix syscall path execution Fix archive cleanup if hash is not valid Limit the archive write bytes * Cover the case with single package for darwin platform after v17 * Move updater logic to tools package * Move context out from the library Base URL renaming * Add more context in comments * Changes in find endpoint * Replace test http server with `httptest` Replace hash for bytes matching Proper temp file close for archive download * Add more context to comments * Move feature flag to main package to be reused * Constant rename * Replace build tag with lib/modules to identify enterprise build * Replace fips tag with modules flag * Client auto updates integration for {tctl,tsh} (#47815) * Client auto updates integration for tctl/tsh * Add version validation Fix recursive version check for darwin platform Fix cleanup for multi-package support * Fix identifying tools removal from home directory * Replace ToolsMode with ToolsAutoUpdate * Reuse insecure flag for tests * Fix CheckRemote with login * Fix windows administrative access requirement Update must be able to be canceled, re-execute with latest version or last updated Show progress bar before request is made * Fix update cancellation for login action Address review comments * Add signal handler with stack context cancellation * Use copy instead of hard link for windows Fix progress bar if we can't receive size of package * Replace with list in order to support manual cancel * Download archive package to temp directory * Decrease timeout for client tools proxy call * Add audit logs for auto update resources (#48218) * Connect: Make sure tsh auto-updates are turned off * Add dir for code shared between Node.js processes * Connect: Make sure tsh auto-updates are turned off * Pass TELEPORT_TOOLS_VERSION=off to tsh vnet-daemon * Disable client tools auto update disabled if there are no home dir (#49159) Move updater to general tools package * Move client auto update helper to lib package (#49247) --------- Co-authored-by: Rafał Cieślak <[email protected]>

vapopov added backport/branch/v14 no-changelog Indicates that a PR does not require a changelog entry backport/branch/v15 backport/branch/v16 labels Sep 19, 2024

github-actions bot requested review from camscale and kimlisa September 19, 2024 20:28

github-actions bot added the size/sm label Sep 19, 2024

Expose client tools auto update for find endpoint

61b2b93

vapopov force-pushed the vapopov/add-webapi-find-autoupdate-fields branch from 21709ac to 61b2b93 Compare September 19, 2024 20:29

vapopov requested review from sclevine, hugoShaka and rosstimothy September 19, 2024 20:31

rosstimothy reviewed Sep 19, 2024

View reviewed changes

Group auto update settings in find response

31b8316

Log error instead returning error Add tests auto update settings in find endpoint Add check for not implemented error

vapopov mentioned this pull request Sep 25, 2024

Client tools auto update logic for multiple platforms #46587

Closed

kimlisa removed their request for review September 30, 2024 16:09

vapopov requested a review from rosstimothy September 30, 2024 17:36

rosstimothy reviewed Sep 30, 2024

View reviewed changes

Add more test cases

e3c3bba

vapopov requested a review from rosstimothy October 3, 2024 15:59

sclevine approved these changes Oct 3, 2024

View reviewed changes

rosstimothy approved these changes Oct 4, 2024

View reviewed changes

public-teleport-github-review-bot bot removed request for camscale and hugoShaka October 4, 2024 15:30

vapopov added this pull request to the merge queue Oct 4, 2024

Merged via the queue into master with commit eed38ad Oct 4, 2024
40 checks passed

vapopov deleted the vapopov/add-webapi-find-autoupdate-fields branch October 4, 2024 16:22

This was referenced Oct 4, 2024

[v16] Expose client tools auto update for find endpoint #47213

Closed

[v15] Expose client tools auto update for find endpoint #47214

Closed

[v14] Expose client tools auto update for find endpoint #47215

Closed

vapopov mentioned this pull request Nov 8, 2024

[v16] Client tools autoupdates #48645

Merged

vapopov mentioned this pull request Nov 8, 2024

[v15] Client tools autoupdates #48648

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Expose client tools auto update for find endpoint #46785

Expose client tools auto update for find endpoint #46785

vapopov commented Sep 19, 2024

rosstimothy left a comment

rosstimothy Sep 19, 2024

rosstimothy Sep 19, 2024

vapopov Sep 19, 2024

rosstimothy Sep 19, 2024

rosstimothy Sep 19, 2024

vapopov Sep 20, 2024

rosstimothy Sep 30, 2024

sclevine Oct 3, 2024 •

edited

Loading

rosstimothy Oct 3, 2024

rosstimothy Sep 30, 2024

rosstimothy Sep 30, 2024

rosstimothy Sep 30, 2024

rosstimothy Sep 30, 2024

public-teleport-github-review-bot bot commented Oct 4, 2024

	ToolsAutoupdate bool `json:"tools_autoupdate"`
	ToolsAutoUpdate bool `json:"tools_auto_update"`

		assert.True(t, pr.AutoUpdate.ToolsAutoupdate, "tools_autoupdate must be enabled")
		assert.Equal(t, teleport.Version, pr.AutoUpdate.ToolsVersion, "tools_version must be equal to current version")

Expose client tools auto update for find endpoint #46785

Expose client tools auto update for find endpoint #46785

Conversation

vapopov commented Sep 19, 2024

rosstimothy left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sclevine Oct 3, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

public-teleport-github-review-bot bot commented Oct 4, 2024

sclevine Oct 3, 2024 •

edited

Loading