introduce service hooks #12166

ndeloof · 2024-09-26T14:01:59Z

What I did

Added support for post_start and pre_stop lifecycle hooks to services

demo: https://asciinema.org/a/Oh1ZFu2IERd3Fo4nK6lcSAhwS

Related issue

(not mandatory) A picture of a cute animal, if possible in relation to what you did

jhrotko · 2024-10-01T16:44:26Z

go.mod

@@ -193,4 +193,4 @@ require (
 	tags.cncf.io/container-device-interface v0.8.0 // indirect
 )

-replace github.com/compose-spec/compose-go/v2 => github.com/compose-spec/compose-go/v2 v2.2.1-0.20240926141145-d3fd7d94aa31
+replace github.com/compose-spec/compose-go/v2 => github.com/ndeloof/compose-go/v2 v2.0.1-0.20240924090029-29a100a62f5f


leaving this comment here not to forget to delete this after compose-spec is bumped

jhrotko · 2024-10-02T08:57:58Z

pkg/compose/down.go

+		for _, hook := range service.PreStop {
+			err := s.runHook(ctx, container, *service, hook, nil)
+			if err != nil {
+				return err
+			}
+		}


We are always iterating over all hooks for PreStop, PostStart, etc.
I think it would make sense to add a function that encapsulates this in hook.go.

func (s composeService) ExecuteHooks(ctx context.Context, container moby.Container, service types.ServiceConfig, listener api.ContainerEventListener, hooks []types.ServiceHook) error { for _, hook := range hooks { err := s.runHook(ctx, container, service, hook, listener) if err != nil { return err } } return nil }

jhrotko · 2024-10-02T09:06:04Z

pkg/compose/down.go

@@ -85,7 +85,8 @@ func (s *composeService) down(ctx context.Context, projectName string, options a

 	err = InReverseDependencyOrder(ctx, project, func(c context.Context, service string) error {
 		serviceContainers := containers.filter(isService(service))
-		err := s.removeContainers(ctx, serviceContainers, options.Timeout, options.Volumes)
+		serv := project.Services[service]


We could add a validation that the service exists here. Since this is the place where we get its value becoming easier later on for debugging.

serv, ok := project.Services[service] if !ok { return fmt.Error() .... }

as we run inside InReverseDependencyOrder service is guaranteed to exist (actually, InReverseDependencyOrder could pass a *ServiceConfig, but this could have impact on immutability)

service is passed as a pointer down to

compose/pkg/compose/down.go

Line 304 in 0aacddf

if service != nil {

, as this same method is used to also stop orphaned container, where we don't have a defined service so need the ability to pass nil

pkg/compose/stop.go

jhrotko · 2024-10-02T09:14:12Z

pkg/compose/hook.go

+		case <-tick.C:
+			inspect, err := s.apiClient().ContainerExecInspect(ctx, exec.ID)
+			if err != nil {
+				return nil


Might have missed something, but shouldn't we return the error in this case? I see the same is done for ContainerExecStart. But in line 84 we do not ignore these errors

jhrotko · 2024-10-02T09:17:10Z

pkg/compose/down.go

@@ -309,32 +320,32 @@ func (s *composeService) stopContainer(ctx context.Context, w progress.Writer, c
 	return nil
 }

-func (s *composeService) stopContainers(ctx context.Context, w progress.Writer, containers []moby.Container, timeout *time.Duration) error {
+func (s *composeService) stopContainers(ctx context.Context, w progress.Writer, serv *types.ServiceConfig, containers []moby.Container, timeout *time.Duration) error {


nit: rename to serv -> service for consistency

jhrotko · 2024-10-02T09:24:09Z

pkg/compose/hook.go

+	wOut := utils.GetWriter(func(line string) {
+		listener(api.ContainerEvent{
+			Type:      api.HookEventLog,
+			Container: getContainerNameWithoutProject(container) + " ->",


I would propose to change the -> to the name of the hook + index? we could pass it as a variable. If all hooks have the same "signature" -> might be complicated to understand from which hook the log comes from

hooks have no name so far (to be debated)
I wanted here to distinguish hook output from main logs without making the prefix a looooooong line, as this will remain for the whole compose execution after hook completed

how about we choose an emoji for each hook? 😆

My first implementation was using 🪝 but this won't run well on a Windows terminal

jhrotko · 2024-10-02T09:27:44Z

Really cool stuff @ndeloof 😊

glours

Sounds good to me 👍
Glad we finally have the first implementation of hooks mechanism 🥳

Signed-off-by: Nicolas De Loof <[email protected]>

## Description documentation for docker/compose#12166 ## Related issues or tickets [](https://docker.atlassian.net/browse/COMP-760) ## Reviews - [ ] Technical review - [ ] Editorial review - [ ] Product review Signed-off-by: Nicolas De Loof <[email protected]>

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [docker/compose](https://github.com/docker/compose) | minor | `v2.29.7` -> `v2.30.3` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>docker/compose (docker/compose)</summary> ### [`v2.30.3`](https://github.com/docker/compose/releases/tag/v2.30.3) [Compare Source](docker/compose@v2.30.2...v2.30.3) #### What's Changed ##### 🐛 Fixes - Avoid starting all services on rebuild by [@jhrotko](https://github.com/jhrotko) [(12258)](docker/compose#12258) ##### ⚙️ Dependencies - Bump compose-go v2.4.4 by [@glours](https://github.com/glours) [(12274)](docker/compose#12274) **Full Changelog**: docker/compose@v2.30.2...v2.30.3 ### [`v2.30.2`](https://github.com/docker/compose/releases/tag/v2.30.2) [Compare Source](docker/compose@v2.30.1...v2.30.2) #### What's Changed ##### 🐛 Fixes - Service being declared in a profile must not trigger re-creation by [@ndeloof](https://github.com/ndeloof) in [(12265)](docker/compose#12265) - Remove ArtifactType from Config in OCI v1.1 definition of the artifact by [@glours](https://github.com/glours) [(12266)](docker/compose#12266) ##### 🔧 Internal - Add Joana Hrotko to Maintainers by [@laurazard](https://github.com/laurazard) [(12253)](docker/compose#12253) - Add profile e2e test case to document in compose by [@jhrotko](https://github.com/jhrotko) [(12252)](docker/compose#12252) ##### ⚙️ Dependencies - Bump `compose-go` to version `v2.4.3` by [@glours](https://github.com/glours) in docker/compose#12261 **Full Changelog**: docker/compose@v2.30.1...v2.30.2 ### [`v2.30.1`](https://github.com/docker/compose/releases/tag/v2.30.1) [Compare Source](docker/compose@v2.30.0...v2.30.1) #### What's Changed ##### 🐛 Fixes Fix regression when using stdin as input of `-f` flag [(12248)](docker/compose#12248) Fix regression when using multiple time the same YAML anchor in a Compose file [(12247)](docker/compose#12247) ##### ⚙️ Dependencies - bump compose-go to version v2.4.2 by [@glours](https://github.com/glours) in docker/compose#12249 **Full Changelog**: docker/compose@v2.30.0...v2.30.1 ### [`v2.30.0`](https://github.com/docker/compose/releases/tag/v2.30.0) [Compare Source](docker/compose@v2.29.7...v2.30.0) #### What's Changed ##### ✨ Improvements - Introduce service hooks by [@ndeloof](https://github.com/ndeloof) [(12166)](docker/compose#12166) - Introduce generate command as alpha command by [@glours](https://github.com/glours) [(12209)](docker/compose#12209) - Add export command by [@jarqvi](https://github.com/jarqvi) [(12120)](docker/compose#12120) - Add support for CDI device request using `devices` by [@ndeloof](https://github.com/ndeloof) [(12184)](docker/compose#12184) - Add support for bind recursive by [@ndeloof](https://github.com/ndeloof) [(12210)](docker/compose#12210) - Allow usage of `-f` flag with OCI Compose artifacts by [@glours](https://github.com/glours) [(12220)](docker/compose#12220) ##### 🐛 Fixes - Append unix-style relative path when computing container target path by [@ndeloof](https://github.com/ndeloof) [(12145)](docker/compose#12145) - Wait for dependent service up to delay set by --wait-timeout by [@ndeloof](https://github.com/ndeloof) [(12156)](docker/compose#12156) - Check secret source exists, as bind mount would create target by [@ndeloof](https://github.com/ndeloof) [(12151)](docker/compose#12151) - After container restart register printer consumer by [@jhrotko](https://github.com/jhrotko) [(12158)](docker/compose#12158) - Fix(down): Fix down command if specified services are not running by [@idsulik](https://github.com/idsulik) [(12164)](docker/compose#12164) - Show watch error message and open DD only when w is pressed by [@jhrotko](https://github.com/jhrotko) [(12165)](docker/compose#12165) - Fix(push): Fix unexpected EOF on alpha publish by [@idsulik](https://github.com/idsulik) [(12169)](docker/compose#12169) - Fix(convergence): Serialize access to observed state by [@anantadwi13](https://github.com/anantadwi13) [(12150)](docker/compose#12150) - Remove feature flag integration with Docker Desktop for ComposeUI and ComposeNav by [@jhrotko](https://github.com/jhrotko) [(12192)](docker/compose#12192) - Support Dockerfile-specific ignore-file with watch by [@ndeloof](https://github.com/ndeloof) [(12193)](docker/compose#12193) - Add support for raw env_file format by [@ndeloof](https://github.com/ndeloof) [(12179)](docker/compose#12179) - Convert GPUs to DeviceRequests with implicit "gpu" capability by [@ndeloof](https://github.com/ndeloof) [(12197)](docker/compose#12197) - Improve error message to include expected network label by [@divinity76](https://github.com/divinity76) [(12213)](docker/compose#12213) - Don't use progress to render restart, which hides logs by [@ndeloof](https://github.com/ndeloof) [(12226)](docker/compose#12226) - One-off containers are not indexed, and must be ignored by `exec --index` command by [@ndeloof](https://github.com/ndeloof) [(12224)](docker/compose#12224) - Don't warn about uid/gid not being supported while ... they are by [@ndeloof](https://github.com/ndeloof) [(12232)](docker/compose#12232) - Connect to external networks by name by [@ndeloof](https://github.com/ndeloof) [(12234)](docker/compose#12234) - Fix push error message typo by [@chris-crone](https://github.com/chris-crone) [(12237)](docker/compose#12237) - Fix(dockerignore): Add wildcard support to dockerignore.go by [@idsulik](https://github.com/idsulik) [(12239)](docker/compose#12239) ##### 🔧 Internal - Remove bind options when creating a volume type by [@jhrotko](https://github.com/jhrotko) [(12177)](docker/compose#12177) - pass device.options to engine by [@ndeloof](https://github.com/ndeloof) [(12183)](docker/compose#12183) - Add security policy by [@thaJeztah](https://github.com/thaJeztah) [(12194)](docker/compose#12194) - Gha: set default permissions to "contents: read" by [@thaJeztah](https://github.com/thaJeztah) [(12195)](docker/compose#12195) - Desktop: allow this client to be identified via user-agent by [@djs55](https://github.com/djs55) [(12212)](docker/compose#12212) - Compose-go clean volume target to avoid ambiguous comparisons by [@ndeloof](https://github.com/ndeloof) [(12208)](docker/compose#12208) ##### ⚙️ Dependencies - Bump docker `v27.3.1` by [@ndeloof](https://github.com/ndeloof) [(12178)](docker/compose#12178) - Build(deps): bump `golang.org/x/sys` from `0.25.0` to `0.26.0` by [@dependabot](https://github.com/dependabot) [(12189)](docker/compose#12189) - Bump `compose-go` to `v2.3.0` by [@glours](https://github.com/glours) [(12198)](docker/compose#12198) - Bump `compose-go` to `v2.4.0` by [@glours](https://github.com/glours) [(12231)](docker/compose#12231) - Bump `compose-go` to `v2.4.1` by [@glours](https://github.com/glours) [(12243)](docker/compose#12242) - Build(deps): bump github.com/containerd/containerd from 1.7.22 to 1.7.23 by [@dependabot](https://github.com/dependabot) [(12211)](docker/compose#12211) - Bump golang minimal version to `1.22` in go.mod by [@glours](https://github.com/glours) [(12246)](docker/compose#12246) - Bump `go.uber.org/mock` to `v0.5.0` and `google.golang.org/grpc` to `v1.67.1` by [@glours](https://github.com/glours) [(12245)](docker/compose#12245) #### New Contributors - [@anantadwi13](https://github.com/anantadwi13) made their first contribution in docker/compose#12150 - [@jarqvi](https://github.com/jarqvi) made their first contribution in docker/compose#12120 - [@djs55](https://github.com/djs55) made their first contribution in docker/compose#12212 - [@divinity76](https://github.com/divinity76) made their first contribution in docker/compose#12213 **Full Changelog**: docker/compose@v2.29.7...v2.30.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

ndeloof requested review from a team, glours and jhrotko and removed request for a team September 26, 2024 14:02

ndeloof marked this pull request as ready for review September 30, 2024 07:05

ndeloof force-pushed the service_hooks branch 4 times, most recently from b870947 to 0aacddf Compare September 30, 2024 07:30

jhrotko reviewed Oct 1, 2024

View reviewed changes

jhrotko reviewed Oct 2, 2024

View reviewed changes

pkg/compose/stop.go Show resolved Hide resolved

jhrotko reviewed Oct 2, 2024

View reviewed changes

ndeloof mentioned this pull request Oct 4, 2024

lifecycle hooks docker/docs#21061

Merged

3 tasks

glours approved these changes Oct 7, 2024

View reviewed changes

ndeloof force-pushed the service_hooks branch from 0aacddf to d0456b9 Compare October 7, 2024 09:48

ndeloof enabled auto-merge (rebase) October 7, 2024 09:49

introduce service hooks

eeaa609

Signed-off-by: Nicolas De Loof <[email protected]>

ndeloof force-pushed the service_hooks branch from d0456b9 to eeaa609 Compare October 7, 2024 13:55

ndeloof merged commit 82b41b9 into docker:main Oct 7, 2024
28 checks passed

ndeloof deleted the service_hooks branch October 7, 2024 14:07

jhrotko mentioned this pull request Oct 14, 2024

Enhance healthcheck blocking dependent behavior like lifecyle #10580

Closed

stormshield-gt mentioned this pull request Nov 11, 2024

depends_on post_start_completed_successfully #12279

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

introduce service hooks #12166

introduce service hooks #12166

ndeloof commented Sep 26, 2024

jhrotko Oct 1, 2024

jhrotko Oct 2, 2024

jhrotko Oct 2, 2024 •

edited

Loading

ndeloof Oct 2, 2024

jhrotko Oct 2, 2024 •

edited

Loading

jhrotko Oct 2, 2024 •

edited

Loading

jhrotko Oct 2, 2024 •

edited

Loading

ndeloof Oct 2, 2024

jhrotko Oct 3, 2024

ndeloof Oct 3, 2024

jhrotko commented Oct 2, 2024

glours left a comment

introduce service hooks #12166

introduce service hooks #12166

Conversation

ndeloof commented Sep 26, 2024

jhrotko Oct 1, 2024

Choose a reason for hiding this comment

jhrotko Oct 2, 2024

Choose a reason for hiding this comment

jhrotko Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

ndeloof Oct 2, 2024

Choose a reason for hiding this comment

jhrotko Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

jhrotko Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

jhrotko Oct 2, 2024 • edited Loading

Choose a reason for hiding this comment

ndeloof Oct 2, 2024

Choose a reason for hiding this comment

jhrotko Oct 3, 2024

Choose a reason for hiding this comment

ndeloof Oct 3, 2024

Choose a reason for hiding this comment

jhrotko commented Oct 2, 2024

glours left a comment

Choose a reason for hiding this comment

jhrotko Oct 2, 2024 •

edited

Loading

jhrotko Oct 2, 2024 •

edited

Loading

jhrotko Oct 2, 2024 •

edited

Loading

jhrotko Oct 2, 2024 •

edited

Loading