git resolver does not work with repositories with a symlink

[concaf]

Expected Behavior

a symlink being present in the remote git repo should be inconsequential for the git resolver to function

Actual Behavior

if the remote git repo has a symlink, then the remote resolution fails with the error -

checkout error: worktree contains unstaged changes

the resolution request shows this as well, but i don't have the output - i do have logs from the remote-resolvers pod though -

{"severity":"error","timestamp":"2024-01-11T11:41:49.669Z","logger":"controller","caller":"controller/controller.go:566","message":"Reconcile error","commit":"e59ee42","knative.dev/traceid":"e14854a9-45fe-4120-a304-5b287fd5e39e","knative.dev/key":"default/git-3bfb7d8d2e23690240e63c76dce6902f","duration":1.072205099,"error":"error getting \"Git\" \"default/git-3bfb7d8d2e23690240e63c76dce6902f\": checkout error: worktree contains unstaged changes","stacktrace":"knative.dev/pkg/controller.(*Impl).handleErr\n\tknative.dev/[email protected]/controller/controller.go:566\nknative.dev/pkg/controller.(*Impl).processNextWorkItem\n\tknative.dev/[email protected]/controller/controller.go:543\nknative.dev/pkg/controller.(*Impl).RunContext.func3\n\tknative.dev/[email protected]/controller/controller.go:491"}

Steps to Reproduce the Problem

1. create a pipeline and push to your git repo:

kind: Pipeline
apiVersion: tekton.dev/v1beta1
metadata:
  name: a-simple-pipeline
spec:
  params:
  - name: username
  tasks:
  - name: task-1
    params:
    - name: username
      value: $(params.username)
    taskSpec:
      params:
      - name: username
      steps:
      - image: alpine:3.15
        script: |
          echo "hello $(params.username)"

2. create a pipelinerun pointing to the remote git repo:

kind: PipelineRun
apiVersion: tekton.dev/v1beta1
metadata:
  name: run-basic-pipeline-from-git
spec:
  pipelineRef:
    resolver: git
    params:
    - name: url
      value: https://github.com/concaf/tmp.git 
    - name: revision
      value: test
    - name: pathInRepo
      value: git-resolver/pipeline.yaml
  params:
  - name: username
    value: liza

3. note the symlink in my remote repo at https://github.com/concaf/tmp/tree/test/kedge-blog

4. when you create the pipelinerun, note the error in tekton resolvers pod and the resolution request:

checkout error: worktree contains unstaged changes

5. removing the symlink from the remote git repo fixes this.

Additional Info

• Kubernetes version:

  Output of kubectl version:

Client Version: v1.24.1
Kustomize Version: v4.5.4
Server Version: v1.27.3

• Tekton Pipeline version:

Client version: 0.33.0
Pipeline version: v0.55.0

while speaking with @vdemeester, he pointed out that this might be caused by go-git/go-git#511

[pjbgf]

@concaf if this is due to go-git, I would recommend trying out opting into the osfs.BoundOS as per go-git/go-git#1155 (comment):

	wt := osfs.New(dir, osfs.WithBoundOS())
	dotfs, err := wt.Chroot(git.GitDirName)
	if err != nil {
		return nil, err
	}
	store := filesystem.NewStorage(dotfs, cache.NewObjectLRUDefault())

	repository, err := git.Clone(store, wt, &git.CloneOptions{
		URL:      repositoryName,
		Progress: os.Stdout,
		Depth:    1,
	})

This will become the default behaviour on V6. For more information, please refer to the comment above.

[vdemeester]

@pjbgf we are using memfs (which doesn't have that option). We could switch to use osfs but this needs some careful redesign (we don't want to run out of storage, …).

[pjbgf]

@vdemeester would you be able to test this using the existing code against the latest pseudo version for go-billy? There are quite a few bug fixes and improvements that are yet to be released on a cut release, some of them touching symlinks, so potentially this is already fixed.

[vdemeester]

@vdemeester would you be able to test this using the existing code against the latest pseudo version for go-billy? There are quite a few bug fixes and improvements that are yet to be released on a cut release, some of them touching symlinks, so potentially this is already fixed.

yes, I think I could 👼🏼

[vdemeester]

@pjbgf wonderful, it is fixed 🙃 thanks ❤️
Ideally, we would want to bump it with a release but not sure if you are willing to create one or not ? (if not, I'll bump it as is)

[pjbgf]

@vdemeester nice, glad to hear that worked out. 🙌

As for the release, unfortunately I don't have permission to cut releases within go-git. I am hoping we will cut new releases for both go-billy and go-git before the end of the year but that will depend on other maintainers availability.