Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

koanf reports empty string key after a file gets modified #305

Open
matheusjunior opened this issue Jul 3, 2024 · 1 comment
Open

koanf reports empty string key after a file gets modified #305

matheusjunior opened this issue Jul 3, 2024 · 1 comment
Labels
needs-investigation

Comments

@matheusjunior
Copy link

Describe the bug
File watcher prints empty string data after file gets modified a couple of times.

To Reproduce

  1. Run the go snippet below
  2. Modify rpc value from config.yaml file
  3. Save config.yaml
  4. Repeat steps 2-3 until issue happens

main.go:

package main

import (
	"fmt"
	"log"
	"time"

	"github.com/knadh/koanf/parsers/yaml"
	"github.com/knadh/koanf/providers/file"
	"github.com/knadh/koanf/v2"
)

func main() {
	k := koanf.New(".")
	provider := file.Provider("config.yaml")
	err := k.Load(provider, yaml.Parser())
	if err != nil {
		panic(err)
	}

	provider.Watch(func(event interface{}, err error) {
		if err != nil {
			log.Printf("watch error: %v", err)
			return
		}

		// Throw away the old config and load a fresh copy.
		log.Println("config changed. Reloading ...")
		k = koanf.New(".")
		err = k.Load(provider, yaml.Parser())
		if err != nil {
			log.Printf("watch error: %v", err)
			return
		}
	})

	go func() {
		for {
			fmt.Println(k.String("rpc"))
			time.Sleep(time.Second)
		}
	}()

	<-make(chan bool)
}

config.yaml:

rpc: "ddf"

Screenshot showing the described behavior:
Screenshot from 2024-07-03 18-53-43

Expected behavior
Updated rpc value is correctly shown on logs.

Please provide the following information):

  • OS: Linux 6.5.0-41-generic # 41~22.04.2-Ubuntu SMP PREEMPT_DYNAMIC
  • Koanf Version: v2.1.1
  • go version: go version go1.21.6 linux/amd64

Additional context
@matheusjunior matheusjunior added the bug Something isn't working label Jul 3, 2024
@rhnvrm
Copy link
Collaborator

rhnvrm commented Jul 4, 2024

Running the sample code with go run -race main.go suggests that there are data races

go run -race main.go 
xddf
xddf
xddf
2024/07/04 11:32:10 config changed. Reloading ...
==================
WARNING: DATA RACE
Read at 0x00c000112040 by goroutine 9:
  main.main.func2()
      /tmp/test/main.go:39 +0x2e

Previous write at 0x00c000112040 by goroutine 8:
  main.main.func1()
      /tmp/test/main.go:29 +0x38e
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a0d0 by goroutine 9:
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:318 +0x65
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a0d0 by goroutine 8:
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:420 +0x279
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a240 by goroutine 9:
  runtime.mapaccess2_faststr()
      /usr/local/go/src/runtime/map_faststr.go:108 +0x0
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:318 +0x8d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a240 by goroutine 8:
  runtime.mapassign_faststr()
      /usr/local/go/src/runtime/map_faststr.go:203 +0x0
  github.com/knadh/koanf/v2.populateKeyParts()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:509 +0x5e7
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:421 +0x2b9
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c0001c21e8 by goroutine 9:
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:318 +0x9e
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c0001c21e8 by goroutine 8:
  github.com/knadh/koanf/v2.populateKeyParts()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:509 +0x5f6
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:421 +0x2b9
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a0c0 by goroutine 9:
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:322 +0xee
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a0c0 by goroutine 8:
  github.com/knadh/koanf/v2.NewWithConf()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:80 +0x28e
  github.com/knadh/koanf/v2.New()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:71 +0x254
  main.main.func1()
      /tmp/test/main.go:29 +0x382
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c000016130 by goroutine 9:
  github.com/knadh/koanf/maps.Search()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/maps.go:216 +0x47
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:322 +0x10d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c000016130 by goroutine 8:
  runtime.slicecopy()
      /usr/local/go/src/runtime/slice.go:325 +0x0
  github.com/knadh/koanf/v2.populateKeyParts()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:510 +0x694
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:421 +0x2b9
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c00019a030 by goroutine 9:
  runtime.mapaccess2_faststr()
      /usr/local/go/src/runtime/map_faststr.go:108 +0x0
  github.com/knadh/koanf/maps.Search()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/maps.go:216 +0x67
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:322 +0x10d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c00019a030 by goroutine 8:
  runtime.mapassign_faststr()
      /usr/local/go/src/runtime/map_faststr.go:203 +0x0
  github.com/knadh/koanf/maps.Merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/maps.go:114 +0x253
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:416 +0x13c
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================
==================
WARNING: DATA RACE
Read at 0x00c0001b42c8 by goroutine 9:
  github.com/knadh/koanf/maps.Search()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/maps.go:216 +0x75
  github.com/knadh/koanf/v2.(*Koanf).Get()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:322 +0x10d
  github.com/knadh/koanf/v2.(*Koanf).String()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/getters.go:375 +0x3e
  main.main.func2()
      /tmp/test/main.go:39 +0x47

Previous write at 0x00c0001b42c8 by goroutine 8:
  github.com/knadh/koanf/maps.Merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/maps.go:114 +0x264
  github.com/knadh/koanf/v2.(*Koanf).merge()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:416 +0x13c
  github.com/knadh/koanf/v2.(*Koanf).Load()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/[email protected]/koanf.go:120 +0x1d9
  main.main.func1()
      /tmp/test/main.go:30 +0x3fd
  github.com/knadh/koanf/providers/file.(*File).Watch.func1()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:108 +0x4db

Goroutine 9 (running) created at:
  main.main()
      /tmp/test/main.go:37 +0x3a7

Goroutine 8 (running) created at:
  github.com/knadh/koanf/providers/file.(*File).Watch()
      /home/rhnvrm/go/pkg/mod/github.com/knadh/koanf/providers/[email protected]/file.go:61 +0x40f
  main.main()
      /tmp/test/main.go:21 +0x336
==================

@knadh knadh added needs-investigation and removed bug Something isn't working labels Jul 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
needs-investigation
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@knadh @rhnvrm @matheusjunior