ipfs init --profile=badgerds, mmap value log error

[leerspace]

Version information: ipfs version 0.4.11-rc1

Type: Bug

Severity: High

Description: Initializing an ipfs repository using the badgerds profile doesn't work in 0.4.11-rc1 and results in a non-working repository. Given the error message about not having enough free space I moved the repository onto a 1TB drive but it still fails.

FWIW this worked in ipfs version 0.4.11-dev-c14a995.

Here's the output from ipfs init --debug --profile=badgerds on Windows:

ipfs init --debug --profile=badgerds
�[0;37m15:43:06.504  INFO �[0;34m  cmd/ipfs: �[0mIPFS_PATH C:\Users\John\.ipfs �[0;37mmain.go:301�[0m
�[0;37m15:43:06.504 �[36mDEBUG �[0;34m  cmd/ipfs: �[0mcalling pre-command hooks... �[0;37mmain.go:295�[0m
�[0;37m15:43:06.505 �[36mDEBUG �[0;34m      lock: �[0mChecking lock �[0;37mlock.go:32�[0m
�[0;37m15:43:06.505 �[36mDEBUG �[0;34m      lock: �[0mFile doesn't exist: C:\Users\John\.ipfs/repo.lock �[0;37mlock.go:34�[0m
�[0;37m15:43:06.506  INFO �[0;34m  cmd/ipfs: �[0mchecking if daemon is running... �[0;37minit.go:66�[0m
�[0;37m15:43:06.506 �[36mDEBUG �[0;34m  cmd/ipfs: �[0mexecuting command locally �[0;37mmain.go:342�[0m
initializing IPFS node at C:\Users\John\.ipfs
generating 2048-bit RSA keypair...done
peer identity: QmcSimR7CDQqGCAsYBokGxpt1YcExA5dtiJm6tK52sEs6p
Error: Unable to open value log: Unable to mmap value log file: Not enough storage is available to process this command.

[leerspace]

PR #4007

[whyrusleeping]

Hey @manishrjain, any idea here?

[manishrjain]

Ah.. yeah this is a known issue. We set the initial mmap size when writing a log file to uint32 max (4GB). And in Windows, this seems to cause issues. We'll fix it urgently.

dgraph-io/badger#212

[leerspace]

From looking at the linked issue it sounds like this is suspected to be a Windows issue and I can confirm that I am getting this error on Windows 10. I also see the OP in the linked issue saying "this should not prevent Badger from working on a Windows machine with enough RAM", but the machine I'm getting this error on has 16 GB of RAM which seems like it should be more than enough.

[manishrjain]

We figured out what the issue was. Just pushed a fix, if you can try it out and confirm it.

dgraph-io/badger#212

[leerspace]

@whyrusleeping does the badger dependency for IPFS need to be updated or something like that in order for new IPFS builds from master to get the fix? I just tried building and retesting and saw the error again.

[ghost]

@leerspace yes, all dependencies are immutable, so the change needs to be bubbled up

[magik6k]

This was fixed with #4291

[Mikaela]

I am seeing this on Arch Linux ARM on Raspberry Pi B+:

nitializing IPFS node at /home/mikaela/.ipfs                                  
generating 2048-bit RSA keypair...done
peer identity: QmdEXRTSA9XJkyccbtmBBsDZYXvGNvx8EJ5ZpnqZcLwTs4                  
badger 2019/04/23 14:31:37 INFO: All 0 tables opened in 0s                     
Error: Mmap value log file. Path=/home/mikaela/.ipfs/badgerds/000000.vlog. Error=cannot allocate memory
┌┌(mikaela@tezagm)┌(174/pts/3)┌(02:31:04/23/19)┌-                              
└┌(%:~)┌- ipfs --version
ipfs version 0.4.19

I think in my case it's just out of memory as that isn't a very powerful device.

[manishrjain]

You can set value log to be served off disk instead of mmaping it.

[nampdn]

You can set value log to be served off disk instead of mmaping it.

@manishrjain Could you please help me where to change this parameter in the config file? I'm facing the same issue

[Stebalien]

@nampdn You can't yet change this through the config. Could you file a feature request?

[jsimnz]

I just received this error this error as well while running ipfs init -p badgerds

generating 2048-bit RSA keypair...done
peer identity: QmUT6JjNRuzuxGQRtHAax5LJdYbCEGabonJjfbFSQDT83p
Error: Mmap value log file. Path=/home/jsimnz/.ipfs/badgerds/000000.vlog. Error=exec format error

My machine has 32GB of memory, and only 10GB is used. Windows 10 using the Ubuntu subsystem.

Thoughts?

(edit: newest IPFS version, v0.4.21)

[Stebalien]

That's a bug in the WSL: microsoft/WSL#3451. We should probably make mmapping configurable (#6449) but WSL really needs to fix this.