Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Strange decode issue in flag cache and build cache with store branch of stack #25

Closed
mgsloan opened this issue May 20, 2016 · 1 comment
Closed

Strange decode issue in flag cache and build cache with store branch of stack #25

mgsloan opened this issue May 20, 2016 · 1 comment

Comments

@mgsloan
Copy link
Owner

mgsloan commented May 20, 2016

The current blocker to releasing store publicly is that I ran into some funky behavior with stack ported to store. Not yet sure if it's a store issue.

Repro:

  1. Checkout https://github.com/commercialhaskell/stack/tree/wip-store next to your store dir. (so that ../store/ exists from the stack dir)

  2. stack build --fast within the stack directory

  3. stack exec bash to enter into a shell with modified PATH

  4. Run stack build -v twice and for the 2nd run observe a log like. It needs to be run twice as decode failures are expected the first run. The caches should be populated after that, though.

mgsloan@computer:~/fpco/stack$ stack build -v
Version 1.1.1, Git revision 10ba9938db538ef734d5b8385b5f84e8ff35c65e (dirty) (3605 commits) x86_64 hpack-0.14.0
2016-05-20 03:22:24.252525: [debug] Checking for project config at: /home/mgsloan/fpco/stack/stack.yaml @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Config src/Stack/Config.hs:813:9)
2016-05-20 03:22:24.252868: [debug] Loading project config file stack.yaml @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Config src/Stack/Config.hs:831:13)
2016-05-20 03:22:24.256728: [debug] Checking whether stack was built with libgmp4 @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Config src/Stack/Config.hs:328:5)
2016-05-20 03:22:24.256900: [debug] Run process: ldd /home/mgsloan/fpco/stack/.stack-work/install/x86_64-linux/lts-5.15/7.10.3/bin/stack @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:24.270410: [debug] Stack was not built with libgmp4 @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Config src/Stack/Config.hs:332:14)
2016-05-20 03:22:24.270684: [debug] Trying to decode /home/mgsloan/.stack/build-plan-cache/x86_64-linux/lts-5.15.cache @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:52:5)
2016-05-20 03:22:24.368780: [debug] Success decoding /home/mgsloan/.stack/build-plan-cache/x86_64-linux/lts-5.15.cache @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:61:13)
2016-05-20 03:22:24.383067: [debug] Getting system compiler version @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Setup src/Stack/Setup.hs:341:17)
2016-05-20 03:22:24.383334: [debug] Run process: ghc --info @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:24.448115: [debug] Asking GHC for its version @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Setup.Installed src/Stack/Setup/Installed.hs:94:13)
2016-05-20 03:22:24.448505: [debug] Run process: ghc --numeric-version @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:24.482177: [debug] Getting Cabal package version @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.GhcPkg src/Stack/GhcPkg.hs:165:5)
2016-05-20 03:22:24.482435: [debug] Run process: ghc-pkg --no-user-package-db field --simple-output Cabal version @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:24.503890: [debug] Resolving package entries @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Setup src/Stack/Setup.hs:221:5)
2016-05-20 03:22:24.504386: [debug] Getting global package database location @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.GhcPkg src/Stack/GhcPkg.hs:48:5)
2016-05-20 03:22:24.504478: [debug] Run process: ghc-pkg --no-user-package-db list --global @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:24.561320: [debug] Error while decoding /home/mgsloan/fpco/stack/.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/stack-build-cache: PeekException {peekExBytesFromEnd = 30381, peekExMessage = "Didn't consume all input."} (this might not be an error, when switching between stack versions) @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:76:18)
2016-05-20 03:22:24.701233: [warn] There were multiple candidates for the Cabal entry "Test.hs" (/home/mgsloan/fpco/stack/Test.hs), picking /home/mgsloan/fpco/stack/src/test/Test.hs @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Package src/Stack/Package.hs:1037:5)
2016-05-20 03:22:24.724766: [debug] Error while decoding /home/mgsloan/fpco/store/.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/stack-build-cache: PeekException {peekExBytesFromEnd = 5788, peekExMessage = "Didn't consume all input."} (this might not be an error, when switching between stack versions) @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:76:18)
2016-05-20 03:22:24.769037: [debug] Run process: ghc-pkg --global --no-user-package-db dump --expand-pkgroot @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:24.814132: [debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.22.7.0 @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-05-20 03:22:24.814300: [debug] Ignoring package haskeline due to wanting version 0.7.2.3 instead of 0.7.2.1 @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-05-20 03:22:24.814378: [debug] Ignoring package terminfo due to wanting version 0.4.0.2 instead of 0.4.0.1 @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-05-20 03:22:24.814754: [debug] Ignoring package Cabal due to wanting version 1.22.8.0 instead of 1.22.5.0 @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-05-20 03:22:24.815047: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/mgsloan/.stack/snapshots/x86_64-linux/lts-5.15/7.10.3/pkgdb dump --expand-pkgroot @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:24.993441: [debug] Ignoring package th-orphans, from (InstalledTo Snap,"/home/mgsloan/.stack/snapshots/x86_64-linux/lts-5.15/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-05-20 03:22:24.993615: [debug] Ignoring package th-reify-many, from (InstalledTo Snap,"/home/mgsloan/.stack/snapshots/x86_64-linux/lts-5.15/7.10.3/pkgdb/"), due to wrong location: (Just (InstalledTo Snap),Local) @(stack_I9M2eJwnG6d3686aQ2OkVk:Stack.Build.Installed src/Stack/Build/Installed.hs:189:5)
2016-05-20 03:22:24.998228: [debug] Run process: ghc-pkg --user --no-user-package-db --package-db /home/mgsloan/fpco/stack/.stack-work/install/x86_64-linux/lts-5.15/7.10.3/pkgdb dump --expand-pkgroot @(stack_I9M2eJwnG6d3686aQ2OkVk:System.Process.Read src/System/Process/Read.hs:283:3)
2016-05-20 03:22:25.067477: [debug] Trying to decode /home/mgsloan/.stack/indices/Hackage/00-index.cache @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:52:5)
2016-05-20 03:22:26.118330: [debug] Success decoding /home/mgsloan/.stack/indices/Hackage/00-index.cache @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:61:13)
2016-05-20 03:22:26.393979: [debug] Error while decoding /home/mgsloan/fpco/stack/.stack-work/install/x86_64-linux/lts-5.15/7.10.3/flag-cache/hpack-0.14.0-78bcd052466e0eb71e90498de3df0500: PeekException {peekExBytesFromEnd = 1, peekExMessage = "Attempted to read too many bytes for Char. Needed 4, but only 1 remain."} (this might not be an error, when switching between stack versions) @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:76:18)
2016-05-20 03:22:26.396649: [debug] Error while decoding /home/mgsloan/fpco/stack/.stack-work/install/x86_64-linux/lts-5.15/7.10.3/flag-cache/path-io-1.1.0-ce75195d16585f47735842b54d8af5b1: PeekException {peekExBytesFromEnd = 1, peekExMessage = "Attempted to read too many bytes for Char. Needed 4, but only 1 remain."} (this might not be an error, when switching between stack versions) @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:76:18)
2016-05-20 03:22:26.399590: [debug] Error while decoding /home/mgsloan/fpco/stack/.stack-work/install/x86_64-linux/lts-5.15/7.10.3/flag-cache/th-lift-instances-0.1.6-f0c252f6ee851126bd2e88464c35455d: PeekException {peekExBytesFromEnd = 3, peekExMessage = "Attempted to read too many bytes for Char. Needed 4, but only 3 remain."} (this might not be an error, when switching between stack versions) @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:76:18)
2016-05-20 03:22:26.402859: [debug] Error while decoding /home/mgsloan/fpco/stack/.stack-work/install/x86_64-linux/lts-5.15/7.10.3/flag-cache/th-reify-many-0.1.6-09dad3e1a00454f9216c8e45fbcd2618: PeekException {peekExBytesFromEnd = 1, peekExMessage = "Attempted to read too many bytes for Char. Needed 4, but only 1 remain."} (this might not be an error, when switching between stack versions) @(stack_I9M2eJwnG6d3686aQ2OkVk:Data.Store.VersionTagged src/Data/Store/VersionTagged.hs:76:18)
...

(Note the PeekExceptions)
kantp pushed a commit to kantp/stack that referenced this issue May 23, 2016
Fix caches.
da98346 
There were two issues that caused the decoding of the caches to
fail (see mgsloan/store#25):

- taggedEncodeFile did not actually write the file.  This prevented the
  caches from being populated, which in turn lead to
  PeekExceptions (because the caches still had contents serialised with
  cereal).

- decodeFileMaybe tried to deserialize to a value of type a, while the
  files contained values of type Tagged a.
kantp pushed a commit to kantp/stack that referenced this issue May 23, 2016
Fix caches.
665b990 
There was an issues that caused the decoding of the caches to fail (see
mgsloan/store#25): decodeFileMaybe tried to deserialize to a value of type
a, while the files contained values of type Tagged a.
@kantp kantp mentioned this issue May 23, 2016
Merged
@kantp
Copy link
Contributor

kantp commented May 23, 2016

This is not an error in store, just needed a small fix in the part of stack that uses store.

See commercialhaskell/stack#2179

@kantp kantp closed this as completed May 23, 2016
mgsloan pushed a commit to commercialhaskell/stack that referenced this issue May 24, 2016
@mgsloan
Fix caches.
12ec101 
There was an issues that caused the decoding of the caches to fail (see
mgsloan/store#25): decodeFileMaybe tried to deserialize to a value of type
a, while the files contained values of type Tagged a.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mgsloan @kantp