Benchmarks fail to run on GHC 7.10.3

[RyanGlScott]

Steps to reproduce:

$ cabal get tar
Unpacking to tar-0.5.0.3/
$ cd tar-0.5.0.3/
$ cabal bench
...                                                   
Running 1 benchmarks...                                                                   
Benchmark bench: RUNNING...                                                               
bench: 01-index.tar: openBinaryFile: does not exist (No such file or directory)           
Benchmark bench: ERROR

See also iu-parfunc/sc-haskell#7

[dcoutts]

I'm not sure what to do with benchmarks that require large data file inputs. Do we really want large data files in the git repo (or much worse, in the distributed tarball)? But on the other hand it is nice to be able to run the benchmarks without any further prep. Any suggestions? What do other packages do? Assume a net connection and run scripts to fetch the data files from elsewhere?

[RyanGlScott]

I'm confused on a more fundamental level here. What exactly is 01-index.tar? Where is it supposed to be located? I can't even figure out how to run the benchmarks at all, let alone run them without any further prep.

[trofi]

It's a file fetched by 'cabal update'. Lurks in
~/.cabal/packages/hackage.haskell.org

[RyanGlScott]

Ah, I see. Interestingly, I don't see any file named 01-index.tar living in my ~/.cabal/packages/hackage.haskell.org folder, only:

• 00-index-cache (2.2 MB)
• 00-index.tar (233 MB)
• 00-index.tar.gz (12 MB)
• 00-index.tar.gz.etag (34 B)

But I agree that bundling a huge tarball for benchmarking purposes is less than ideal. Off the top of my head, I'm not aware of any Haskell benchmarks in the wild that download prerequisite files from the Internet before running, but such a thing is possible with env.

Alternatively, I'd be happy with an explanation of the steps you'd need (in the README or otherwise) to run before invoking the benchmark program.

As an aside: is this 01-index.tar guaranteed to be the same across all runs of this benchmark? I'd be worried about getting bad results due to, say, more packages being added to Hackage over time.

[dcoutts]

Aye documenting in the README and/or in the error messages would be helpful

[Bodigrim]

Fixed in 7d0517d.

[RyanGlScott]

Thanks!

FWIW, my 01-index.tar was not located under ~/.cabal/packages/hackage.haskell.org, but rather ~/.cache/cabal/packages/hackage.haskell.org.

[Bodigrim]

Yeah, potentially it could be anywhere depending on $CABAL_DIR and XDG configuration, but I think it's enough of a hint.