Upgraded the downloaded stack version to 2.7.5 #1751

symbiont-eric-torreborre · 2022-06-02T15:49:23Z

to avoid expiry issues on mirrors.json: haskell/hackage-server#834.

I don't know if the additional field in STACK_MIN_VERSION works though. Maybe @facundominguez would know this?

I also don't know if you prefer keeping a wider range of stack versions available despite this expiry issue.

expiry issues on mirrors.json: haskell/hackage-server#834

avdv · 2022-06-09T07:05:25Z

thanks for the PR! Could you elaborate a bit why you think this fixes the expiry issue? AFAIU, this is a server side problem and can only be fixed by updating the mirrors.json file?

symbiont-eric-torreborre · 2022-06-09T07:29:05Z

Hi, @avdv

Could you elaborate a bit why you think this fixes the expiry issue?

That's because this check has been disabled by default in stack commercialhaskell/stack@4d54687.

The mirrors.json file has indeed been eventually uploaded but our CI was down during the whole time. And as soon as you have SLAs your CI becomes really part of production :-)

symbiont-eric-torreborre · 2022-06-09T07:29:46Z

Is there anything I can / should do about the failing tests?

avdv · 2022-06-09T08:33:52Z

That's because this check has been disabled by default in stack commercialhaskell/stack@4d54687.

Oh, that is quite old and is already part of stack 2.1.3. We had stack 2.3.1 which means you should not run into this problem... 🤔

Is there anything I can / should do about the failing tests?

For one, there was a problem downloading from hackage in the Build & Test - Nixpkgs (macos-10.15) check (was this the error you were seeing too?):

ERROR: An error occurred during the fetch of repository 'rules_haskell_stack_update':
   Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/9e75b231e6ff36fa9852d8b1f1fbcd40/external/rules_haskell/haskell/cabal.bzl", line 2284, column 28, in _stack_update_impl
		_execute_or_fail_loudly(repository_ctx, [stack_cmd, "update"])
	File "/private/var/tmp/_bazel_runner/9e75b231e6ff36fa9852d8b1f1fbcd40/external/rules_haskell/haskell/private/workspace_utils.bzl", line 27, column 13, in execute_or_fail_loudly
		fail("\n".join(["Command failed: " + " ".join(arguments), exec_result.stderr]))
Error in fail: Command failed: /private/var/tmp/_bazel_runner/9e75b231e6ff36fa9852d8b1f1fbcd40/external/rules_haskell_stack/stack update
Selected mirror https://hackage.haskell.org/
Downloading root
HttpExceptionRequest Request {
  host                 = "hackage.haskell.org"
  port                 = 443
  secure               = True
  requestHeaders       = [("Accept-Encoding",""),("User-Agent","Haskell pantry package")]
  path                 = "/root.json"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (InternalException (HandshakeFailed (Error_Protocol ("certificate rejected: security: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)",True,CertificateUnknown))))
ERROR: Error fetching repository: Traceback (most recent call last):
	File "/private/var/tmp/_bazel_runner/9e75b231e6ff36fa9852d8b1f1fbcd40/external/rules_haskell/haskell/cabal.bzl", line 2284, column 28, in _stack_update_impl
		_execute_or_fail_loudly(repository_ctx, [stack_cmd, "update"])
	File "/private/var/tmp/_bazel_runner/9e75b231e6ff36fa9852d8b1f1fbcd40/external/rules_haskell/haskell/private/workspace_utils.bzl", line 27, column 13, in execute_or_fail_loudly
		fail("\n".join(["Command failed: " + " ".join(arguments), exec_result.stderr]))
Error in fail: Command failed: /private/var/tmp/_bazel_runner/9e75b231e6ff36fa9852d8b1f1fbcd40/external/rules_haskell_stack/stack update
Selected mirror https://hackage.haskell.org/
Downloading root
HttpExceptionRequest Request {
  host                 = "hackage.haskell.org"
  port                 = 443
  secure               = True
  requestHeaders       = [("Accept-Encoding",""),("User-Agent","Haskell pantry package")]
  path                 = "/root.json"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (InternalException (HandshakeFailed (Error_Protocol ("certificate rejected: security: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)",True,CertificateUnknown))))
ERROR: /private/tmp/bazel-run-start-script/BUILD.bazel:29:15: no such package '@stackage//': no such package '@rules_haskell_stack_update//': Command failed: /private/var/tmp/_bazel_runner/9e75b231e6ff36fa9852d8b1f1fbcd40/external/rules_haskell_stack/stack update
Selected mirror https://hackage.haskell.org/
Downloading root
HttpExceptionRequest Request {
  host                 = "hackage.haskell.org"
  port                 = 443
  secure               = True
  requestHeaders       = [("Accept-Encoding",""),("User-Agent","Haskell pantry package")]
  path                 = "/root.json"
  queryString          = ""
  method               = "GET"
  proxy                = Nothing
  rawBody              = False
  redirectCount        = 10
  responseTimeout      = ResponseTimeoutDefault
  requestVersion       = HTTP/1.1
}
 (InternalException (HandshakeFailed (Error_Protocol ("certificate rejected: security: createProcess: runInteractiveProcess: exec: does not exist (No such file or directory)",True,CertificateUnknown))))
 and referenced by '//:example'

I restarted this job.

Then, the Build & Test - bindist (windows-latest) check failed with a bazel crash after all tests were successful.

Both, Build & Test - bindist (macos-10.15) and Build & Test - bindist (ubuntu-latest), failed because of this error:

 ==================== Test output for //tests/stack-snapshot-deps:hs_override_stack_test:
--- FAIL: TestHsBinRepl (4.54s)
    hs_override_stack_test.go:119: Starting local Bazel server and connecting to it...
        Loading: 
        Loading: 0 packages loaded
        Analyzing: target //:hs-bin@repl (1 packages loaded)
        Analyzing: target //:hs-bin@repl (1 packages loaded, 0 targets configured)
        Analyzing: target //:hs-bin@repl (27 packages loaded, 44 targets configured)
        INFO: Repository rules_haskell_stack instantiated at:
          /home/runner/.cache/bazel/_bazel_runner/c91220d87f26118b19246641d5266d21/bazel_testing/bazel_go_test/main/WORKSPACE:46:10: in <toplevel>
          /tmp/.cache/bazel/_bazel_runner/89fc7584aa1161de06e152a8d573e8c0/external/rules_haskell/haskell/cabal.bzl:2704:15: in use_stack
        Repository rule _use_stack defined at:
          /tmp/.cache/bazel/_bazel_runner/89fc7584aa1161de06e152a8d573e8c0/external/rules_haskell/haskell/cabal.bzl:2669:29: in <toplevel>
        ERROR: An error occurred during the fetch of repository 'rules_haskell_stack':
           Traceback (most recent call last):
        	File "/tmp/.cache/bazel/_bazel_runner/89fc7584aa1161de06e152a8d573e8c0/external/rules_haskell/haskell/cabal.bzl", line 2665, column 13, in _use_stack_impl
        		fail(error)
        Error in fail: Stack 2.3.1 found. Need 2.7.5 or newer.
        ERROR: Error fetching repository: Traceback (most recent call last):
        	File "/tmp/.cache/bazel/_bazel_runner/89fc7584aa1161de06e152a8d573e8c0/external/rules_haskell/haskell/cabal.bzl", line 2665, column 13, in _use_stack_impl
        		fail(error)
        Error in fail: Stack 2.3.1 found. Need 2.7.5 or newer.
        ERROR: /home/runner/.cache/bazel/_bazel_runner/c91220d87f26118b19246641d5266d21/bazel_testing/bazel_go_test/main/BUILD.bazel:6:13: no such package '@stackage//': no such package '@rules_haskell_stack//': Stack 2.3.1 found. Need 2.7.5 or newer. and referenced by '//:hs-bin'
        ERROR: Analysis of target '//:hs-bin@repl' failed; build aborted: Analysis failed
        INFO: Elapsed time: 4.485s
        INFO: 0 processes.
        FAILED: Build did NOT complete successfully (33 packages loaded, 72 targets configured)
        ERROR: Build failed. Not running target
        FAILED: Build did NOT complete successfully (33 packages loaded, 72 targets configured)
        exit status 1 build does not use specified dummy `stack`
FAIL

Seems like we still use stack 2.3.1 somewhere. Could you look into this, please?

symbiont-eric-torreborre · 2022-06-09T09:12:22Z

@avdv oh, it looks like I misinterpreted this tag:

Oh, that is quite old and is already part of stack 2.1.3. We had stack 2.3.1 which means you should not run into this problem...

Good point! I dug into our CI logs when we had this issue and saw this that I missed the first time:

DEBUG: /home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/rules_haskell/haskell/cabal.bzl:2349:10: Stack 1.9.3.1 found. Need 2.3 or newer.
DEBUG: /home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/rules_haskell/haskell/cabal.bzl:2350:10: Downloading Stack 2.3.1 ...
INFO: Repository rules_haskell_stack_update instantiated at:
  /home/circleci/workdir/WORKSPACE:125:11: in <toplevel>
  /home/circleci/workdir/src/capybara/haskell-ws.bzl:14:19: in haskell_ws
  /home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/rules_haskell/haskell/cabal.bzl:2601:10: in stack_snapshot
  /home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/bazel_tools/tools/build_defs/repo/utils.bzl:201:18: in maybe
Repository rule _stack_update defined at:
  /home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/rules_haskell/haskell/cabal.bzl:2287:32: in <toplevel>
ERROR: An error occurred during the fetch of repository 'rules_haskell_stack_update':
   Traceback (most recent call last):
	File "/home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/rules_haskell/haskell/cabal.bzl", line 2283, column 28, in _stack_update_impl
		_execute_or_fail_loudly(repository_ctx, [stack_cmd, "update"])
	File "/home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/rules_haskell/haskell/private/workspace_utils.bzl", line 27, column 13, in execute_or_fail_loudly
		fail("\n".join(["Command failed: " + " ".join(arguments), exec_result.stderr]))
Error in fail: Command failed: /home/circleci/.cache/bazel/_bazel_circleci/2c85019e61165975674d47d04cde4c93/external/rules_haskell_stack/stack update

That would explain the mirrors.json thing and also give us a way to speed up CI that we didn't think about before. Thanks!

Seems like we still use stack 2.3.1 somewhere. Could you look into this, please?

I'm going to have a look

avdv

Thank you!

avdv · 2022-06-10T11:34:29Z

@symbiont-eric-torreborre seems like the CI got stuck / wasn't triggered. Could you git commit --amend --no-edit and force push the branch again?

The default downloaded stack can be a new version but we should still be able to accept old ones

symbiont-eric-torreborre · 2022-06-10T12:42:52Z

done.

ylecornec · 2022-06-10T12:48:31Z

I think the CI does not run on PRs from other repositories, so we usually push a copy of the branch to origin to trigger it:

gh pr checkout 1751 && git push origin

symbiont-eric-torreborre · 2022-06-10T14:09:43Z

🎉

upgraded the minimum stack version to 2.7.5 to avoid

1574b2d

expiry issues on mirrors.json: haskell/hackage-server#834

symbiont-eric-torreborre changed the title ~~Upgraded the minimum stack version to 2.7.5~~ Upgraded the downloaded stack version to 2.7.5 Jun 9, 2022

avdv approved these changes Jun 10, 2022

View reviewed changes

avdv added the merge-queue merge on green CI label Jun 10, 2022

reverted the change on the minimum required version for stack

e05b545

The default downloaded stack can be a new version but we should still be able to accept old ones

symbiont-eric-torreborre force-pushed the master branch from ce688ca to e05b545 Compare June 10, 2022 12:42

Merge branch 'master' into master

b62ea9e

mergify bot merged commit b8ac6c1 into tweag:master Jun 10, 2022

mergify bot removed the merge-queue merge on green CI label Jun 10, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgraded the downloaded stack version to 2.7.5 #1751

Upgraded the downloaded stack version to 2.7.5 #1751

symbiont-eric-torreborre commented Jun 2, 2022

avdv commented Jun 9, 2022

symbiont-eric-torreborre commented Jun 9, 2022

symbiont-eric-torreborre commented Jun 9, 2022

avdv commented Jun 9, 2022

symbiont-eric-torreborre commented Jun 9, 2022

avdv left a comment

avdv commented Jun 10, 2022

symbiont-eric-torreborre commented Jun 10, 2022

ylecornec commented Jun 10, 2022

symbiont-eric-torreborre commented Jun 10, 2022

Upgraded the downloaded stack version to 2.7.5 #1751

Upgraded the downloaded stack version to 2.7.5 #1751

Conversation

symbiont-eric-torreborre commented Jun 2, 2022

avdv commented Jun 9, 2022

symbiont-eric-torreborre commented Jun 9, 2022

symbiont-eric-torreborre commented Jun 9, 2022

avdv commented Jun 9, 2022

symbiont-eric-torreborre commented Jun 9, 2022

avdv left a comment

Choose a reason for hiding this comment

avdv commented Jun 10, 2022

symbiont-eric-torreborre commented Jun 10, 2022

ylecornec commented Jun 10, 2022

symbiont-eric-torreborre commented Jun 10, 2022